Training of single neuron by means of DELTA rule.

   Theory        Project        Author    



Behavior of single neuron is determined by its weights vector W, behavior of the whole network – by the weights matrix W’. To assure the possibility of learning we have to add to neuron model two additional elements: weight change processor and error detector. Neuron like this is called ADALINE. Input signal y is bound with input signal X by following equation:

y =f(X)


It's not necessary to know the exact equation describing f function, it's enough if we can point the values for each input vector

z = f(X)


which is our desired response for output signal y.


This algorithm is known as DELTA rule. It's assumed that with each input vector X the corresponding z signal is passed to neuron. Neuron responses, on signal X, with:

y = W * X


If neuron hasn't reached its steady state, this signal is different than the desired one(y≠z). Inside the neuron exists a block for error estimation

δ = z - y


This block consists of inverter and adder. On base of error signal and input vector X it's possible to correct weights vector so that neuron could better execute given function y= f(X). New weights vector W’ is calculated with equation:

W’ = W + ηδX


where η is a learning-rate parameter.


References
Ryszard Tadeusiewcz "Sieci neuronowe", Kraków 1992
mgr inæ. Adam Go³da (2005)
Electronics Department AGH