Behavior of single neuron is determined by its weights vector W, behavior of the whole network – by the weights matrix W’. To assure the possibility of learning we have to add to neuron model two additional elements: weight change processor and error detector. Neuron like this is called ADALINE. Input signal y is bound with input signal X by following equation:
It's not necessary to know the exact equation describing f function, it's enough if we can point the values for each input vector
which is our desired response for output signal y.
This algorithm is known as DELTA rule. It's assumed that with each input vector X the corresponding z signal is passed to neuron. Neuron responses, on signal X, with:
y = W * X
If neuron hasn't reached its steady state, this signal is different than the desired one(y≠z). Inside the neuron exists a block for error estimation
δ = z - y
This block consists of inverter and adder. On base of error signal and input vector X it's possible to correct weights vector so that neuron could better execute given function y= f(X). New weights vector W’ is calculated with equation:
W’ = W + ηδX
where η is a learning-rate parameter.