The scheme of the neuron can be made on the basis of the biological cell. Such element consists of several
inputs. The input signals are multiplied by the appropriate weights and then summed. The result is
recalculated by an activation function.
In accordance with such model, the formula of the activation potential φ
is as follows
Signal φ is processed by activation function, which can take different shapes. If the function is linear the output signal can be described as:
Neural networks described by above formula are called linear neural networks.
The other type of activation function is threshold function:
where φh is a given constant threshold value.
Functions that more accurate describe the non-linear characteristic of the biological neuron activation function are:
sigmoid function:
where β is a parameter,
and hyperbolic tangent function:
where α is a parameter.
The next picture presents the graphs of particular activation functions:
-
linear function,
threshold function,
sigmoid function.
