The simplest way to describe the function of bias is to present it’s graphical interpretation for single input neuron with two activation functions. This interpretation is shown in the figure below.

fig 1a. Neuron without bias activate function. Signum and sigmoidal function.

fig 1b. Neuron with bias activate function. Signum and sigmoidal function.

In these two figures we can see that the bias enables moving the activation threshold along the x-axis. When the bias is negative the movement is made to the right side, and when the bias is positive it the movement is made to the left side. Conclusion is that the biased neuron should learn even such input vectors that non biased neuron is not able to learn. We come to the conclusion that the additional weight cost us more calculations but it improves neuron properties. Normalization does not have any sense for single input neuron, because every normalized point could have only three different values –1, 0 or 1. Lets see normalization for biased single dimensional neuron. Realization of normalization of input vectors (the bias is the input equal to 1) and weights cause movement of all points to the edge of the circle with unitary radius. Result of that operation is shown in the figure below.

fig 2. Result of the normalize operation for single dimensional neuron.

According to the sign of bias normalization all points are moved to the adequate part of the circle, for positive bias, to the upper part of the circle and for negative bias, to the bottom one. Increase of the dimension cause that we can simply draw the line to separate points of different neuron responses. This straight line passes through the center of coordinate system and its gradient depends on w0 (bias weight). So, bias causes move of the result to the “additional dimension”, and makes solution of some problem possible to solve.