I have a question of understanding about the Delta Rule:

Δwᵢ = (y – ŷ)*xᵢ

Why does x have to be multiplied again after the difference? If the input is 0, the product of w and x remains 0 anyway. Then it should not matter if the weight changes with an input of 0.

let us take the following example:

- w_i = the ith weight of the weight vector
- Δw_i = the change in weight w_i
- y = the desired output of the neuron for the learning example
- ŷ = the actual, calculated output for the learning example
- x_i = the input

Step t:

- Input (x) = 1 0 0
- Random weights = 0.1 0.1 0.1
- Scalar product –> 0.1
- Step function outputs 1 –> ŷ = 1

BUT we want as output y = 0 So the weights have to be adjusted as follows:

Step t+1:

- Δw_0 = (0 – 1) * 1 = -1
- Δw_1 = (0 – 1) * 0 = 0
- Δw_2 = (0 – 1) * 0 = 0

w_i(new) = w_i(old) + Δw_i

- New weights: -0.9 0.1 0.1
- Scalar product = -0.9
- Output (y = 0)

You can totally skip the multiplication by x, can’t you? Because at the latest when the scalar product is created, the weight which is multiplied by x = 0 is still not included. So why multiply by x in the delta w?