Zero weights?

[hdon]

From here:

https://github.com/llSourcell/How_to_use_Tensorflow_for_classification-LIVE/blob/master/demo.ipynb

This code:

W = tf.Variable(tf.zeros([2, 2]))           # Maintain a 2 x 2 float matrix for the weights that we'll keep updating 

Won't GradientDescentOptimizer train all the weights the same if all they are initialized all the same?

[JacobChrist]

I think the weights get updated with each (back propagation) iteration. These initial values maybe bad choices but clearly they are getting updated because the cost is going down during training.

Training step: 0000 cost= 0.114958666
Training step: 0050 cost= 0.109539941

Also, you can see at the end of the training that the 2x2 W matrix is non-zero:

Optimization Finished!
Training cost= 0.109537 W= [[ 2.14149564e-04 -2.14149914e-04]
[ 5.12748193e-05 -5.12747974e-05]] b= [ 1.19155184e-05 -1.19155284e-05]

[hdon]

@JacobChrist you definitely appear to be correct. I am clearly missing something here...