Backpropagation Algorithm

From Ufldl

Jump to: navigation, search
m (Fixed quotes)
Line 115: Line 115:
the algorithm using matrix-vectorial notation.
the algorithm using matrix-vectorial notation.
We will use "<math>\textstyle \bullet</math>" to denote the element-wise product
We will use "<math>\textstyle \bullet</math>" to denote the element-wise product
-
operator (denoted ``<tt>.*</tt>'' in Matlab or Octave, and also called the Hadamard product),
+
operator (denoted "<tt>.*</tt>
 +
in Matlab or Octave, and also called the Hadamard product),
so
so
that if <math>\textstyle a = b \bullet c</math>, then <math>\textstyle a_i = b_ic_i</math>.
that if <math>\textstyle a = b \bullet c</math>, then <math>\textstyle a_i = b_ic_i</math>.
Line 153: Line 154:
below, <math>\textstyle \Delta W^{(l)}</math> is a matrix (of the same dimension as <math>\textstyle W^{(l)}</math>), and  
below, <math>\textstyle \Delta W^{(l)}</math> is a matrix (of the same dimension as <math>\textstyle W^{(l)}</math>), and  
<math>\textstyle \Delta b^{(l)}</math> is a vector (of the same dimension as <math>\textstyle b^{(l)}</math>).  Note that in this notation,  
<math>\textstyle \Delta b^{(l)}</math> is a vector (of the same dimension as <math>\textstyle b^{(l)}</math>).  Note that in this notation,  
-
``<math>\textstyle \Delta W^{(l)}</math>'' is a matrix, and in particular it isn't ``<math>\textstyle \Delta</math> times <math>\textstyle W^{(l)}</math>.''
+
"<math>\textstyle \Delta W^{(l)}</math>" is a matrix, and in particular it isn't "<math>\textstyle \Delta</math> times <math>\textstyle W^{(l)}</math>."
We implement one iteration of batch gradient descent as follows:
We implement one iteration of batch gradient descent as follows:

Revision as of 01:05, 23 April 2011

Personal tools