反向传导算法

From Ufldl

Jump to: navigation, search
Line 102: Line 102:
  = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2  
  = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2  
  = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\
  = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\
-
&= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-a_j^{(n_l)})^2
+
&= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_j-a_j^{(n_l)})^2
-
  = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-f(z_j^{(n_l)})^2 \\
+
  = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_j-f(z_j^{(n_l)}))^2 \\
&= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot \frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)})
&= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot \frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)})
  = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot  f'(z_j^{(n_l)}) \cdot \frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\
  = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot  f'(z_j^{(n_l)}) \cdot \frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\
Line 113: Line 113:
</math>  
</math>  
-
将上式中的<math>\textstyle n_l-1</math>与<math>\textstyle n_l</math>的关系替换为<math>\textstyle l</math>与<math>\textstyle l-1</math>的关系,就可以得到:
+
将上式中的<math>\textstyle n_l-1</math>与<math>\textstyle n_l</math>的关系替换为<math>\textstyle l</math>与<math>\textstyle l+1</math>的关系,就可以得到:
: <math>  
: <math>  
\delta^{(l)}_i = \left( \sum_{j=1}^{s_{l+1}} W^{(l)}_{ji} \delta^{(l+1)}_j \right) f'(z^{(l)}_i)
\delta^{(l)}_i = \left( \sum_{j=1}^{s_{l+1}} W^{(l)}_{ji} \delta^{(l+1)}_j \right) f'(z^{(l)}_i)

Revision as of 02:35, 15 April 2013

Personal tools