# 反向传导算法

 Revision as of 14:46, 9 April 2013 (view source)Kandeng (Talk | contribs)← Older edit Revision as of 14:56, 9 April 2013 (view source)Kandeng (Talk | contribs) Newer edit → Line 101: Line 101: \delta^{(n_l-1)}_i &=\frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) \delta^{(n_l-1)}_i &=\frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 - = \frac{\partial}{\partial z^{n_1-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\ + = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\ &= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-a_j^{(n_l)})^2 &= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-a_j^{(n_l)})^2 = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-f(z_j^{(n_l)})^2 \\ = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-f(z_j^{(n_l)})^2 \\ - &= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)})\frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)}) + &= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot \frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)}) - = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) f'(z_j^{(n_l)})\frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\ + = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot  f'(z_j^{(n_l)}) \cdot \frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\ - &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)}\frac{\partial z_j^{(n_l)}}{z_i^{n_l-1}} + &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)} \cdot \frac{\partial z_j^{(n_l)}}{z_i^{n_l-1}} - = \sum_{j=1}^{S_{n_l}} \left(\delta_j^{(n_l)}\frac{\partial}{\partial z_i^{n_l-1}}\sum_{k=1}^{S_{n_l-1}}f(z_k^{n_l-1})W_{jk}^{n_l-1}\right) \\ + = \sum_{j=1}^{S_{n_l}} \left(\delta_j^{(n_l)} \cdot \frac{\partial}{\partial z_i^{n_l-1}}\sum_{k=1}^{S_{n_l-1}}f(z_k^{n_l-1}) \cdot W_{jk}^{n_l-1}\right) \\ - &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)} W_{ji}^{n_l-1}f'(z_i^{n_l-1}) + &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)} \cdot  W_{ji}^{n_l-1} \cdot f'(z_i^{n_l-1}) = \left(\sum_{j=1}^{S_{n_l-1}}W_{ji}^{n_l-1}\delta_j^{(n_l)}\right)f'(z_i^{n_l-1}) = \left(\sum_{j=1}^{S_{n_l-1}}W_{ji}^{n_l-1}\delta_j^{(n_l)}\right)f'(z_i^{n_l-1}) \end{align} \end{align} [/itex] [/itex] + + 将上式中的$n_l-1$与$n_l$的关系替换为$l$与$l-1$的关系，就可以得到： + : $+ \delta^{(l)}_i = \left( \sum_{j=1}^{s_{l+1}} W^{(l)}_{ji} \delta^{(l+1)}_j \right) f'(z^{(l)}_i) +$ + 以上逐次从后向前求导的过程即为“反向传导”的本意所在。 ] ]