反向传导算法
From Ufldl
Line 102: | Line 102: | ||
= \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | ||
= \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\ | = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\ | ||
- | &= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}( | + | &= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_j-a_j^{(n_l)})^2 |
- | = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}( | + | = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_j-f(z_j^{(n_l)}))^2 \\ |
&= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot \frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)}) | &= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot \frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)}) | ||
= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot f'(z_j^{(n_l)}) \cdot \frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\ | = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) \cdot f'(z_j^{(n_l)}) \cdot \frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\ | ||
Line 113: | Line 113: | ||
</math> | </math> | ||
- | 将上式中的<math>\textstyle n_l-1</math>与<math>\textstyle n_l</math>的关系替换为<math>\textstyle l</math>与<math>\textstyle l | + | 将上式中的<math>\textstyle n_l-1</math>与<math>\textstyle n_l</math>的关系替换为<math>\textstyle l</math>与<math>\textstyle l+1</math>的关系,就可以得到: |
: <math> | : <math> | ||
\delta^{(l)}_i = \left( \sum_{j=1}^{s_{l+1}} W^{(l)}_{ji} \delta^{(l+1)}_j \right) f'(z^{(l)}_i) | \delta^{(l)}_i = \left( \sum_{j=1}^{s_{l+1}} W^{(l)}_{ji} \delta^{(l+1)}_j \right) f'(z^{(l)}_i) |