反向传导算法
From Ufldl
| Line 96: | Line 96: | ||
</math> | </math> | ||
| + | {译者注： | ||
| + | :<math> | ||
| + | \begin{align} | ||
| + | \delta^{(n_l-1)}_i &=\frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) | ||
| + | = \frac{\partial}{\partial z^{n_l-1}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | ||
| + | = \frac{\partial}{\partial z^{n_1-1}_i}\frac{1}{2} \sum_{j=1}^{S_{n_l}}(y_j-a_j^{(n_l)})^2 \\ | ||
| + | &= \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-a_j^{(n_l)})^2 | ||
| + | = \frac{1}{2} \sum_{j=1}^{S_{n_l}}\frac{\partial}{\partial z^{n_l-1}_i}(y_i-f(z_j^{(n_l)})^2 \\ | ||
| + | &= \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)})\frac{\partial}{\partial z_i^{(n_l-1)}}f(z_j^{(n_l)}) | ||
| + | = \sum_{j=1}^{S_{n_l}}(y_j-f(z_j^{(n_l)}) f'(z_j^{(n_l)})\frac{\partial z_j^{(n_l)}}{\partial z_i^{(n_l-1)}} \\ | ||
| + | &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)}\frac{\partial z_j^{(n_l)}}{z_i^{n_l-1}} | ||
| + | = \sum_{j=1}^{S_{n_l}} \left(\delta_j^{(n_l)}\frac{\partial}{\partial z_i^{n_l-1}}\sum_{k=1}^{S_{n_l-1}}f(z_k^{n_l-1})W_{jk}^{n_l-1}\right) \\ | ||
| + | &= \sum_{j=1}^{S_{n_l}} \delta_j^{(n_l)} W_{ji}^{n_l-1}f'(z_i^{n_l-1}) | ||
| + | = \left(\sum_{j=1}^{S_{n_l-1}}W_{ji}^{n_l-1}\delta_j^{(n_l)}\right)f'(z_i^{n_l-1}) | ||
| + | \end{align} | ||
| + | </math> | ||
| + | ] | ||
<li>计算我们需要的偏导数，计算方法如下： | <li>计算我们需要的偏导数，计算方法如下： | ||
