反向传导算法
From Ufldl
| Line 139: | Line 139: | ||
</math> | </math> | ||
[译者注:由于原作者简化了推导过程，会影响理解，我将推导过程补全为以下公式： | [译者注:由于原作者简化了推导过程，会影响理解，我将推导过程补全为以下公式： | ||
| - | + | :<math> | |
\delta^{(n_l)}_i = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y) | \delta^{(n_l)}_i = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y) | ||
= \frac{\partial}{\partial z^{n_l}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | = \frac{\partial}{\partial z^{n_l}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | ||
| Line 150: | Line 150: | ||
</math> | </math> | ||
[译者注：由于原作者简化了推导过程，使我本人看着十分费解，于是就自己推导了一遍，将过程写在这里： | [译者注：由于原作者简化了推导过程，使我本人看着十分费解，于是就自己推导了一遍，将过程写在这里： | ||
| - | + | :<math> | |
\delta^{(n_{l-1})}_i = \frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y)\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} | \delta^{(n_{l-1})}_i = \frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y)\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} | ||
</math> | </math> | ||
| - | + | ||
| + | :<math> | ||
= \delta^{(n_l)}_i\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} = \delta^{(n_l)}_i\cdot\frac{\partial}{\partial z^{n_{l-1}}_i}\sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} f(z^{n_l-1}_i) | = \delta^{(n_l)}_i\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} = \delta^{(n_l)}_i\cdot\frac{\partial}{\partial z^{n_{l-1}}_i}\sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} f(z^{n_l-1}_i) | ||
= \left( \sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} \delta^{(n_l)}_i \right) f(z^{n_l-1}_i) | = \left( \sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} \delta^{(n_l)}_i \right) f(z^{n_l-1}_i) | ||
