Self-Taught Learning to Deep Networks
From Ufldl
(→From Self-Taught Learning to Deep Networks) |
(→Discussion) |
||
| Line 65: | Line 65: | ||
work well. In contrast, by first initializing the parameters using an | work well. In contrast, by first initializing the parameters using an | ||
unsupervised feature learning/pre-training step, we can end up at much better | unsupervised feature learning/pre-training step, we can end up at much better | ||
| - | solutions. | + | solutions. (Actually, pre-training has benefits beyond just helping to |
get out of local optima. In particular, it has been shown to also have | get out of local optima. In particular, it has been shown to also have | ||
a useful "regularization" effect. (Erhan et al., 2010) But a full discussion | a useful "regularization" effect. (Erhan et al., 2010) But a full discussion | ||
| - | is beyond the scope of these notes | + | is beyond the scope of these notes) |
</ul> | </ul> | ||
