稀疏编码自编码表达
From Ufldl
(→Batching examples into mini-batches) |
(→Good initialization of s[良好的s初始值]) |
||
Line 313: | Line 313: | ||
[原文] | [原文] | ||
- | === Good initialization of <math>s</math>[ | + | === Good initialization of <math>s</math>[良好的<math>s</math>初始值] === |
Another important trick in obtaining faster and better convergence is good initialization of the feature matrix <math>s</math> before using gradient descent (or other methods) to optimize for the objective function for <math>s</math> given <math>A</math>. In practice, initializing <math>s</math> randomly at each iteration can result in poor convergence unless a good optima is found for <math>s</math> before moving on to optimize for <math>A</math>. A better way to initialize <math>s</math> is the following: | Another important trick in obtaining faster and better convergence is good initialization of the feature matrix <math>s</math> before using gradient descent (or other methods) to optimize for the objective function for <math>s</math> given <math>A</math>. In practice, initializing <math>s</math> randomly at each iteration can result in poor convergence unless a good optima is found for <math>s</math> before moving on to optimize for <math>A</math>. A better way to initialize <math>s</math> is the following: |