Exercise:Independent Component Analysis
From Ufldl
(→Step 4a: Orthonormal ICA) |
(→Step 4a: Orthonormal ICA) |
||
| Line 54: | Line 54: | ||
[[File:OrthonormalICAFeatures.png | 350px]] | [[File:OrthonormalICAFeatures.png | 350px]] | ||
| - | Observe that few of the bases have been completely learned even after 10 000 iterations, highlighting a weakness of orthonormal ICA - it is difficult to optimize for the objective while enforcing the orthonormality constraint using gradient descent, and convergence can be very slow. Hence, in situations where an orthonormal basis is not required, reconstruction ICA or other faster methods of learning bases (such as [[Sparse Coding: Autoencoder Interpretation | + | Observe that few of the bases have been completely learned even after 10 000 iterations, highlighting a weakness of orthonormal ICA - it is difficult to optimize for the objective while enforcing the orthonormality constraint using gradient descent, and convergence can be very slow. Hence, in situations where an orthonormal basis is not required, reconstruction ICA or other faster methods of learning bases (such as [[Sparse Coding: Autoencoder Interpretation | sparse coding]]) may be preferable. |
==== Step 4b: Reconstruction ICA ==== | ==== Step 4b: Reconstruction ICA ==== | ||
