Exercise:Sparse Coding

From Ufldl

Jump to: navigation, search
(Step 2: Implement and check sparse coding cost functions)
(Step 3: Iterative optimization)
Line 40: Line 40:
=== Step 3: Iterative optimization ===
=== Step 3: Iterative optimization ===
-
In this step, you will iteratively optimize for the weights and features to learn a basis for the data. However, you need to first fill in the analytic solution to the the optimization problem with respect to the weight matrix, given the feature matrix. Once that is done, you should check that your solution is correct using the given checking code, which checks that the gradient at the point determined by your analytic solution is close to 0. Once your solution has been verified, comment out the checking code, and run the iterative optimization code. 200 iterations should take less than 45 minutes to run, and by 100 iterations you should be able to see bases that look like edges, similar to those you learned in [[Exercise:Sparse Autoencoder | the sparse autoencoder exercise]].  
+
In this step, you will iteratively optimize for the weights and features to learn a basis for the data, as described in the section on [[Sparse Coding: Autoencoder Interpretation | sparse coding ]]. Mini-batching and initialization of the features <math>s</math> has already been done for you. However, you need to still need to fill in the analytic solution to the the optimization problem with respect to the weight matrix, given the feature matrix.  
 +
 
 +
Once that is done, you should check that your solution is correct using the given checking code, which checks that the gradient at the point determined by your analytic solution is close to 0. Once your solution has been verified, comment out the checking code, and run the iterative optimization code. 200 iterations should take less than 45 minutes to run, and by 100 iterations you should be able to see bases that look like edges, similar to those you learned in [[Exercise:Sparse Autoencoder | the sparse autoencoder exercise]].  
For the non-topographic case, these features will not be "ordered", and will look something like the following:
For the non-topographic case, these features will not be "ordered", and will look something like the following:

Revision as of 06:54, 29 May 2011

Personal tools