Exercise:Vectorization
From Ufldl
(Initial commit) |
|||
| Line 1: | Line 1: | ||
| + | == Vectorization == | ||
| + | |||
In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt the sparse autoencoder to work on images of handwritten digits. You will be given a working but unvectorized implementation, and your task will be to vectorize a key step to improve its performance. | In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt the sparse autoencoder to work on images of handwritten digits. You will be given a working but unvectorized implementation, and your task will be to vectorize a key step to improve its performance. | ||
In the file <tt>vec_assign.zip</tt>, you will find MATLAB code implementing a sparse autoencoder. To run the code, you will need to download an additional data set from the [http://yann.lecun.com/exdb/mnist/|MNIST handwritten digit database]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it to the <tt>MNIST/</tt> folder in the project path. You should then be able to run <tt>train.m</tt> and obtain some preliminary results. | In the file <tt>vec_assign.zip</tt>, you will find MATLAB code implementing a sparse autoencoder. To run the code, you will need to download an additional data set from the [http://yann.lecun.com/exdb/mnist/|MNIST handwritten digit database]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it to the <tt>MNIST/</tt> folder in the project path. You should then be able to run <tt>train.m</tt> and obtain some preliminary results. | ||
| + | |||
| + | === MNIST === | ||
| + | |||
| + | sparsityParam = 0.1 | ||
| + | lambda = 3e-3 | ||
| + | beta = 3 | ||
| + | normalizeData = linear | ||
| + | |||
| + | The autoencoder should learn pen strokes as features. These features should start to become obvious after 400 iterations of minFunc, which takes around 20 - 25 minutes on the Corn cluster. Visualised, the features look as follows: | ||
| + | |||
| + | [[File:MNIST-false-0.1-3e-3-3-linear.png]] | ||
| + | |||
| + | If your parameters are improperly tuned, or if your implementation of the autoencoder is buggy, you may get one of the following images instead: | ||
| + | |||
| + | <table> | ||
| + | <tr><td>[[File:MNIST-false-0.1-3e-3-3-linear.png]]</td></tr> | ||
| + | </table> | ||
| + | |||
| + | If your image looks like one of the above images, check your code and parameters again. | ||
| + | |||
| + | === Natural images === | ||
| + | |||
| + | TODO | ||
