Exercise:Self-Taught Learning
From Ufldl
(→Support Code/Data) |
|||
Line 3: | Line 3: | ||
In this exercise, we will use the self-taught learning paradigm with the sparse autoencoder and softmax classifier to build a classifier for handwritten digits. | In this exercise, we will use the self-taught learning paradigm with the sparse autoencoder and softmax classifier to build a classifier for handwritten digits. | ||
- | + | You will be building upon your code from the earlier exercises. First, you will train your sparse autoencoder on an "unlabeled" training dataset of handwritten digits. This produces feature that are penstroke-like. We then extract these learned features from a labeled dataset of handwritten digits. These features will then be used as inputs to the softmax classifier that you wrote in the previous exercise. | |
- | + | Concretely, for each example in the the labeled training dataset, <math>\textstyle x^{(k)}</math>, we forward propagate the example to obtain the activation of the hidden units <math>\textstyle a^{(2)}</math>. The data is now represented in term of <math>\textstyle a^{(2)}</math> used to train the softmax classifier. | |
- | + | Finally, we also extract the same features from the test dataset to obtain predictions. | |
+ | |||
+ | We will use the the digits 5 to 9 as an "unlabeled" dataset. while the digits 0 to 4 are used as the labeled training set. | ||
=== Support Code/Data === | === Support Code/Data === | ||
Line 26: | Line 28: | ||
Download and decompress <tt>[http://ufldl.stanford.edu/wiki/resources/stl_exercise.zip stl_exercise.zip]</tt>, which contains starter code for this exercise. Additionally, you will need to download the datasets from the MNIST Handwritten Digit Database for this project. | Download and decompress <tt>[http://ufldl.stanford.edu/wiki/resources/stl_exercise.zip stl_exercise.zip]</tt>, which contains starter code for this exercise. Additionally, you will need to download the datasets from the MNIST Handwritten Digit Database for this project. | ||
- | |||
- | |||
===Step Two: Train the sparse autoencoder=== | ===Step Two: Train the sparse autoencoder=== | ||
- | Next we will train the | + | Next, we will train the unlabeled dataset on the sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function from the previous assignments. (Use the frameworks from previous assignments to ensure that your code is working and vectorized.) The training step should take less than 20 minutes (on a reasonably fast computer). When it is completed, a visualization of pen strokes should be displayed. |
- | + | ||
- | + | ||
[[File:selfTaughtFeatures.png]] | [[File:selfTaughtFeatures.png]] | ||
- | + | The features learned by the sparse autoencoder should correspond to penstrokes. | |
- | + | ===Step Three: Extracting features=== | |
- | + | After the sparse autoencoder is trained, we can use it to extract features from the handwritten digit images. | |
- | After doing so, this step will use your modified function to convert the raw image data to feature unit activations | + | Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to activation of the hidden layer for each example i.e. the vector <math>a^{(2)}</math> corresponding to activation of layer 2. |
+ | |||
+ | After doing so, this step will use your modified function to convert the raw image data to feature unit activations. | ||
===Step Four: Training and testing the logistic regression model=== | ===Step Four: Training and testing the logistic regression model=== | ||
- | + | In this step, you should use your code from the softmax exercise (<tt>softmaxTrain.m</tt>) to train the softmax classifier using the training features (<tt>trainFeatures</tt>) and labels (<tt>trainLabels</tt>). | |
+ | |||
+ | ===Step Five: Classifying on the test set=== | ||
+ | |||
+ | Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about 98% percent. | ||
[[Category:Exercises]] | [[Category:Exercises]] |