http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&feed=atom&action=historyExercise:Self-Taught Learning - Revision history2024-03-28T10:38:33ZRevision history for this page on the wikiMediaWiki 1.16.2http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=896&oldid=prevWatsuen at 11:02, 26 May 20112011-05-26T11:02:52Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 11:02, 26 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 60:</td>
<td colspan="2" class="diff-lineno">Line 60:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">{{STL}}</ins></div></td></tr>
</table>Watsuenhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=712&oldid=prevJngiam: /* Step 5: Classifying on the test set */2011-05-11T03:05:31Z<p><span class="autocomment">Step 5: Classifying on the test set</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 03:05, 11 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 57:</td>
<td colspan="2" class="diff-lineno">Line 57:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent. </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">''</del>As a comparison, when raw pixels are used (instead of the learned features), we obtained a test accuracy of only around 96% (for the same train and test sets).<del class="diffchange diffchange-inline">''</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>As a comparison, when <ins class="diffchange diffchange-inline">''</ins>raw pixels<ins class="diffchange diffchange-inline">'' </ins>are used (instead of the learned features), we obtained a test accuracy of only around 96% (for the same train and test sets).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td></tr>
</table>Jngiamhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=711&oldid=prevJngiam: /* Step 5: Classifying on the test set */2011-05-11T03:05:11Z<p><span class="autocomment">Step 5: Classifying on the test set</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 03:05, 11 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 55:</td>
<td colspan="2" class="diff-lineno">Line 55:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 5: Classifying on the test set===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 5: Classifying on the test set===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent. </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">''As a comparison, when raw pixels are used (instead of the learned features), we obtained a test accuracy of only around 96% (for the same train and test sets).''</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td></tr>
</table>Jngiamhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=698&oldid=prevAng: /* Step 4: Training and testing the logistic regression model */2011-05-10T23:45:28Z<p><span class="autocomment">Step 4: Training and testing the logistic regression model</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:45, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 51:</td>
<td colspan="2" class="diff-lineno">Line 51:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4: Training and testing the logistic regression model===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4: Training and testing the logistic regression model===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">In this step, you should use </del>your code from the softmax exercise (<tt>softmaxTrain.m</tt>) to train <del class="diffchange diffchange-inline">the </del>softmax classifier using the training features (<tt>trainFeatures</tt>) and labels (<tt>trainLabels</tt>).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Use </ins>your code from the softmax exercise (<tt>softmaxTrain.m</tt>) to train <ins class="diffchange diffchange-inline">a </ins>softmax classifier using the training <ins class="diffchange diffchange-inline">set </ins>features (<tt>trainFeatures</tt>) and labels (<tt>trainLabels</tt>).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 5: Classifying on the test set===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 5: Classifying on the test set===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=697&oldid=prevAng: /* Step 3: Extracting features */2011-05-10T23:45:02Z<p><span class="autocomment">Step 3: Extracting features</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:45, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 45:</td>
<td colspan="2" class="diff-lineno">Line 45:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>After the sparse autoencoder is trained, you will use it to extract features from the handwritten digit images. </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>After the sparse autoencoder is trained, you will use it to extract features from the handwritten digit images. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to <del class="diffchange diffchange-inline">activation </del>of the hidden layer for each example, i.e., the vector <math>a^{(2)}</math> corresponding to activation of layer 2. (Recall that we treat the inputs as layer 1).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to <ins class="diffchange diffchange-inline">activations </ins>of the hidden layer for each example, i.e., the vector <math>a^{(2)}</math> corresponding to activation of layer 2. (Recall that we treat the inputs as layer 1).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>After completing this step, calling <tt>feedForwardAutoencoder.m</tt> should convert the raw image data to hidden unit activations <math>a^{(2)}</math>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>After completing this step, calling <tt>feedForwardAutoencoder.m</tt> should convert the raw image data to hidden unit activations <math>a^{(2)}</math>.</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=696&oldid=prevAng: /* Step 3: Extracting features */2011-05-10T23:44:41Z<p><span class="autocomment">Step 3: Extracting features</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:44, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 43:</td>
<td colspan="2" class="diff-lineno">Line 43:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3: Extracting features===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3: Extracting features===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>After the sparse autoencoder is trained, <del class="diffchange diffchange-inline">we can </del>use it to extract features from the handwritten digit images. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>After the sparse autoencoder is trained, <ins class="diffchange diffchange-inline">you will </ins>use it to extract features from the handwritten digit images. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to activation of the hidden layer for each example i.e. the vector <math>a^{(2)}</math> corresponding to activation of layer 2 (<del class="diffchange diffchange-inline">recall </del>that we treat the inputs as layer 1).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to activation of the hidden layer for each example<ins class="diffchange diffchange-inline">, </ins>i.e.<ins class="diffchange diffchange-inline">, </ins>the vector <math>a^{(2)}</math> corresponding to activation of layer 2<ins class="diffchange diffchange-inline">. </ins>(<ins class="diffchange diffchange-inline">Recall </ins>that we treat the inputs as layer 1).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>After <del class="diffchange diffchange-inline">doing so, </del>this step <del class="diffchange diffchange-inline">will use your modified function to </del>convert the raw image data to <del class="diffchange diffchange-inline">feature </del>unit activations.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>After <ins class="diffchange diffchange-inline">completing </ins>this step<ins class="diffchange diffchange-inline">, calling <tt>feedForwardAutoencoder.m</tt> should </ins>convert the raw image data to <ins class="diffchange diffchange-inline">hidden </ins>unit activations <ins class="diffchange diffchange-inline"><math>a^{(2)}</math></ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4: Training and testing the logistic regression model===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4: Training and testing the logistic regression model===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=695&oldid=prevAng: /* Step 2: Train the sparse autoencoder */2011-05-10T23:43:06Z<p><span class="autocomment">Step 2: Train the sparse autoencoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:43, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 35:</td>
<td colspan="2" class="diff-lineno">Line 35:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Next, use the unlabeled data (digits 5<del class="diffchange diffchange-inline">-</del>9) to train a sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function as you had written in the previous exercise. (From the earlier exercise, you should have a working and vectorized implementation of the sparse autoencoder.) For us, the training step took less than 25 minutes on a fast desktop. When training is complete, you should get a visualization of pen strokes like the image shown below: </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Next, use the unlabeled data (<ins class="diffchange diffchange-inline">the </ins>digits <ins class="diffchange diffchange-inline">from </ins>5 <ins class="diffchange diffchange-inline">to </ins>9) to train a sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function as you had written in the previous exercise. (From the earlier exercise, you should have a working and vectorized implementation of the sparse autoencoder.) For us, the training step took less than 25 minutes on a fast desktop. When training is complete, you should get a visualization of pen strokes like the image shown below: </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=694&oldid=prevAng: /* Step 2: Train the sparse autoencoder */2011-05-10T23:42:49Z<p><span class="autocomment">Step 2: Train the sparse autoencoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:42, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 35:</td>
<td colspan="2" class="diff-lineno">Line 35:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Next, use the unlabeled data to train a sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function as you had written in the previous exercise. (From the earlier exercise, you should have a working and vectorized implementation of the sparse autoencoder.) For us, the training step took less than 25 minutes on a fast desktop. When training is complete, you should get a visualization of pen strokes like the image shown below: </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Next, use the unlabeled data <ins class="diffchange diffchange-inline">(digits 5-9) </ins>to train a sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function as you had written in the previous exercise. (From the earlier exercise, you should have a working and vectorized implementation of the sparse autoencoder.) For us, the training step took less than 25 minutes on a fast desktop. When training is complete, you should get a visualization of pen strokes like the image shown below: </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=693&oldid=prevAng: /* Step 2: Train the sparse autoencoder */2011-05-10T23:42:32Z<p><span class="autocomment">Step 2: Train the sparse autoencoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:42, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 35:</td>
<td colspan="2" class="diff-lineno">Line 35:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 2: Train the sparse autoencoder===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Next, <del class="diffchange diffchange-inline">we will train </del>the unlabeled <del class="diffchange diffchange-inline">dataset on the </del>sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function <del class="diffchange diffchange-inline">from </del>the previous <del class="diffchange diffchange-inline">assignments</del>. (<del class="diffchange diffchange-inline">Use </del>the <del class="diffchange diffchange-inline">frameworks from previous assignments to ensure that your code is </del>working and vectorized.) <del class="diffchange diffchange-inline">The </del>training step <del class="diffchange diffchange-inline">should take </del>less than 25 minutes <del class="diffchange diffchange-inline">(</del>on a <del class="diffchange diffchange-inline">reasonably </del>fast <del class="diffchange diffchange-inline">computer)</del>. When <del class="diffchange diffchange-inline">it </del>is <del class="diffchange diffchange-inline">completed</del>, a visualization of pen strokes <del class="diffchange diffchange-inline">should be displayed.</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Next, <ins class="diffchange diffchange-inline">use </ins>the unlabeled <ins class="diffchange diffchange-inline">data to train a </ins>sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function <ins class="diffchange diffchange-inline">as you had written in </ins>the previous <ins class="diffchange diffchange-inline">exercise</ins>. <ins class="diffchange diffchange-inline"> </ins>(<ins class="diffchange diffchange-inline">From </ins>the <ins class="diffchange diffchange-inline">earlier exercise, you should have a </ins>working and vectorized <ins class="diffchange diffchange-inline">implementation of the sparse autoencoder</ins>.) <ins class="diffchange diffchange-inline">For us, the </ins>training step <ins class="diffchange diffchange-inline">took </ins>less than 25 minutes on a fast <ins class="diffchange diffchange-inline">desktop</ins>. <ins class="diffchange diffchange-inline"> </ins>When <ins class="diffchange diffchange-inline">training </ins>is <ins class="diffchange diffchange-inline">complete</ins>, <ins class="diffchange diffchange-inline">you should get </ins>a visualization of pen strokes <ins class="diffchange diffchange-inline">like the image shown below: </ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:selfTaughtFeatures.png]]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">The </del>features learned by the sparse autoencoder should correspond to penstrokes.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Informally, the </ins>features learned by the sparse autoencoder should correspond to penstrokes.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3: Extracting features===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3: Extracting features===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Self-Taught_Learning&diff=692&oldid=prevAng: /* Overview */2011-05-10T23:36:46Z<p><span class="autocomment">Overview</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:36, 10 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 5:</td>
<td colspan="2" class="diff-lineno">Line 5:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You will be building upon your code from the earlier exercises. First, you will train your sparse autoencoder on an "unlabeled" training dataset of handwritten digits. This produces feature that are penstroke-like. We then extract these learned features from a labeled dataset of handwritten digits. These features will then be used as inputs to the softmax classifier that you wrote in the previous exercise. </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You will be building upon your code from the earlier exercises. First, you will train your sparse autoencoder on an "unlabeled" training dataset of handwritten digits. This produces feature that are penstroke-like. We then extract these learned features from a labeled dataset of handwritten digits. These features will then be used as inputs to the softmax classifier that you wrote in the previous exercise. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Concretely, for each example in the the labeled training dataset <math>\textstyle <del class="diffchange diffchange-inline">x^{(k)}</del></math>, we forward propagate the example to obtain the activation of the hidden units <math>\textstyle a^{(2)}</math>. <del class="diffchange diffchange-inline">The data is </del>now <del class="diffchange diffchange-inline">represented in term of </del><math>\textstyle a^{(2)}</math> <del class="diffchange diffchange-inline">used </del>to train the softmax classifier. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Concretely, for each example in the the labeled training dataset <math>\textstyle <ins class="diffchange diffchange-inline">x_l</ins></math>, we forward propagate the example to obtain the activation of the hidden units <math>\textstyle a^{(2)}</math>. <ins class="diffchange diffchange-inline"> We </ins>now <ins class="diffchange diffchange-inline">represent this example using </ins><math>\textstyle a^{(2)}</math> <ins class="diffchange diffchange-inline">(the "replacement" representation), and use this to as the new feature representation with which </ins>to train the softmax classifier. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Finally, we also extract the same features from the test <del class="diffchange diffchange-inline">dataset </del>to obtain predictions.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Finally, we also extract the same features from the test <ins class="diffchange diffchange-inline">data </ins>to obtain predictions.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>We will use <del class="diffchange diffchange-inline">the </del>the digits 5 to 9 as <del class="diffchange diffchange-inline">an </del>"unlabeled" dataset<del class="diffchange diffchange-inline">. while </del>the digits 0 to 4 <del class="diffchange diffchange-inline">are used as </del>the <del class="diffchange diffchange-inline">labeled training set</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">In this exercise, our goal is to distinguish between the digits from 0 to 4. </ins>We will use the digits 5 to 9 as <ins class="diffchange diffchange-inline">our </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>"unlabeled" dataset <ins class="diffchange diffchange-inline">which which to learn the features; we will then use a labeled dataset with </ins>the digits 0 to 4 <ins class="diffchange diffchange-inline">with</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">which to train </ins>the <ins class="diffchange diffchange-inline">softmax classifier</ins>. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In the starter code, we have <del class="diffchange diffchange-inline">also </del>provided '''<tt>stlExercise.m</tt>''' that will help walk you through the steps in this exercise.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In the starter code, we have provided <ins class="diffchange diffchange-inline">a file </ins>'''<tt>stlExercise.m</tt>''' that will help walk you through the steps in this exercise.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td></tr>
</table>Ang