http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&feed=atom&action=historyExercise:Vectorization - Revision history2024-03-28T14:41:12ZRevision history for this page on the wikiMediaWiki 1.16.2http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=887&oldid=prevWatsuen at 11:00, 26 May 20112011-05-26T11:00:11Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 11:00, 26 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 55:</td>
<td colspan="2" class="diff-lineno">Line 55:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>As with the first problem, the autoencoder should learn edge features. Your code should run in under 10 minutes on a reasonably fast machine. If it takes significantly longer, check your code and ensure that it is vectorized.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>As with the first problem, the autoencoder should learn edge features. Your code should run in under 10 minutes on a reasonably fast machine. If it takes significantly longer, check your code and ensure that it is vectorized.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>[[Category:Exercises]] <ins class="diffchange diffchange-inline">--></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>{{Vectorized Implementation}}</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>{{Vectorized Implementation}}</div></td></tr>
</table>Watsuenhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=886&oldid=prevWatsuen at 10:57, 26 May 20112011-05-26T10:57:28Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 10:57, 26 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 56:</td>
<td colspan="2" class="diff-lineno">Line 56:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[Category:Exercises]]</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">{{Vectorized Implementation}}</ins></div></td></tr>
</table>Watsuenhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=494&oldid=prevZellyn: /* Support Code/Data */2011-04-29T20:48:27Z<p><span class="autocomment">Support Code/Data</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:48, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 7:</td>
<td colspan="2" class="diff-lineno">Line 7:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following additional files are required for this exercise:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following additional files are required for this exercise:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz MNIST Dataset (Training Images)]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz MNIST Dataset (Training Images)]</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">* [http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz MNIST Dataset (Training Labels)]</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [[Using the MNIST Dataset | Support functions for loading MNIST in Matlab ]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [[Using the MNIST Dataset | Support functions for loading MNIST in Matlab ]]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
</table>Zellynhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=487&oldid=prevAng: /* Vectorization */2011-04-29T19:40:24Z<p><span class="autocomment">Vectorization</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:40, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will vectorize your code to make it run much faster, and further adapt your sparse autoencoder to work on images of handwritten digits. <del class="diffchange diffchange-inline">Our </del>network for learning from handwritten digits will be much larger than the one <del class="diffchange diffchange-inline">we</del>'d trained on the natural images, and so using the original implementation would have been painfully slow. But with a vectorized implementation of the autoencoder, you will be able to get this to run in a reasonable amount of computation time. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will vectorize your code to make it run much faster, and further adapt your sparse autoencoder to work on images of handwritten digits. <ins class="diffchange diffchange-inline">Your </ins>network for learning from handwritten digits will be much larger than the one <ins class="diffchange diffchange-inline">you</ins>'d trained on the natural images, and so using the original implementation would have been painfully slow. But with a vectorized implementation of the autoencoder, you will be able to get this to run in a reasonable amount of computation time. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=486&oldid=prevAng: /* Vectorization */2011-04-29T19:37:07Z<p><span class="autocomment">Vectorization</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:37, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will vectorize your code to make it run much faster, and further adapt your sparse autoencoder to work on images of handwritten digits. Our network for learning from handwritten digits will be much larger than the one we'd trained on the natural images, and so using the original implementation would have been painfully slow. But with a vectorized implementation of the autoencoder, you will be able to <del class="diffchange diffchange-inline">learn interesting features from the handwritten characeters</del>. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will vectorize your code to make it run much faster, and further adapt your sparse autoencoder to work on images of handwritten digits. Our network for learning from handwritten digits will be much larger than the one we'd trained on the natural images, and so using the original implementation would have been painfully slow. But with a vectorized implementation of the autoencoder, you will be able to <ins class="diffchange diffchange-inline">get this to run in a reasonable amount of computation time</ins>. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=485&oldid=prevAng: /* Vectorization */2011-04-29T19:36:20Z<p><span class="autocomment">Vectorization</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:36, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Vectorization ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt your sparse autoencoder to work on images of handwritten digits.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will <ins class="diffchange diffchange-inline">vectorize your code to make it run much faster, and further </ins>adapt your sparse autoencoder to work on images of handwritten digits<ins class="diffchange diffchange-inline">. Our network for learning from handwritten digits will be much larger than the one we'd trained on the natural images, and so using the original implementation would have been painfully slow. But with a vectorized implementation of the autoencoder, you will be able to learn interesting features from the handwritten characeters</ins>. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Support Code/Data ===</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features for handwritten digits ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features for handwritten digits ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset. <del class="diffchange diffchange-inline">This is a large enough data set that running your older, unvectorized implementation would have been painfully slow. </del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it. After obtaining the source images, you should use [[Using the MNIST Dataset | helper functions that we provide]] to load the data into Matlab as matrices. While the helper functions that we provide will load both the input examples <math>x</math> and the class labels <math>y</math>, for this assignment, you will only need the input examples <math>x</math> since the sparse autoencoder is an ''unsupervised'' learning algorithm. (In a later assignment, we will use the labels <math>y</math> as well.) </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it. After obtaining the source images, you should use [[Using the MNIST Dataset | helper functions that we provide]] to load the data into Matlab as matrices. While the helper functions that we provide will load both the input examples <math>x</math> and the class labels <math>y</math>, for this assignment, you will only need the input examples <math>x</math> since the sparse autoencoder is an ''unsupervised'' learning algorithm. (In a later assignment, we will use the labels <math>y</math> as well.) </div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=484&oldid=prevAng: /* Step 2: Learn features for handwritten digits */2011-04-29T19:11:06Z<p><span class="autocomment">Step 2: Learn features for handwritten digits</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:11, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 30:</td>
<td colspan="2" class="diff-lineno">Line 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> patches = first 10000 images from the MNIST dataset</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> patches = first 10000 images from the MNIST dataset</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>After 400 iterations of updates using minFunc, your autoencoder should have learned features that resemble pen strokes. Our implementation takes around 15-20 minutes on a fast machine. Visualized, the features should look like <del class="diffchange diffchange-inline">in </del>the following image:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>After 400 iterations of updates using minFunc, your autoencoder should have learned features that resemble pen strokes. <ins class="diffchange diffchange-inline"> In other words, this has learned to represent handwritten characters in terms of what pen strokes appear in an image. </ins>Our implementation takes around 15-20 minutes on a fast machine. Visualized, the features should look like the following image: </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:mnistVectorizationEx.png|400px]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:mnistVectorizationEx.png|400px]]</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=483&oldid=prevAng: /* Step 2: Learn features for handwritten digits */2011-04-29T19:07:39Z<p><span class="autocomment">Step 2: Learn features for handwritten digits</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:07, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features for handwritten digits ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features for handwritten digits ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset<ins class="diffchange diffchange-inline">. This is a large enough data set that running your older, unvectorized implementation would have been painfully slow</ins>. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it. After obtaining the source images, <del class="diffchange diffchange-inline">we have </del>[[Using the MNIST Dataset | <del class="diffchange diffchange-inline">provided </del>functions ]] <del class="diffchange diffchange-inline">help you </del>load <del class="diffchange diffchange-inline">them up as </del>Matlab matrices. While the <del class="diffchange diffchange-inline">provided </del>functions <del class="diffchange diffchange-inline">allow you to </del>load <del class="diffchange diffchange-inline">up </del>both the <del class="diffchange diffchange-inline">labels </del>and <del class="diffchange diffchange-inline">data</del>, for this assignment, you will only need the <del class="diffchange diffchange-inline">data </del>since the <del class="diffchange diffchange-inline">training </del>is ''unsupervised''.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it. After obtaining the source images, <ins class="diffchange diffchange-inline">you should use </ins>[[Using the MNIST Dataset | <ins class="diffchange diffchange-inline">helper </ins>functions <ins class="diffchange diffchange-inline">that we provide</ins>]] <ins class="diffchange diffchange-inline">to </ins>load <ins class="diffchange diffchange-inline">the data into </ins>Matlab <ins class="diffchange diffchange-inline">as </ins>matrices. <ins class="diffchange diffchange-inline"> </ins>While the <ins class="diffchange diffchange-inline">helper </ins>functions <ins class="diffchange diffchange-inline">that we provide will </ins>load both the <ins class="diffchange diffchange-inline">input examples <math>x</math> </ins>and <ins class="diffchange diffchange-inline">the class labels <math>y</math></ins>, for this assignment, you will only need the <ins class="diffchange diffchange-inline">input examples <math>x</math> </ins>since the <ins class="diffchange diffchange-inline">sparse autoencoder </ins>is <ins class="diffchange diffchange-inline">an </ins>''unsupervised'' <ins class="diffchange diffchange-inline">learning algorithm. (In a later assignment, we will use the labels <math>y</math> as well</ins>.<ins class="diffchange diffchange-inline">) </ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following set of parameters worked well for us to learn good features on the MNIST dataset:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following set of parameters worked well for us to learn good features on the MNIST dataset:</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=482&oldid=prevAng: /* Step 1: Vectorize your Sparse Autoencoder Implementation */2011-04-29T19:03:08Z<p><span class="autocomment">Step 1: Vectorize your Sparse Autoencoder Implementation</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:03, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 11:</td>
<td colspan="2" class="diff-lineno">Line 11:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Vectorize your Sparse Autoencoder Implementation ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Vectorize your Sparse Autoencoder Implementation ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Using the <del class="diffchange diffchange-inline">suggestions </del>from [[Vectorization]] and [[Neural Network Vectorization]], vectorize your implementation of <tt>sparseAutoencoderCost.m</tt>. In our implementation, we were able to remove all for-loops with the use of matrix operations<del class="diffchange diffchange-inline">, </del><tt>repmat</tt> (<del class="diffchange diffchange-inline">and</del>/<del class="diffchange diffchange-inline">or </del><tt>bsxfun</tt>). A vectorized version of our code ran in under one minute on a fast computer (for learning 25 features from 10000 8x8 image patches). </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Using the <ins class="diffchange diffchange-inline">ideas </ins>from [[Vectorization]] and [[Neural Network Vectorization]], vectorize your implementation of <tt>sparseAutoencoderCost.m</tt>. In our implementation, we were able to remove all for-loops with the use of matrix operations <ins class="diffchange diffchange-inline">and </ins><tt>repmat</tt><ins class="diffchange diffchange-inline">. </ins>(<ins class="diffchange diffchange-inline">If you want to play with more advanced vectorization ideas, also type <tt>help bsxfun<</ins>/<ins class="diffchange diffchange-inline">tt>. The </ins><tt>bsxfun</tt> <ins class="diffchange diffchange-inline">function provides an alternative to <tt>repmat</tt> for some of the vectorization steps, but is not necessary for this exercise</ins>). <ins class="diffchange diffchange-inline"> </ins>A vectorized version of our <ins class="diffchange diffchange-inline">sparse autoencoder </ins>code ran in under one minute on a fast computer (for learning 25 features from 10000 8x8 image patches). </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>(Note that you do not need to vectorize the code in the other files.)</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>(Note that you do not need to vectorize the code in the other files.)</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Vectorization&diff=424&oldid=prevJngiam at 02:38, 29 April 20112011-04-29T02:38:39Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 02:38, 29 April 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 24:</td>
<td colspan="2" class="diff-lineno">Line 24:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> visibleSize = 28*28</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> visibleSize = 28*28</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"> hiddenSize = 196</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> sparsityParam = 0.1</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> sparsityParam = 0.1</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> lambda = 3e-3</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div> lambda = 3e-3</div></td></tr>
</table>Jngiam