http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&feed=atom&action=historyExercise:Learning color features with Sparse Autoencoders - Revision history2024-03-29T12:31:57ZRevision history for this page on the wikiMediaWiki 1.16.2http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=1017&oldid=prevYuze: /* Step 1: Modify your sparse autoencoder to use a linear decoder */2011-06-21T21:00:09Z<p><span class="autocomment">Step 1: Modify your sparse autoencoder to use a linear decoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:00, 21 June 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 29:</td>
<td colspan="2" class="diff-lineno">Line 29:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Copy <tt>sparseAutoencoderCost.m</tt> to the directory for this exercise and rename it to <tt><del class="diffchange diffchange-inline">sparseAutoencoderLinear</del>.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your gradients to ensure that they are correct.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Copy <tt>sparseAutoencoderCost.m</tt> to the directory for this exercise and rename it to <tt><ins class="diffchange diffchange-inline">sparseAutoencoderLinearCost</ins>.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your gradients to ensure that they are correct.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td></tr>
</table>Yuzehttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=1005&oldid=prevCyfoo: /* Dependencies */2011-06-18T19:01:41Z<p><span class="autocomment">Dependencies</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:01, 18 June 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 9:</td>
<td colspan="2" class="diff-lineno">Line 9:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">The following additional file is required for this exercise:</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>You will need:</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the STL-10 dataset (stl10_patches_100k.zip)]</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>You will <del class="diffchange diffchange-inline">also </del>need:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* <tt>sparseAutoencoderCost.m</tt> (and related functions) from [[Exercise:Sparse Autoencoder]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* <tt>sparseAutoencoderCost.m</tt> (and related functions) from [[Exercise:Sparse Autoencoder]]</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">The following additional file is also required for this exercise:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the STL-10 dataset (stl10_patches_100k.zip)]</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>''If you have not completed the exercise listed above, we strongly suggest you complete it first.''</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>''If you have not completed the exercise listed above, we strongly suggest you complete it first.''</div></td></tr>
</table>Cyfoohttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=1004&oldid=prevCyfoo: /* Dependencies */2011-06-18T18:59:34Z<p><span class="autocomment">Dependencies</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 18:59, 18 June 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 9:</td>
<td colspan="2" class="diff-lineno">Line 9:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Dependencies ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>The following additional <del class="diffchange diffchange-inline">files are </del>required for this exercise:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>The following additional <ins class="diffchange diffchange-inline">file is </ins>required for this exercise:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the STL-10 dataset (stl10_patches_100k.zip)]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the STL-10 dataset (stl10_patches_100k.zip)]</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">* [http://ufldl.stanford.edu/wiki/resources/linear_decoder_exercise.zip Starter Code (linear_decoder_exercise.zip)]</del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You will also need:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You will also need:</div></td></tr>
</table>Cyfoohttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=996&oldid=prevJngiam: /* Step 1: Modify your sparse autoencoder to use a linear decoder */2011-06-03T19:15:14Z<p><span class="autocomment">Step 1: Modify your sparse autoencoder to use a linear decoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:15, 3 June 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 30:</td>
<td colspan="2" class="diff-lineno">Line 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Copy <tt><del class="diffchange diffchange-inline">sparseAutoencoder</del>.m</tt> to the directory for this exercise and rename it to <tt>sparseAutoencoderLinear.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your gradients to ensure that they are correct.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Copy <tt><ins class="diffchange diffchange-inline">sparseAutoencoderCost</ins>.m</tt> to the directory for this exercise and rename it to <tt>sparseAutoencoderLinear.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your gradients to ensure that they are correct.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td></tr>
</table>Jngiamhttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=926&oldid=prevAng: /* Step 2: Learn features on small patches */2011-05-27T17:38:24Z<p><span class="autocomment">Step 2: Learn features on small patches</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:38, 27 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 34:</td>
<td colspan="2" class="diff-lineno">Line 34:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>You will now use your sparse autoencoder to learn features on a set of 100 000 small 8x8 patches sampled from the larger 96x96 <del class="diffchange diffchange-inline">STL10 </del>images (The [http://www.stanford.edu/~acoates//stl10/ <del class="diffchange diffchange-inline">STL10 </del>dataset] comprises 5000 <del class="diffchange diffchange-inline">test </del>and 8000 <del class="diffchange diffchange-inline">train </del>96x96 labelled color <del class="diffchange diffchange-inline">images </del>belonging to one of ten classes: airplane, bird, car, cat, deer, dog, horse, monkey, ship, truck<del class="diffchange diffchange-inline">)</del>. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>You will now use your sparse autoencoder to learn features on a set of 100<ins class="diffchange diffchange-inline">,</ins>000 small 8x8 patches sampled from the larger 96x96 <ins class="diffchange diffchange-inline">STL-10 </ins>images (The [http://www.stanford.edu/~acoates//stl10/ <ins class="diffchange diffchange-inline">STL-10 </ins>dataset] comprises 5000 <ins class="diffchange diffchange-inline">training </ins>and 8000 <ins class="diffchange diffchange-inline">test examples, with each example being a </ins>96x96 labelled color <ins class="diffchange diffchange-inline">image </ins>belonging to one of ten classes: airplane, bird, car, cat, deer, dog, horse, monkey, ship, truck.<ins class="diffchange diffchange-inline">) </ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>The code provided in this step trains your sparse autoencoder for 400 iterations with the default parameters initialized in step 0. This should take around 45 minutes. Your sparse autoencoder should learn features which when visualized, look like edges and opponent colors, as in the figure below. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>The code provided in this step trains your sparse autoencoder for 400 iterations with the default parameters initialized in step 0. This should take around 45 minutes. Your sparse autoencoder should learn features which when visualized, look like edges and <ins class="diffchange diffchange-inline">"</ins>opponent colors,<ins class="diffchange diffchange-inline">" </ins>as in the figure below. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:CNN_Features_Good.png|480px]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>[[File:CNN_Features_Good.png|480px]]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might get one of the following <del class="diffchange diffchange-inline">images instead</del>:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might <ins class="diffchange diffchange-inline">instead </ins>get <ins class="diffchange diffchange-inline">images that look like </ins>one of the following:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><table cellpadding=5px></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><table cellpadding=5px></div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=925&oldid=prevAng: /* Step 1: Modify your sparse autoencoder to use a linear decoder */2011-05-27T17:33:05Z<p><span class="autocomment">Step 1: Modify your sparse autoencoder to use a linear decoder</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:33, 27 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 30:</td>
<td colspan="2" class="diff-lineno">Line 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Copy <tt>sparseAutoencoder.m</tt> to the directory for this exercise and rename it to <tt>sparseAutoencoderLinear.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your <del class="diffchange diffchange-inline">gradient </del>to ensure that they are correct.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Copy <tt>sparseAutoencoder.m</tt> to the directory for this exercise and rename it to <tt>sparseAutoencoderLinear.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your <ins class="diffchange diffchange-inline">gradients </ins>to ensure that they are correct.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 2: Learn features on small patches ===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=924&oldid=prevAng: /* Step 0: Initialization */2011-05-27T17:30:45Z<p><span class="autocomment">Step 0: Initialization</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:30, 27 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 26:</td>
<td colspan="2" class="diff-lineno">Line 26:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 0: Initialization ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 0: Initialization ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In this step, we initialize some parameters used in the exercise (see <del class="diffchange diffchange-inline">started </del>code for details).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In this step, we initialize some parameters used in the exercise (see <ins class="diffchange diffchange-inline">starter </ins>code for details).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=923&oldid=prevAng: /* Step 0: Initialization */2011-05-27T17:30:34Z<p><span class="autocomment">Step 0: Initialization</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:30, 27 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 26:</td>
<td colspan="2" class="diff-lineno">Line 26:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 0: Initialization ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 0: Initialization ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In this step, we initialize some parameters used in the exercise.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In this step, we initialize some parameters used in the exercise <ins class="diffchange diffchange-inline">(see started code for details)</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Step 1: Modify your sparse autoencoder to use a linear decoder ===</div></td></tr>
</table>Anghttp://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=916&oldid=prev172.24.68.131: /* Learning color features with Sparse Autoencoders */2011-05-27T01:47:41Z<p><span class="autocomment">Learning color features with Sparse Autoencoders</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 01:47, 27 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Learning color features with Sparse Autoencoders ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Learning color features with Sparse Autoencoders ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In this exercise, you will implement a [[Linear Decoders | linear decoder]] (<del class="diffchange diffchange-inline">with the </del>sparse autoencoder) to learn features on color images from the <del class="diffchange diffchange-inline">STL10 </del>dataset. These features will be used in an later [[Exercise:Convolution and Pooling | exercise on convolution and pooling]] for classifying <del class="diffchange diffchange-inline">STL10 </del>images.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In this exercise, you will implement a [[Linear Decoders | linear decoder]] (<ins class="diffchange diffchange-inline">a </ins>sparse autoencoder <ins class="diffchange diffchange-inline">whose output layer uses a linear activation function</ins>)<ins class="diffchange diffchange-inline">. You will then apply it </ins>to learn features on color images from the <ins class="diffchange diffchange-inline">STL-10 </ins>dataset. These features will be used in an later [[Exercise:Convolution and Pooling | exercise on convolution and pooling]] for classifying <ins class="diffchange diffchange-inline">STL-10 </ins>images.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>In the file <tt>[http://ufldl.stanford.edu/wiki/resources/linear_decoder_exercise.zip linear_decoder_exercise.zip]</tt> we have provided some starter code. You should write your code at the places indicated "YOUR CODE HERE" in the files.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>In the file <tt>[http://ufldl.stanford.edu/wiki/resources/linear_decoder_exercise.zip linear_decoder_exercise.zip]</tt> we have provided some starter code. You should write your code at the places indicated "YOUR CODE HERE" in the files.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 10:</td>
<td colspan="2" class="diff-lineno">Line 10:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following additional files are required for this exercise:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The following additional files are required for this exercise:</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the <del class="diffchange diffchange-inline">STL10 </del>dataset (stl10_patches_100k.zip)]</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the <ins class="diffchange diffchange-inline">STL-10 </ins>dataset (stl10_patches_100k.zip)]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/linear_decoder_exercise.zip Starter Code (linear_decoder_exercise.zip)]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* [http://ufldl.stanford.edu/wiki/resources/linear_decoder_exercise.zip Starter Code (linear_decoder_exercise.zip)]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 20:</td>
<td colspan="2" class="diff-lineno">Line 20:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Learning from color image patches ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Learning from color image patches ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>In all the exercises so far, you have been working only with grayscale images. In this exercise, you will get <del class="diffchange diffchange-inline">the opportunity </del>to work with RGB color images for the first time. </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>In all the exercises so far, you have been working only with grayscale images. In this exercise, you will get to work with RGB color images for the first time. </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Conveniently, the fact that an image has three color channels (RGB), rather than a single gray channel, presents little difficulty for the sparse autoencoder. You can just combine the intensities from all the color channels for the pixels into one long vector, as if you were working with a grayscale image with 3x the number of pixels as the original image. </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Conveniently, the fact that an image has three color channels (RGB), rather than a single gray channel, presents little difficulty for the sparse autoencoder. You can just combine the intensities from all the color channels for the pixels into one long vector, as if you were working with a grayscale image with 3x the number of pixels as the original image. </div></td></tr>
</table>172.24.68.131http://ufldl.stanford.edu/wiki/index.php?title=Exercise:Learning_color_features_with_Sparse_Autoencoders&diff=852&oldid=prevJngiam: /* Step 2: Learn features on small patches */2011-05-23T05:39:17Z<p><span class="autocomment">Step 2: Learn features on small patches</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 05:39, 23 May 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 42:</td>
<td colspan="2" class="diff-lineno">Line 42:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might get one of the following images instead:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might get one of the following images instead:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><table></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><table <ins class="diffchange diffchange-inline">cellpadding=5px</ins>></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><tr><td>[[File:cnn_Features_Bad1.png|240px]]</td><td>[[File:cnn_Features_Bad2.png|240px]]</td></tr></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><tr><td>[[File:cnn_Features_Bad1.png|240px]]</td><td>[[File:cnn_Features_Bad2.png|240px]]</td></tr></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div></table></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div></table></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The learned features will be saved to <tt>STL10Features.mat</tt>, which will be used in the later [[Exercise:Convolution and Pooling | exercise on convolution and pooling]].</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The learned features will be saved to <tt>STL10Features.mat</tt>, which will be used in the later [[Exercise:Convolution and Pooling | exercise on convolution and pooling]].</div></td></tr>
</table>Jngiam