Exercise:Convolution and Pooling

From Ufldl

Jump to: navigation, search
Line 112: Line 112:
To <tt>C</tt>, you can then add <tt>b</tt>, the corresponding bias for the fth feature. If you had done no preprocessing of the patches, you could now apply the sigmoid function to <tt>C</tt> obtain the convolved features. However, because you preprocessed the patches before learning features on them, you must also apply the same preprocessing steps to the convolved patches to get the correct feature activations.
To <tt>C</tt>, you can then add <tt>b</tt>, the corresponding bias for the fth feature. If you had done no preprocessing of the patches, you could now apply the sigmoid function to <tt>C</tt> obtain the convolved features. However, because you preprocessed the patches before learning features on them, you must also apply the same preprocessing steps to the convolved patches to get the correct feature activations.
-
In particular, you did the following to the patches: (1) divide by 255 to normalize them into the range <math>[0, 1]</math> (2) subtract the mean patch, <tt>meanPatch</tt> to zero the mean of the patches (3) ZCA whiten using the whitening matrix <tt>ZCAWhite</tt>. These same three steps must also be applied to the convolved patches.  
+
In particular, you did the following to the patches:
 +
<ol>
 +
<li> divide by 255 to normalize them into the range <math>[0, 1]</math>
 +
<li> subtract the mean patch, <tt>meanPatch</tt> to zero the mean of the patches  
 +
<li> ZCA whiten using the whitening matrix <tt>ZCAWhite</tt>.
 +
</ol>
 +
These same three steps must also be applied to the convolved patches.  
-
Taking the preprocessing steps into account, the feature activations that you should compute is <math>\sigma(W(T(x-\bar{x})) + b)</math>, where <math>T</math> is the whitening matrix and <math>\bar(x)</math> is the mean patch. Expanding this, you obtain <math>\sigma(WTx - WT\bar{x} + b)</math>, which suggests that you should convolve the images with <tt>WT</tt> rather than <tt>W</tt> as earlier, and you should add <tt>(b - WT\bar{x})</tt>, rather than just <tt>b</tt> to the resulting matrix <tt>C</tt>, before finally applying the sigmoid function.
+
Taking the preprocessing steps into account, the feature activations that you should compute is <math>\sigma(W(T(x-\bar{x})) + b)</math>, where <math>T</math> is the whitening matrix and <math>\bar(x)</math> is the mean patch. Expanding this, you obtain <math>\sigma(WTx - WT\bar{x} + b)</math>, which suggests that you should convolve the images with <math>WT</math> rather than <math>W</math> as earlier, and you should add <math>(b - WT\bar{x})</math>, rather than just <math>b</math> to the resulting matrix <tt>C</tt>, before finally applying the sigmoid function.
==== Step 3b: Checking ====
==== Step 3b: Checking ====

Revision as of 07:54, 14 May 2011

Personal tools