PCA

From Ufldl

Jump to: navigation, search
(Number of components to retain)
(What works well)
Line 250: Line 250:
suppose we are training our algorithm on '''natural images''', so that <math>\textstyle x_j</math> is
suppose we are training our algorithm on '''natural images''', so that <math>\textstyle x_j</math> is
the value of pixel <math>\textstyle j</math>.  By "natural images," we informally mean the type of image that
the value of pixel <math>\textstyle j</math>.  By "natural images," we informally mean the type of image that
-
a typical animal or person might see over their lifetime.\footnote{Usually we use
+
a typical animal or person might see over their lifetime.
 +
 
 +
'''(NOTE: Usually we use
images of outdoor scenes with grass, trees, etc., and cut out small (say 16x16) image
images of outdoor scenes with grass, trees, etc., and cut out small (say 16x16) image
patches randomly from these to train the algorithm.  But in practice most
patches randomly from these to train the algorithm.  But in practice most
feature learning algorithms are extremely robust to the exact type of image  
feature learning algorithms are extremely robust to the exact type of image  
it is trained on, so most images taken with a normal camera, so long as they
it is trained on, so most images taken with a normal camera, so long as they
-
aren't excessively blurry or have strange artifacts, should work.}
+
aren't excessively blurry or have strange artifacts, should work.)'''
 +
 
In this case, it makes little sense to estimate a separate mean and
In this case, it makes little sense to estimate a separate mean and
variance for each pixel, because the statistics in one part
variance for each pixel, because the statistics in one part
Line 287: Line 290:
a 16x16 image patch (<math>\textstyle n=256</math>), we might normalize the intensity of each image
a 16x16 image patch (<math>\textstyle n=256</math>), we might normalize the intensity of each image
<math>\textstyle x^{(i)}</math> as follows:  
<math>\textstyle x^{(i)}</math> as follows:  
-
\begin{align}
+
 
-
\mu^{(i)} &:= \frac{1}{n} \sum_{j=1}^n x^{(i)}_j \\
+
<math>\mu^{(i)} &:= \frac{1}{n} \sum_{j=1}^n x^{(i)}_j</math>
-
x^{(i)}_j &:= x^{(i)}_j - \mu^{(i)}  \;\;\;\;\hbox{for all <math>\textstyle j</math>
+
 
-
\end{align}
+
<math>x^{(i)}_j &:= x^{(i)}_j - \mu^{(i)}  \;\;\;\;\hbox</math>, for all <math>\textstyle j</math>
 +
 
Note that the two steps above are done separately for each image <math>\textstyle x^{(i)}</math>,
Note that the two steps above are done separately for each image <math>\textstyle x^{(i)}</math>,
and that <math>\textstyle \mu^{(i)}</math> here is the mean intensity of the image <math>\textstyle x^{(i)}</math>.  In particular,
and that <math>\textstyle \mu^{(i)}</math> here is the mean intensity of the image <math>\textstyle x^{(i)}</math>.  In particular,
-
this is not the same thing as estimating a mean value separately for each pixel <math>\textstyle x_j</math>.  
+
this is not the same thing as estimating a mean value separately for each pixel <math>\textstyle x_j</math>.
== Non-natural images ==
== Non-natural images ==

Revision as of 19:46, 4 April 2011

Personal tools