PCA
From Ufldl
(→Number of components to retain) |
(→What works well) |
||
Line 250: | Line 250: | ||
suppose we are training our algorithm on '''natural images''', so that <math>\textstyle x_j</math> is | suppose we are training our algorithm on '''natural images''', so that <math>\textstyle x_j</math> is | ||
the value of pixel <math>\textstyle j</math>. By "natural images," we informally mean the type of image that | the value of pixel <math>\textstyle j</math>. By "natural images," we informally mean the type of image that | ||
- | a typical animal or person might see over their lifetime. | + | a typical animal or person might see over their lifetime. |
+ | |||
+ | '''(NOTE: Usually we use | ||
images of outdoor scenes with grass, trees, etc., and cut out small (say 16x16) image | images of outdoor scenes with grass, trees, etc., and cut out small (say 16x16) image | ||
patches randomly from these to train the algorithm. But in practice most | patches randomly from these to train the algorithm. But in practice most | ||
feature learning algorithms are extremely robust to the exact type of image | feature learning algorithms are extremely robust to the exact type of image | ||
it is trained on, so most images taken with a normal camera, so long as they | it is trained on, so most images taken with a normal camera, so long as they | ||
- | aren't excessively blurry or have strange artifacts, should work. | + | aren't excessively blurry or have strange artifacts, should work.)''' |
+ | |||
In this case, it makes little sense to estimate a separate mean and | In this case, it makes little sense to estimate a separate mean and | ||
variance for each pixel, because the statistics in one part | variance for each pixel, because the statistics in one part | ||
Line 287: | Line 290: | ||
a 16x16 image patch (<math>\textstyle n=256</math>), we might normalize the intensity of each image | a 16x16 image patch (<math>\textstyle n=256</math>), we might normalize the intensity of each image | ||
<math>\textstyle x^{(i)}</math> as follows: | <math>\textstyle x^{(i)}</math> as follows: | ||
- | + | ||
- | \mu^{(i)} &:= \frac{1}{n} \sum_{j=1}^n x^{(i)}_j | + | <math>\mu^{(i)} &:= \frac{1}{n} \sum_{j=1}^n x^{(i)}_j</math> |
- | x^{(i)}_j &:= x^{(i)}_j - \mu^{(i)} \;\;\;\;\hbox | + | |
- | + | <math>x^{(i)}_j &:= x^{(i)}_j - \mu^{(i)} \;\;\;\;\hbox</math>, for all <math>\textstyle j</math> | |
+ | |||
Note that the two steps above are done separately for each image <math>\textstyle x^{(i)}</math>, | Note that the two steps above are done separately for each image <math>\textstyle x^{(i)}</math>, | ||
and that <math>\textstyle \mu^{(i)}</math> here is the mean intensity of the image <math>\textstyle x^{(i)}</math>. In particular, | and that <math>\textstyle \mu^{(i)}</math> here is the mean intensity of the image <math>\textstyle x^{(i)}</math>. In particular, | ||
- | this is not the same thing as estimating a mean value separately for each pixel <math>\textstyle x_j</math>. | + | this is not the same thing as estimating a mean value separately for each pixel <math>\textstyle x_j</math>. |
== Non-natural images == | == Non-natural images == |