Whitening

From Ufldl

Jump to: navigation, search
(ZCA Whitening)
(Regularizaton)
Line 92: Line 92:
When implementing PCA whitening or ZCA whitening in practice, sometimes some
When implementing PCA whitening or ZCA whitening in practice, sometimes some
of the eigenvalues <math>\textstyle \lambda_i</math> will be numerically close to 0, and thus the scaling
of the eigenvalues <math>\textstyle \lambda_i</math> will be numerically close to 0, and thus the scaling
-
step in the scaling equation from the 2D example above would involve dividing by a value close to zero, and may cause
+
step where we divide by <math>\sqrt{\lambda_i}</math> would involve dividing by a value close to zero; this
-
the data to blow up (take on large values) or otherwise be numerically unstable.  In practice, we  
+
may cause the data to blow up (take on large values) or otherwise be numerically unstable.  In practice, we  
-
implement the scaling step using  
+
therefore implement this scaling step using  
a small amount of regularization, and add a small constant <math>\textstyle \epsilon</math>  
a small amount of regularization, and add a small constant <math>\textstyle \epsilon</math>  
to the eigenvalues before taking their square root and inverse:
to the eigenvalues before taking their square root and inverse:
Line 101: Line 101:
\end{align}</math>
\end{align}</math>
When <math>\textstyle x</math> takes values around <math>\textstyle [-1,1]</math>, a value of <math>\textstyle \epsilon \approx 10^{-5}</math>
When <math>\textstyle x</math> takes values around <math>\textstyle [-1,1]</math>, a value of <math>\textstyle \epsilon \approx 10^{-5}</math>
-
might be typical. <!-- With this form of regularization, the features won't all -->
+
might be typical.  
For the case of images, adding <math>\textstyle \epsilon</math> here also has the effect of slightly smoothing (or low-pass
For the case of images, adding <math>\textstyle \epsilon</math> here also has the effect of slightly smoothing (or low-pass

Revision as of 23:40, 29 April 2011

Personal tools