Exercise:PCA in 2D

From Ufldl

Jump to: navigation, search
Line 1: Line 1:
-
In this exercise you will implement PCA and PCA whitening, as described in the notes, to generate the images displayed in the notes yourself. You will build on the starter code that has been provided to you at [http://ufldl.stanford.edu/wiki/resources/pca_2d.zip pca_2d.zip]. You need only write code at the places indicated by "YOUR CODE HERE" in the files. The only file you need to modify is <tt>pca_2d.m</tt>. Implementing this exercise will make the next exercise significantly easier to understand and complete.
+
== PCA, PCA whitening and ZCA whitening in 2D ==
 +
 
 +
In this exercise you will implement PCA, PCA whitening and ZCA whitening, as described in the earlier sections of this tutorial, and generate the images shown in the earlier sections yourself. You will build on the starter code that has been provided at [http://ufldl.stanford.edu/wiki/resources/pca_2d.zip pca_2d.zip]. You need only write code at the places indicated by "YOUR CODE HERE" in the files. The only file you need to modify is <tt>pca_2d.m</tt>. Implementing this exercise will make the next exercise significantly easier to understand and complete.
=== Step 0: Load data ===
=== Step 0: Load data ===
-
The starter code contains code to load 45 2D data points. When plotted using the scatter function, the results should look like the following
+
The starter code contains code to load 45 2D data points. When plotted using the <tt>scatter</tt> function, the results should look like the following:
[[File:raw_images_2d.png|400px|alt=Raw images|Raw images]]
[[File:raw_images_2d.png|400px|alt=Raw images|Raw images]]
Line 9: Line 11:
=== Step 1: Implement PCA ===
=== Step 1: Implement PCA ===
-
==== Step 1a: Implement PCA ====
+
In this step, you will implement PCA to obtain <math>x_{rot}</math>, the matrix in which the data is "rotated" to the basis comprising <math>\textstyle u_1, \ldots, u_n</math> made up of the principal components. As mentioned in the implementation notes, you should make use of MATLAB's <tt>svd</tt> function here.
-
In this step, you will implement PCA to obtain <math>x_{rot}</math>, the matrix in which the data is "rotated" to the basis comprising the principal components (i.e. the eigenbasis of <math>\Sigma</math>). As mentioned in the implementation notes, you should make use of MATLAB's <tt>svd</tt> function here.
+
==== Step 1a: Finding the PCA basis ====
 +
 
 +
Find <math>\textstyle u_1</math> and <math>\textstyle u_2</math>, and draw two lines in your figure to show the resulting basis on top of the given data points. You may find it useful to use MATLAB's <tt>hold on</tt> and <tt>hold off</tt> functions. (After calling <tt>hold on</tt>, plotting functions such as <tt>plot</tt> will draw the new data on top of the previously existing figure rather than erasing and replacing it; and <tt>hold off</tt> turns this off.)  You can use <tt>plot([x1,x2], [y1,y2], '-')</tt> to draw a line between <tt>(x1,y1)</tt> and <tt>(x2,y2)</tt>. Your figure should look like this:
-
Plot the resulting basis on top of the given data points. You may find it useful to use MATLAB's <tt>hold on</tt> and <tt>hold off</tt> functions.
 
[[File:pca_2d_basis.png | 400px]]
[[File:pca_2d_basis.png | 400px]]
 +
 +
If you are doing this in Matlab, you will probably get a plot that's identical to ours.  However, eigenvectors are defined only up to a sign.  I.e., instead of returning <math>\textstyle u_1</math> as the first eigenvector, Matlab/Octave could just as easily have returned <math>\textstyle -u_1</math>, and similarly instead of <math>\textstyle u_2</math> Matlab/Octave could have returned <math>\textstyle -u_2</math>.  So if you wound up with one or both of the eigenvectors pointing in a direction opposite (180 degrees difference) from what's shown above, that's okay too.
==== Step 1b: Check xRot ====
==== Step 1b: Check xRot ====
-
Use the scatter function to check that xRot looks as it should, which should be something like the following:
+
Compute <tt>xRot</tt>, and use the <tt>scatter</tt> function to check that xRot looks as it should, which should be something like the following:
[[File:pca_xrot_2d.png|360px]]
[[File:pca_xrot_2d.png|360px]]
 +
 +
Because Matlab/Octave could have returned <math>\textstyle -u_1</math> and/or <math>\textstyle -u_2</math> instead of <math>\textstyle u_1</math> and <math>\textstyle u_2</math>, it's also possible that you might have gotten a figure which is "flipped" or "reflected" along the <math>\textstyle x</math>- and/or <math>\textstyle y</math>-axis; this is also completely correct.
=== Step 2: Dimension reduce and replot ===
=== Step 2: Dimension reduce and replot ===
-
In the next step, set <math>k</math>, the number of components to retain, to be 1 (we have already done this for you). Compute the resulting <math>xHat</math> and plot the results, which should look like the following:  
+
In the next step, set <math>k</math>, the number of components to retain, to be 1 (we have already done this for you). Compute the resulting <math>xHat</math> and plot the results.  You should get the following (this figure should '''not''' be flipped along the <math>\textstyle x</math>- or <math>\textstyle y</math>-axis):  
[[File:pca_xhat_2d.png|400px]]
[[File:pca_xhat_2d.png|400px]]
=== Step 3: PCA Whitening ===
=== Step 3: PCA Whitening ===
-
Implement PCA whitening using the formula from the notes. The resulting <tt>xPCAWhite</tt> should look like the following:
+
Implement PCA whitening using the formula from the notes. Plot <tt>xPCAWhite</tt>, and verify that it looks like the following (a figure that is flipped/reflected on either/both axes is also correct):
[[File:pca_white_2d.png|400px]]
[[File:pca_white_2d.png|400px]]
=== Step 4: ZCA Whitening ===
=== Step 4: ZCA Whitening ===
-
Implement ZCA whitening and plot the results. The results should look like the following:
+
Implement ZCA whitening and plot the results. The results should look like the following (this should not be flipped/reflected along the <math>\textstyle x</math>- or <math>\textstyle y</math>-axis):
[[File:zca_white_2d.png|400px]]
[[File:zca_white_2d.png|400px]]
[[Category:Exercises]]
[[Category:Exercises]]

Revision as of 00:59, 30 April 2011

Personal tools