Visualizing a Trained Autoencoder
From Ufldl
(Created page with "Having trained a (sparse) autoencoder, we would now like to visualize the function learned by the algorithm, to try to understand what it has learned. Consider the case of traini...") |
|||
Line 6: | Line 6: | ||
a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | ||
\end{align}</math> | \end{align}</math> | ||
- | + | <!-- This is the activation function <math>\textstyle g(\cdot)</math> applied to an affine function of the input.!--> | |
We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | ||
parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | ||
- | the bias term for now) using a 2D image. In particular, we think of | + | the bias term for now)---using a 2D image. In particular, we think of |
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> as some non-linear feature of the input <math>\textstyle x</math>. |
We ask: | We ask: | ||
What input image <math>\textstyle x</math> would cause | What input image <math>\textstyle x</math> would cause | ||
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> to be maximally activated? |
- | + | (Less formally, what is the feature that hidden unit <math>\textstyle i</math> is looking for?) | |
For this question to have a non-trivial answer, | For this question to have a non-trivial answer, | ||
we must impose some constraints on <math>\textstyle x</math>. If we suppose that | we must impose some constraints on <math>\textstyle x</math>. If we suppose that | ||
Line 34: | Line 34: | ||
When we do this for a sparse autoencoder (trained with 100 hidden units on | When we do this for a sparse autoencoder (trained with 100 hidden units on | ||
- | 10x10 pixel inputs | + | 10x10 pixel inputs<sup>1</sup> we get the following result: |
- | + | ||
- | + | ||
- | + | ||
- | [[Image:ExampleSparseAutoencoderWeights.png|400px|center]] | + | [[Image:ExampleSparseAutoencoderWeights.png|thumb|400px|center]] |
Each square in the figure above shows the (norm bounded) input image <math>\textstyle x</math> that | Each square in the figure above shows the (norm bounded) input image <math>\textstyle x</math> that | ||
Line 50: | Line 47: | ||
as audio), this algorithm also learns useful representations/features for those | as audio), this algorithm also learns useful representations/features for those | ||
domains too. | domains too. | ||
+ | |||
+ | ---- | ||
+ | |||
+ | <sup>1</sup> ''The learned features were obtained by training on '''whitened''' natural images. Whitening is a preprocessing step which removes redundancy in the input, by causing adjacent pixels to become less correlated.'' | ||
+ | |||
+ | |||
+ | {{Sparse_Autoencoder}} | ||
+ | |||
+ | |||
+ | {{Languages|可视化自编码器训练结果|中文}} |