Self-Taught Learning to Deep Networks
From Ufldl
| Line 1: | Line 1: | ||
| - | |||
| - | |||
In the previous section, you used an autoencoder to learn features that were then fed as input | In the previous section, you used an autoencoder to learn features that were then fed as input | ||
to a softmax or logistic regression classifier. In that method, the features were learned using | to a softmax or logistic regression classifier. In that method, the features were learned using | ||
| Line 11: | Line 9: | ||
features <math>\textstyle a</math>. This is illustrated in the following diagram: | features <math>\textstyle a</math>. This is illustrated in the following diagram: | ||
| - | [[File:STL_SparseAE_Features.png| | + | [[File:STL_SparseAE_Features.png|300px]] |
We are interested in solving a classification task, where our goal is to | We are interested in solving a classification task, where our goal is to | ||
| Line 22: | Line 20: | ||
To illustrate this step, similar to [[Neural Networks|our earlier notes]], we can draw our logistic regression unit (shown in orange) as follows: | To illustrate this step, similar to [[Neural Networks|our earlier notes]], we can draw our logistic regression unit (shown in orange) as follows: | ||
| - | [[File:STL_Logistic_Classifier.png| | + | ::::[[File:STL_Logistic_Classifier.png|380px]] |
Now, consider the overall classifier (i.e., the input-output mapping) that we have learned | Now, consider the overall classifier (i.e., the input-output mapping) that we have learned | ||
| Line 37: | Line 35: | ||
as part of the sparse autoencoder training process. The second layer | as part of the sparse autoencoder training process. The second layer | ||
of weights <math>\textstyle W^{(2)}</math> mapping from the activations <math>\textstyle a</math> to the output <math>\textstyle y</math> was | of weights <math>\textstyle W^{(2)}</math> mapping from the activations <math>\textstyle a</math> to the output <math>\textstyle y</math> was | ||
| - | trained using logistic regression (or softmax regression). | + | trained using logistic regression (or softmax regression). |
But the form of our overall/final classifier is clearly just a whole big neural network. So, | But the form of our overall/final classifier is clearly just a whole big neural network. So, | ||
| Line 52: | Line 50: | ||
The effect of fine-tuning is that the labeled data can be used to modify the weights <math>W^{(1)}</math> as | The effect of fine-tuning is that the labeled data can be used to modify the weights <math>W^{(1)}</math> as | ||
well, so that adjustments can be made to the features <math>a</math> extracted by the layer | well, so that adjustments can be made to the features <math>a</math> extracted by the layer | ||
| - | of hidden units. | + | of hidden units. |
So far, we have described this process assuming that you used the "replacement" representation, where | So far, we have described this process assuming that you used the "replacement" representation, where | ||
| Line 73: | Line 71: | ||
only a relatively small labeled training set, then fine-tuning is significantly less likely to | only a relatively small labeled training set, then fine-tuning is significantly less likely to | ||
help. | help. | ||
| + | |||
| + | |||
| + | {{CNN}} | ||
| + | |||
| + | |||
| + | {{Languages|从自我学习到深层网络|中文}} | ||
