Exercise: Implement deep networks for digit classification
From Ufldl
(Edited overview. Previous anonymous edit was mine.) |
|||
| Line 19: | Line 19: | ||
Run the training set through the first autoencoder to obtain hidden unit activation, then train this data on the second autoencoder. Since this is just an adapted application of a standard autoencoder, no changes to your coder should be required. | Run the training set through the first autoencoder to obtain hidden unit activation, then train this data on the second autoencoder. Since this is just an adapted application of a standard autoencoder, no changes to your coder should be required. | ||
| - | === Step 3: Implement fine-tuning | + | === Step 3: Implement fine-tuning === |
To implement fine tuning, we need to consider all three layers as a single model. Implement <tt>stackedAECost.m</tt> to return the cost, gradient and predictions of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed using back-propogation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model. | To implement fine tuning, we need to consider all three layers as a single model. Implement <tt>stackedAECost.m</tt> to return the cost, gradient and predictions of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed using back-propogation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model. | ||
