Exercise: Implement deep networks for digit classification

From Ufldl

Jump to: navigation, search
(Step 3: Implement fine-tuning)
Line 23: Line 23:
To implement fine tuning, we need to consider all three layers as a single model. Implement <tt>stackedAECost.m</tt> to return the cost, gradient and predictions of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed using back-propogation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model.
To implement fine tuning, we need to consider all three layers as a single model. Implement <tt>stackedAECost.m</tt> to return the cost, gradient and predictions of the model. The cost function should be as defined as the log likelihood and a gradient decay term. The gradient should be computed using back-propogation as discussed earlier. The predictions should consist of the activations of the output layer of the softmax model.
-
To help you check that your implementation is correct, you can use the <tt>stackedAECheck.m</tt> script. The first part of the script script runs the same input on your combined-model function, and on your separate autoencoder and softmax functions, and checks that they return the same cost and predictions. The second part of the script checks that the numerical gradient of the function is the same as your computed analytic gradient. If these two checks pass, you will have implemented fine-tuning correctly.
+
To help you check that your implementation is correct, you can use the <tt>stackedAECheck.m</tt> script. The first part of the script runs the same input on your combined-model function, and on your separate autoencoder and softmax functions, and checks that they return the same cost and predictions. The second part of the script checks that the numerical gradient of the function is the same as your computed analytic gradient. If these two checks pass, you will have implemented fine-tuning correctly.
'''Note:''' Recall that the cost function is given by:
'''Note:''' Recall that the cost function is given by:

Revision as of 21:03, 21 April 2011

Personal tools