Exercise: Implement deep networks for digit classification
From Ufldl
m (→Step 4: Implement fine-tuning) |
(→Step 4: Implement fine-tuning) |
||
Line 46: | Line 46: | ||
To help you check that your implementation is correct, you should also check your gradients on a synthetic small dataset. We have implemented <tt>checkStackedAECost.m</tt> to help you check your gradients. If this checks passes, you will have implemented fine-tuning correctly. | To help you check that your implementation is correct, you should also check your gradients on a synthetic small dataset. We have implemented <tt>checkStackedAECost.m</tt> to help you check your gradients. If this checks passes, you will have implemented fine-tuning correctly. | ||
- | '''Note:''' When adding the weight decay term to the cost, you should regularize | + | '''Note:''' When adding the weight decay term to the cost, you should regularize only the softmax weights (do not regularize the weights that compute the hidden layer activations). |
'''Implementation Tip:''' It is always a good idea to implement the code modularly and check (the gradient of) each part of the code before writing the more complicated parts. | '''Implementation Tip:''' It is always a good idea to implement the code modularly and check (the gradient of) each part of the code before writing the more complicated parts. |