Gradient checking and advanced optimization

From Ufldl

Jump to: navigation, search
m
Line 1: Line 1:
Backpropagation is a notoriously difficult algorithm to debug and get right,
Backpropagation is a notoriously difficult algorithm to debug and get right,
-
especially since many subtly buggy implementations of it---for example, one
+
especially since many subtly buggy implementations of it—for example, one
that has an off-by-one error in the indices and that thus only trains some of
that has an off-by-one error in the indices and that thus only trains some of
-
the layers of weights, or an implementation that omits the bias term---will
+
the layers of weights, or an implementation that omits the bias term—will
manage to learn something that can look surprisingly reasonable
manage to learn something that can look surprisingly reasonable
(while performing less well than a correct implementation).  Thus, even with a
(while performing less well than a correct implementation).  Thus, even with a

Revision as of 20:46, 23 April 2011

Personal tools