UFLDL Recommended Readings

Revision as of 01:31, 19 March 2011 (view source)

Wzou (Talk | contribs)

← Older edit

Revision as of 04:39, 19 March 2011 (view source)

Wzou (Talk | contribs)

Newer edit →

Line 20:

Analyzing deep learning/why does deep learning work:

* [http://www.cs.toronto.edu/~larocheh/publications/deep-nets-icml-07.pdf] H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio. An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation. ICML 2007.

-

** (Someone read this and let us know if this is worth keeping,.)

+

** (Someone read this and let us know if this is worth keeping,. [Could serve as a easy reinforcement to other papers, but most model related material already covered by other papers, results do not seem to have strong impact])

* [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010

* [http://cs.stanford.edu/~ang/papers/nips09-MeasuringInvariancesDeepNetworks.pdf] Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee and Andrew Y. Ng. Measuring invariances in deep networks. NIPS 2009.

Line 28:

* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs.

** But ignore the Theano code examples.

-

** (Someone tell us if this should be moved later. Useful for understanding some of DL literature, but not needed for many of the later papers?)

+

** (Someone tell us if this should be moved later. Useful for understanding some of DL literature, but not needed for many of the later papers? [Agree to move to later])

Line 61:

* [http://www.cs.toronto.edu/~ranzato/publications/ranzato_aistats2010.pdf] M. Ranzato, A. Krizhevsky, G. Hinton. Factored 3-Way Restricted Boltzmann Machines for Modeling Natural Images. In AISTATS 2010.

* [http://www.cs.toronto.edu/~ranzato/publications/ranzato_cvpr2010.pdf] M. Ranzato, G. Hinton, Modeling Pixel Means and Covariances Using Factorized Third-Order Boltzmann Machines. CVPR 2010

-

** (someone and tell us if you need to read the 3-way RBM paper before the mcRBM one)

+

** (someone and tell us if you need to read the 3-way RBM paper before the mcRBM one [yes found it helpful])

* [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010.

* [http://www.nature.com/nature/journal/v457/n7225/pdf/nature07481.pdf] Y. Karklin and M. S. Lewicki, Emergence of complex cell properties by learning to generalize in natural scenes, Nature, 2008.

-

** (someone tell us if this should be here. Interesting algorithm + nice visualizations, though maybe slightly hard to understand.)

+

** (someone tell us if this should be here. Interesting algorithm + nice visualizations, though maybe slightly hard to understand. [seems a good reminder there are other existing models])

UFLDL Recommended Readings

From Ufldl

Revision as of 04:39, 19 March 2011

Views

Personal tools

ufldl resources

wiki

Search

Toolbox

@@ Line 20: / Line 20: @@
 Analyzing deep learning/why does deep learning work:
 * [http://www.cs.toronto.edu/~larocheh/publications/deep-nets-icml-07.pdf] H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio. An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation. ICML 2007.
-** (Someone read this and let us know if this is worth keeping,.)
+** (Someone read this and let us know if this is worth keeping,. [Could serve as a easy reinforcement to other papers, but most model related material already covered by other papers, results do not seem to have strong impact])
 * [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010
 * [http://cs.stanford.edu/~ang/papers/nips09-MeasuringInvariancesDeepNetworks.pdf] Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee and Andrew Y. Ng. Measuring invariances in deep networks. NIPS 2009.
@@ Line 28: / Line 28: @@
 * [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs.
 ** But ignore the Theano code examples.
-** (Someone tell us if this should be moved later.  Useful for understanding some of DL literature, but not needed for many of the later papers?)
+** (Someone tell us if this should be moved later.  Useful for understanding some of DL literature, but not needed for many of the later papers? [Agree to move to later])
@@ Line 61: / Line 61: @@
 * [http://www.cs.toronto.edu/~ranzato/publications/ranzato_aistats2010.pdf] M. Ranzato, A. Krizhevsky, G. Hinton. Factored 3-Way Restricted Boltzmann Machines for Modeling Natural Images. In AISTATS 2010.
 * [http://www.cs.toronto.edu/~ranzato/publications/ranzato_cvpr2010.pdf] M. Ranzato, G. Hinton, Modeling Pixel Means and Covariances Using Factorized Third-Order Boltzmann Machines. CVPR 2010
-** (someone and tell us if you need to read the 3-way RBM paper before the mcRBM one)
+** (someone and tell us if you need to read the 3-way RBM paper before the mcRBM one [yes found it helpful])
 * [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010.
 * [http://www.nature.com/nature/journal/v457/n7225/pdf/nature07481.pdf] Y. Karklin and M. S. Lewicki, Emergence of complex cell properties by learning to generalize in natural scenes, Nature, 2008.
-** (someone tell us if this should be here.  Interesting algorithm + nice visualizations, though maybe slightly hard to understand.)
+** (someone tell us if this should be here.  Interesting algorithm + nice visualizations, though maybe slightly hard to understand. [seems a good reminder there are other existing models])