UFLDL Recommended Readings

From Ufldl

Jump to: navigation, search
Line 12: Line 12:
* Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010   
* Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010   
* Larochelle, Erhan, Courville, Bergstra, BBengio, 2007.  (Someone read this and let us know if this is worth keeping,.)  
* Larochelle, Erhan, Courville, Bergstra, BBengio, 2007.  (Someone read this and let us know if this is worth keeping,.)  
-
* Salakhutdinov and Hinton Science paper
+
* [http://www.cs.toronto.edu/~hinton/science.pdf] [http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html] Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006
-
* Bengio's denoising autoencoders paper from ICML  
+
* [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008.
-
*  
+
Deep Belief Networks:
 +
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples.
-
 
+
Practical Guides:
-
Practical Guides  
+
* [http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf] Geoff Hinton. A practical guide to training restricted Boltzmann machines. UTML TR 2010–003
-
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples.
+
-
* If you're implemeting RBMs, then read Hinton's "practical guide to RBMs."  But otherwise not great as a tutorial.  
+
Natural Language Processing:
Natural Language Processing:
* [http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/57] Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin, A Neural Probabilistic Language Model. JMLR 2003.
* [http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/57] Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin, A Neural Probabilistic Language Model. JMLR 2003.
* [http://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf] R. Collobert and J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. ICML 2008.
* [http://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf] R. Collobert and J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. ICML 2008.
-
* [http://www.cs.toronto.edu/~hinton/absps/threenew.pdf] Mnih, A. and Hinton, G. E.
+
* [http://www.cs.toronto.edu/~hinton/absps/threenew.pdf] Mnih, A. and Hinton, G. E. Three New Graphical Models for Statistical Language Modelling. ICML 2007
-
Three New Graphical Models for Statistical Language Modelling. ICML 2007
+
Advanced stuff:
Advanced stuff:

Revision as of 01:43, 1 March 2011

Personal tools