ECCV-2010 Tutorial: Feature Learning for Image Classification

Organizers

Kai Yu (NEC Laboratories America, kyu@sv.nec-labs.com),

Andrew Ng (Stanford University, ang@cs.stanford.edu)

Place & Time: Creta Maris Hotel, Crete, Greece, 9:00 – 13:00, September 5th, 2010

Course Material and Software

The quality of visual features is crucial for a wide range of computer vision topics, e.g., scene classification, object recognition, and object detection, which are very popular in recent computer vision venues. All these image classification tasks have traditionally relied on hand-crafted features to try to capture the essence of different visual patterns. Fundamentally, a long-term goal in AI research is to build intelligent systems that can automatically learn meaningful feature representations from a massive amount of image data. We believe a comprehensive coverage of the latest advances on image feature learning will be of broad interest to ECCV attendees.

The primary objective of this tutorial is to introduce a paradigm of feature learning from unlabeled images, with an emphasis on applications to supervised image classification. We provide a comprehensive coverage of recently developed algorithms for learning powerful sparse nonlinear features, and showcase their superior performance on a number of challenging image classification benchmarks, including Caltech101, PASCAL, and the recent large-scale problem ImageNet. Furthermore, we describe deep learning and a variety of deep learning algorithms, which learn rich feature hierarchies from unlabeled data and can capture complex invariance in visual patterns.

Syllabus

• Overview: Image Classification Overview

• Part I: State-of-the-art Image Classification Methods

• Discriminative Classifiers using BoW Representation and Spatial Pyramid Matching
• Alternative Methods: Generative Models and Part-based Models

• Part II: Image Classification using Sparse Coding

• Self-taught Learning
• BoW Representation from a Coding Perspective
• Feature Learning using Sparse Coding
• Alternative Sparse Coding Methods: Sparse RBM, Sparse Autoencoder, etc.

• Part III: Advanced Topics on Image Classification using Sparse Coding

• Intuitions, Topic-model View, and Geometric View
• Local Coordinate Coding: Theory and Applications
• Recent Advances in Sparse Coding for Image Classification

• Part IV: Learning Feature Hierarchies and Deep Learning

• Feature Hierarchies and the Importance of Depth
• Deep Belief Networks (DBNs) and Convolution DBNs
• Learning Invariance (ICA, SFA, etc.)
• Other Deep Architectures
• Application to Image Classification

• Open questions and discussion

Course Material and Software

The slides:

• Part 0: Introduction (by Andrew Ng)
• Part 1: State-of-the-art Image Classification Methods (by Kai Yu)
• Part 2: Image Classification using Sparse Coding (by Andrew Ng)
• Part 3: Advanced Topics on Image Classification using Sparse Coding (by Kai Yu)
• Part 4: Learning Feature Hierarchies and Deep Learning (by Andrew Ng)

Software available online:

• Matlab toolbox for sparse coding using the feature-sign algorithm [link]
• Matlab codes for image classification using sparse coding on SIFT features [link]
• Matlab codes for a fast approximation to Local Coordinate Coding [link]

Relevant Tutorials

• CVPR-2010 Tutorial on Sparse Coding and Dictionary Learning for Image Analysis, by Francis Bach (INRIA), Julien Mairal (INRIA), Jean Ponce (Ecole Normale Superieure), and Guillermo Sapiro (University of Minnesota).
• ICCV-2009 Tutorial on Recognizing and Learning Object Categories, by Li Fei-Fei (Stanford), Rob Fergus (NYU), and Antonio Torralba (MIT)

Biographies

Kai Yu is a Department Head at NEC Labs America, where he leads the research in image understanding, video surveillance, and data mining. He served as Session Chair at ICML 2009 and Area Chair at ICML 2010, and received the best paper runner-up award in PKDD-05. His team won the Winner Prizes in PASCAL VOC Challenge 2009 and the ImageNet Large-scale Visual Recognition Challenge 2010, and was among the top performers in TRECVID Video Event Detection Evaluations in 2008 and 2009. He received Ph.D in CS from University of Munich, Germany, in 2004.

Andrew Ng is an Associate Professor of Computer Science at Stanford University. His research interests include machine learning, robotics, and broad-competence AI. His group has won best paper/best student paper awards at ACL, CEAS, 3DRR and ICML. He is also a recipient of the Alfred P. Sloan Fellowship, and the IJCAI 2009 Computers and Thought award.