COMP 790-125 (Spring 2017) — Advanced Machine Learning

Modern techniques in machine learning


Time: Tuesday, Thursday 12:30-1:45
Place: FB 007
Prerequisites: Linear algebra, Probability or Statistics, programming (Matlab/R/Python)
Instructor: Vladimir Jojic (
Office hours: SN 319 Tuesdays and Thursdays 2pm-3pm


Rapid accumulation of diverse types of data necessitates innovation in data analysis. Machine learning is a growing field that has found numerous applications ranging from computer vision, speech, and medicine.


The course aims to engage you in solving problems using machine learning. In order to achieve this, the course will consist of three components

  1. Lectures covering ML and applications
  2. Homework assignments
  3. Student project or a written survey


Topics covered

Machine Learning

  1. Generative and discriminative models
  2. Fully Bayesian approaches: Gaussian Processes
  3. Graphical models: inference and learning
  4. Approximate inference: Variational and MCMC
  5. Deep learning


There is no textbook for this course, but you may find following helpful:
  1. "Machine Learning: A Probabilistic Perspective." Kevin P. Murphy
  2. "Pattern Recognition and Machine Learning," Chris M. Bishop
  3. "Probabilistic Graphical Models," Daphne Koller and Nir Friedman
  4. "The Elements of Statistical Learning: Data Mining, Inference, and Prediction," T. Hastie, R. Tibshirani, J. Friedman, download
  5. "Information Theory, Inference, and Learning Algorithms," David MacKay, download
  6. "Deep Learning," Y. Bengio, I. J. Goodfellow, A. Courville download



Date Topic Slides Code/HW
1/12 Organizational Introduction [PDF] Example project proposal [Zip]
1/17 Review Optimization [PDF]
1/19 Review Optimization and penalized linear regression [PDF]
1/24 Review Multiclass classification, learning representations [PDF]
1/26 Review Information Theory review, Expectation Maximization EM for MoG [Zip]
1/31 Graphical models Bayes Nets, Markov Random Fields, Factor graphs
2/9 Graphical models Hidden Markov Models HMM code [Zip]
2/14 Graphical models Variational EM
2/16 Graphical models Approximate inference and optimization
Graphical models Approximate inference Markov Chain Monte Carlo Gibbs code [Zip]
2/21 Graphical models Approximate inference Markov Chain Monte Carlo Homework Assignment #1 (updated 2/26) [Zip]
2/23 Graphical models Gaussian Processes
2/28 Feed-forward neural networks Intro to feed forward networks
3/2 Feed-forward neural networks Theano Tutorial (Tianxiang Gao)
3/7 Feed-forward neural networks Autoencoders
3/21 Feed-forward neural networks Variational AutoEncoders
3/23 Feed-forward neural networks Generative Adversarial Networks
3/28 Feed-forward neural networks Practical tricks for training deep nets Homework Assignment #2 [Zip]
3/30 Feed-forward neural networks Optimization algorithms
4/4 Feed-forward neural networks Residual networks and related architectures
4/11 Representations Random projections, count matrix factorization, skip-gram models Homework Assignment #3 [Zip]
4/13 Recurrent neural networks Recurrent neural networks (introduction)
4/18 Recurrent neural networks Recurrent neural networks (applications)
4/20 Reinforcement learning Basics of reinforcement learning
4/20 Reinforcement learning Policy gradient learning