COMP 551: Applied Machine Learning - Fall 2025

[expand]

[expand all]

[collapse all]

Overview

This course is a mathematically-grounded intro to the fundamentals of machine learning. By the end of the term you will be able to (i) explain the assumptions and math behind common ML models, (ii) choose and justify an approach for a new problem, and (iii) implement, evaluate, and debug models on real data. We cover supervised learning (regression, trees, neural networks), basics of unsupervised learning (clustering, PCA), generalization and model selection, regularization, and gradient-based optimization. We're committed to inclusive learning, please reach out if we can do anything to support you.
Prerequisites [click to expand]
Solid Python, plus comfort with linear algebra, calculus, and probability. If you'd like a quick self-check, see the short prerequisites quiz linked on MyCourses.

Practical advice for students

Time & pace
Plan steady weekly time for quizzes, lectures, and your project team. Consistency beats cramming and is the best way to succeed this class.
Math refresh
The class is math-heavy. Consider refreshing your mind on mathematical notations (like index notation), basic optimization "by hand" (e.g., when functions reach a max/min) and probability theory. A quick review at the beginning will make later weeks much smoother. Youtube is a great resource for that.
Linear algebra notation video
Optimization review video
Probability review video , Probability theory fundamentals document
Effective code debugging
Coding assignments are an important part of the class. If you're stuck, switch to your IDE debugger and step line-by-line, use conditional breakpoints for issues like NaNs. It's faster than sprinkling print statements.
Group workflow
Everyone should understand every part of the assignment and gets the same mark. The goal is not to divide and conquer, you should review everything together. Discussion with other teams (the Ed platform is here for that) is highly encouraged.
Course logistics
Don't miss quiz windows, they are easy points. Register groups on time each assignment if required by your section's rules.

Textbooks

• [Bishop] Pattern Recognition and Machine Learning by Christopher Bishop (2007)
• [Goodfellow] Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016)
• [Murphy] Machine Learning: A Probabilistic Perspective by Kevin Murphy (2012)
• [Murphy'22] Probabilistic Machine Learning: An Introduction, by Kevin P. Murphy (2022)
Chapters from these four books are cited as optional reference materials for the slides.
There are several other related references. [click to expand the list]
The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman (2009)
Information Theory, Inference, and Learning Algorithms, by David MacKay (2003)
Bayesian Reasoning and Machine Learning, by David Barber (2012).
Understanding Machine Learning: From Theory to Algorithms, by Shai Shalev-Shwartz and Shai Ben-David (2014)
Foundations of Machine Learning, by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2018)
Dive into Deep Learning, by Aston Zhang, Zachary Lipton, Mu Li, and Alexander J. Smola (2019)
Mathematics for Machine Learning, by Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong (2019)
A Course in Machine Learning, by Hal Daume III (2017)
Hands-on Machine Learning with Scikit-Learn and TensorFlow, by Aurelien Geron (2017)
Machine Learning, by Tom Mitchell (1997)
Introduction to Data Mining, by Pang-Ning Tan, Michael Steinbach, Anuj Karpatne, and Vipin Kumar (2020)
Machine Learning, Dynamical Systems and Control, by Steven L. Brunton and J. Nathan Kutz (2019)

Schedule

Wed., Aug. 27
Syllabus
Mon., Sept. 1
Note: Labour day, No class
Wed., Sept. 3
Introduction
Mon., Sept. 8
Maximum likelihood, Note: Add/Drop deadline
Wed., Sept. 10
Linear regression
Mon., Sept. 15
Logistic regression
Wed., Sept. 17
Mon., Sept. 22
Wed., Sept. 24
Mon., Sept. 29
Assignment 1 due on Tue., Sept. 30
Wed., Oct. 1
Mon., Oct. 6
Midterm Exam 1
Wed., Oct. 8
Mon., Oct. 13
Note: Reading week, No class
Wed., Oct. 15
Note: Reading week, No class
Mon., Oct. 20
Wed., Oct. 22
Assignment 2 due date
Mon., Oct. 27
Wed., Oct. 29
Mon., Nov. 3
Wed., Nov. 5
Mon., Nov. 10
Midterm Exam 2
Wed., Nov. 12
Mon., Nov. 17
Assignment 3 due date
Wed., Nov. 19
Mon., Nov. 24
Wed., Nov. 26
Mon., Dec. 1
Wed., Dec. 3
Assignment 4 due date

Outline

Introduction
slides , reference: 1 [Murphy22]
Parameter Estimation
slides, notebook (Colab), reference: 4 [Murphy22], 2-2.3 [Bishop], 3-3.5 [Murphy]
Linear regression
slides, notebook (Colab), reference: 7-7.3.3 [Murphy], 3-3.1.2 [Bishop]
Logistic and softmax regression
slides, notebook (Colab), reference: 8.1-8.3.3 [Murphy], 4.1-4.1.3 + 4.3-4.3.3 [Bishop]
Gradient descent methods
slides from previous year, notebook (Colab), reference: 8.3.2 [Murphy] and this overview by S. Ruder (in pdf )
Regularization
slides from previous year, notebook (Colab), reference: 3.1.4-3.3 [Bishop]
Generalization
slides from previous year, notebook for model selection (Colab), optional notebook for curse of dimensionality (Colab)
Perceptrons & multilayer perceptrons
slides from previous year, Perceptrons Colab, MLP demo, reference: 4.1.1-4.1.3 + 4.1.7 [Bishop], 6-6.5 + parts of 7 [Goodfellow]
Gradient computation and automatic differentiation
slides from previous year, notebook (Colab), reference: 6.5 + 8.2 [Goodfellow], blog post, visualization
Convolutional neural networks
slides from previous year, reference: 9 [Goodfellow], blog post, optional reading
Neural Networks for Sequences
slides from previous year, reference: 15 [Murphy'22], optional reading
Naive Bayes
slides from previous year, notebook (Colab), reference: 3.5-3.5.4 [Murphy]
Nearest neighbours
slides from previous year, notebook (Colab), reference: chapter 1 [Murphy]
Classification and regression trees
slides from previous year, notebook (Colab), reference: 16.1-16.2.6 [Murphy], 14.4 [Bishop]
Linear support vector machines
slides from previous year, notebook (Colab), reference: 4.1.1-4.1.3 + 4.1.7 + 7.1-7.1.4 excluding kernels [Bishop]
Bagging & boosting
slides from previous year, notebook (Colab), reference: 3.2 [Bishop], demos for Bias-Variance Tradeoff, Gradient Boosting explanation, and Interactive playground
Unsupervised learning
slides from previous year, notebook (Colab), reference: 25.5 [Murphy] and 9.1 [Bishop], demos for K-Means and DB-SCAN
Dimensionality reduction
slides from previous year, notebook (Colab), reference: 12.2 [Murphy], 12.1 [Bishop], demo

Evaluation

Regular Practice Quizzes [10%]
You will find them on MyCourses
One quiz per week to check the key topics discussed
There are no extensions for quizzes, the late submission does not apply
First Mid-term Exam [15%]
In person, during class on October 6th
Closed book, but you can bring 5 pages of your hand written notes
Second Mid-term Exam [25%]
In person, during class on November 10th
Closed book, but you can bring 5 pages of your hand written notes
Assignments [50%]
You will find them on MyCourses
Four programming assignments to be done in groups of three*, *no exception to this given the grading load on TAs
Groups can stay the same between projects, or change
All group members receive the same mark unless there are major complains on not contributing, responding, etc. from group-mates, which will be resolved in a case by case basis. If a significant difficulty/conflict arises, please send an email to the course email, put 'Group-Issue' in the title
Work submitted for evaluation as part of this course may be checked with text-matching software within myCourses
Bonus point [6.5%]
More bonus points can be awarded to our discretion
Late submission policy
All due dates are 11:59 pm in Montreal, unless specified otherwise [e.g. check the due dates for quizzes].
No make-up quizzes will be given.
For mini-projects, 2^k% percent will be deducted per k days of delay (no points after a week).
If you experience barriers to learning in this course, submitting the projects, etc., please do not hesitate to discuss them with me directly, and please make sure to put "551 special" in the header to make sure I see your email [for general course correspondence, please use the course email: comp551mcgill@gmail.com]. As a point of reference, you can reach the Office for Students with Disabilities at 514-398-6009.

Academic Integrity

"McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures" (see McGill's webpage for more information). (Approved by Senate on 29 January 2003)

Online Resources

Learning plan
metacademy
Video Playlists
StatQuest
FreeCodeCamp
Essence of linear algebra and Neural Networks by 3Blue1Brown
Mathematics for ML by David Rolnick
Courses with Playlist and/or Code
Introduction to Machine Learning by Google
Machine Learning by Stanford
Deep Learning by UC Berkeley
Hinton's Lectures on Neural Networks for Machine Learning
Deep Learning & Linear Algebra courses by fastai
Learning from Data by Caltech
Deep Learning (with PyTorch) playlist and course by NYU
Deep Learning by Stanford
Deep Learning by deeplearning.ai
Introduction to Deep Learning by MIT
Information Theory, Pattern Recognition, and Neural Networks by David MacKay
Books with Code
Probabilistic Machine Learning: An Introduction by Kevin Murphy (book 1)
Dive into Deep Learning BY by Aston Zhang, Zachary Lipton, Mu Li, and Alexander J. Smola
Machine Learning Notebooks for O'Reilly book Hands-on Machine Learning with Scikit-Learn and TensorFlow
Similar Courses - Graduate Level
https://www.cs.toronto.edu/~rgrosse/courses/csc2515_2019/
https://www.cs.cornell.edu/courses/cs4780/2019fa/
Similar Courses - Undergraduate Level
https://cs.mcgill.ca/~wlh/comp451/schedule.html
https://www.cs.toronto.edu/~rgrosse/courses/csc311_f20/
https://www.cs.toronto.edu/~rgrosse/courses/csc411_f18/
http://cs229.stanford.edu/syllabus-fall2020.html
https://cs230.stanford.edu/lecture/
Cheatsheets: https://stanford.edu/~shervine/teaching/
Similar Courses - Last Versions
Fall 2019
Winter 2020
Fall 2020

FAQ

• Class/waitlist is full, can I still register?
Unfortunately you will have to wait for the next semester. As an alternative, consider Fundamental of Machine Learning Course, Comp 451, and please check the list of free online courses below.
• Who to contact for department approval required for taking the course?
Please contact teresa.pian@mcgill.ca.
• Do I have the prerequisites to take the course?
This course requires strong Python programming skills and basic knowledge of probabilities, [multivariate] calculus and linear algebra. Please check this quiz to test if your background is strong enough for taking the course. It can also be used to diagnose where your background might be lacking and be used to self-study before taking the course. Most concepts covered in these questions will be used throughout the course in the slides.
• How similar is it to the last years?
Very similar, please check last year's websites to get a glimpse of the slides, expectations, etc. We will have an updated version and not exactly the same materials but very similar overall.
• Will there be lecture recordings?
Yes, but class participation is encouraged and rewarded.
• What do I learn in this course?
You will learn how the most common machine learning algorithms are designed, how they are implemented, and how to apply them in practice. This course has a heavy theory component, since it is important to understand the inner-workings of the algorithms in order to effectively utilize them in practice. Please check below for more information and note that everything below is tentative.