CS 446/ECE 449: Machine Learning

Spring 2025

Course Description:
The goal of machine learning is to develop algorithms and models that enable computers to learn from data and make predictions or decisions without being explicitly programmed for a particular task. In this course, we will cover the common algorithms and models encountered in both traditional machine learning and modern deep learning, those in unsupervised learning, supervised learning, and reinforcement learning. The algorithms that we will cover include k-means, Gaussian mixture models, expectation maximization, decision trees, Naive Bayes, linear regression, logistic regression, support vector machines, kernel methods, boosting, learning theory, common neural network architectures including FCN, CNN, RNN, LSTM, Transformer, and training algorithms, basic reinforcement learning algorithms such as Q-learning and policy gradient.
Course Credit:
- Undergraduate: 3
- Graduate: 3 or 4
Prerequisites:
- Linear algebra (and calculus)
- Probability
- Basic statistics
- Programming in python (numpy and pytorch)
Class time:
Tues, Thu, 9:30am–10:45am, SC 1404
Instructor:
Prof. Tong Zhang (tozhang@illinois.edu)
- Office: SC 2118
- Office Hour: Thu 10:50Am –11:50Am
Teaching Assistants:
- Jane Du
  Email: zd16@illinois.edu
  Office Hour: Monday, 10AM – 11AM
  Office Hour Location: SC basement (near whiteboard marked with course name) or via zoom
- Yao Xiao
  Email: yaox11@illinois.edu
  Office Hour: Monday, 4PM – 5PM
  Office Hour Location: SC basement (near whiteboard marked with course name)
- Yangyi Chen
  Email: yangyic3@illinois.edu
  Office Hour: Friday, 4PM – 5PM
  Office Hour Location: SC basement (near whiteboard marked with course name)
- Avani Puranik
  Email: avanip2@illinois.edu
  Office Hour: Wednesday, 10AM-11AM
  Office Hour Location: SC basement (near whiteboard marked with course name) or via zoom
Course resources:
- website: https://courses.grainger.illinois.edu/cs446/sp2025/
- canvas: https://canvas.illinois.edu/courses/53223
Grading:
- 3-credit: best five out of six homeworks (50%) + midterm 1 (25%) + mideterm 2 (25%)
- 4-credit: six homeworks (50%) + midterm 1 (25%) + midterm 2 (25%)
Course Material:
- Lecture slides (distributed before each lecture)
- Reference books:
  Tom Mitchell. Machine learning. McGraw-Hill New York, 1997. URL https://www.cs.cmu.edu/~tom/mlbook.html
  Ethem Alpaydin. Introduction to machine learning. MIT Press, 2020. URL https://mitpress.mit.edu/9780262043793/introduction-to-machine-learning/
  Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014. URL https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html
  J Shawe-Taylor and N Christianini. Kernel Methods for Pattern Analysis. 2004. URL https://www.cambridge.org/core/books/kernel-methods-for-pattern-analysis/811462F4D6CD6A536A05127319A8935A
  Bernhard Scholkopf and Alexander J Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, 2018. URL https://direct.mit.edu/books/monograph/1821/Learning-with-KernelsSupport-Vector-Machines
  Yoshua Bengio, Ian Goodfellow, and Aaron Courville. Deep learning. MIT Press, 2017. URL https://www.deeplearningbook.org
  Christopher M Bishop and Hugh Bishop. Deep learning: Foundations and concepts. Springer Nature, 2023. URL https://www.bishopbook.com
  Kevin P. Murphy. Probabilistic Machine Learning: An introduction. MIT Press, 2022. URL http://probml.github.io/book1
  Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, and Jonathan Taylor. An introduction to statistical learning: With applications in python. Springer Nature, 2023. URL https://www.statlearning.com
Lectures:
- Introduction (one lecture)
- Unsupervised learning (three lectures on clustering and dimension reduction)
- Supervised learning (nine lectures on linear models, decision trees, boosting, kernel methods, model selection)
- Learning theory (one lecture)
- Midterm 1 (cover materials up to Midterm 1)
- Deep learning (ten lectures on common architectures and training)
- Reinforcement learning (three lectures)
- Midterm 2 (cover materials after Midterm 1)

Lecture	Date	Topic
1	Jan 21	Introduction
2	Jan 23	Nearest Neighbor and k-Means
3	Jan 28	Gaussian Mixture Models, EM
4	Jan 30	Principal Component Analysis, SVD, and Auto-encoder
	Jan 30	Homework #1 posted by midnight
5	Feb 4	Classification, Bayes Rule, and Naive Bayes
6	Feb 6	Linear Regression
7	Feb 11	Logistic Regression
	Feb 11	Homework #1 due by midnight
8	Feb 13	Support Vector Machines
	Feb 13	Homework #2 posted by midnight
9	Feb 18	Regularization and Feature Selection
10	Feb 20	Kernel Methods
11	Feb 25	Decision Trees and Random Forest
	Feb 25	Homework #2 due by midnight
12	Feb 27	Boosting
	Feb 27	Homework #3 posted by midnight
13	Mar 4	Evaluation and Model Selection
14	Mar 6	Learning Theory
15	Mar 11	Midterm 1
16	Mar 13	Two Layer Neural Networks
	Mar 14	Homework #3 due by midnight
17	Mar 25	Fully Connected Deep Neural Networks
18	Mar 27	Stochastic Optimization
	Mar 27	Homework #4 posted by midnight
19	Apr 1	Convolutional Neural Networks (I)
20	Apr 3	Convolutional Neural Networks (II)
21	Apr 8	Sequence Models, RNN, and LSTM
	Apr 8	Homework #4 due by midnight
22	Apr 10	Attention, Tokenization, and Sequence Generation
	Apr 10	Homework #5 posted by midnight
23	Apr 15	Encoder-Decoder based Representation Learning
24	Apr 17	Transformer
25	Apr 22	Transformer in Practice
	Apr 22	Homework #5 due by midnight
26	Apr 24	Reinforcement Learning
	Apr 24	Homework #6 posted by midnight
27	Apr 29	Q-Learning
28	May 1	Policy Gradient
29	May 6	Midterm 2
	May 9	Homework #6 due by midnight