When
3:30 PM - 4:45 PM Tuesday/Thursday
Location
1320 Digital Computer Library
Professor
Trevor Walker - taw@illinois.edu
TAs
Section 498-AML
Section 498-AMO
We will start at the beginning of the textbook and proceed to the end, covering approximately one chapter per week. There are 14 substantive chapters and 15 weeks; this is to allow a little spreading out.
Read the textbook. It was written specifically for this course, AND it's free. We will split time in lecture between sketching important points described in the text and solving problems. If you haven't read the text, this might be quite puzzling.
Topics
Important: This will change as we purge typos. If you print it, you'll have to do it again. We recommend working from the electronic copy if possible.
Applied Machine Learning, D.A. Forsyth, (approximate 18'th draft)
Location: Siebel 0207
See Coursera Site for TA hours and instructions
There is a total of 10 homeworks. There will be no final exam - one homework will be designated a take-home final.
Dates and topics may be adjusted later. Tentative dates and topics are shown in italics.
| # | Due Date | Topic | Link |
|---|---|---|---|
| 1 | Mon 1/28/19, 11:59 PM CST (AML) Thur 1/31/19, 11:59 PM CST (AMO)* |
Classification w/Naive Bayes | Homework 1 |
| 2 | Wed 2/6/19, 11:59 PM CST |
Classification w/SVM | Homework 2 (Important update on 2/2) |
| 3 | Mon 2/11/19, 11:59 PM CST | Smoothing Noise w/PCA | Homework 3 |
| 4 | Mon 2/18/19, 11:59 PM CST | More PCA | Homework 4 |
| 5 | Mon 3/4/19, 11:59 PM CST | Classifying w/Variable Length Inputs | Homework 5 |
| 6 | Mon 3/11/19, 11:59 PM CST | Outlier Detection | Homework 6 |
| 7 | Mon 4/1/19, 11:59 PM CST (updated 3/14) |
Text bag-of-words Search and classification (updated 3/14) |
Homework 7 |
| 8 | Mon 4/15/19, 11:59 PM CST (updated 4/1) |
Classification with Artificial Neural Networks | Homework 8 |
| 9 | Wed 4/24/19, 11:59 PM CST | Variational Autoencoders | Homework 9 |
| 10 | Wed 5/1/19, 11:59 PM CST (updated 4/19) |
Deep Neural Networks (Take-home Final) (updated 4/19) |
Homework 10 |
* AMO students are allowed a 3-day extension for homework 1 due to the late release of Coursera materials.
We will use Gradescope to submit homework and provide grading feedback. Additional Gradescope details will be provided once we are closer to the homework deadlines.
Late homeworks will be assessed a 5% penalty per day, up to a maximum of 30%. After the homework due date is passed, the most recent homework submission will be graded and the relevant penalty will be applied.
After grading is complete, one resubmission. The resubmission will be regraded, with with a 30% late penalty, and the score will be updated. Please submit a regrade request through Gradescope if you resubmit after grading. Five days after grading is complete, homework submission will be locked and homeworks cannot be submitted or resubmitted without obtaining an explicit exception.
No resubmissions of regular homeworks will be accepted after May 1st (last day of instruction). The Take-home final may not be submitted after May 10th (last date of final examination period).
Very few exceptions will be allowed, excepting serious illness or family emergencies. Please submit a written request (i.e. email) to obtain exceptions.
Students who add the class late with have up to 5 days to complete homework 1 after their add date without late penalty for that homework. Students who add late must request the extension via email (mostly for bookkeeping purposes). Other homework will not be allowed exceptions due to late adds.
Python is the recommended language for homework, with the exception of homework 7, which must be done in R. Also, Homework 11 must be done in Python. Students may use other languages if desired for homeworks other than 7 & 11.
Students using Python may consider using a free online notebook such as Google CoLab or Microsoft Azure. These provide free notebook and computation with convenient web access.
All homeworks can be done in teams of up to 2 people. The teams may consist of members from any section of the course (online, on-campus, etc.). Both group members should contribute to the work equally. Each student must submit their homework individually, with team members explicitly specified in homework submission.
In this course, students are allowed to use library code to solve high-level problems, unless an assignment specifically requests that students implement an algorithm completely by themselves. Students should always cite any library code they have used. Students must cite online code references (documentation examples, StackOverflow, Piazza posts, Slack discussions, etc.) that they have referred to. If students collaborate to any extent, they must cite each other (name and NetID where appropriate) in their code comments and PDF reports. Students may talk to members of other groups and cite them, but they are limited to at most two students in a submission group.)
In summary, this policy reflects the University policy on plagiarism as it applies to academic writing.
Students must cite all references, including any code they have used that they did not write themselves. Failure to cite references will be considered an academic integrity violation and be pursued according to University policy, which may include receiving a failing grade on an assignment or in the entire course.
Citations do not need to follow any specific format (such as ACM style, etc.) but should mention the author's name and where the cited work can be found (including a URL, if applicable). In code, a citation can be left in a comment. The PDF report should summarize any citations.
(Updated) The policy on homeworks has not changed. There will only be 10 homeworks, and you will still be able to drop your lowest one excluding HW1 and HW10.
All homeworks will receive equal weight in the determining the final grade. In determining the final grade, the lowest scored homework between homework 2 to 9 will be dropped, and the remaining 9 homework scores will be averaged. Homework 1 and 10 will always be counted in the final grade (updated)
Once the final scores are determined, a grading scale will be determined to assign a letter grade. Historically, the score distribution has been bimodal so we won't attempt to fit a bell-curve.
| Data | Hi-res | Low-res | Topic | Additional Materials |
|---|---|---|---|---|
| 15-Jan | Video | Video | Class Overview, Classification | |
| 17-Jan | Video | Video | Classification, Evaluation, Cross-Validation, Nearst Neighbors, Naive Bayes | simple_nb.py |
| 22-Jan | Video | Video | Cross-Validation, Naive Bayes, Decision Trees | |
| 24-Jan | Video | Video | Decision Trees, Support Vector Machines | |
| 29-Jan | Video | Video | Support Vector Machines, High-Dimensional Data | sgd_demo.ipynb (In colab format) |
| 31-Jan | Video | Video | SVM Clarifications, High-Dimensional Data, Principal Component Analysis | |
| 5-Feb | Video | Video | Principal Component Analysis, NIPALS | |
| 7-Feb | Video | Video | Principal Component Analysis via SVD, Multi-Dimensional Scaling, Canonical Corrolation Analysis | |
| 12-Feb | Video | Video | Homework 2 Debugging, Homework 3 Demo, K-Means Clustering | |
| 14-Feb | Video | Video | Vector Quantization, Vector Quantization Demo, Expectation-Maximization | |
| 19-Feb | Video | Video | Gaussian Mixture Models & Expectation-Maximization, Text Processing with Bag-of-words | |
| 21-Feb | Video | Video | Text Latent Sematic Analysis and Topic Models, Linear Regression | |
| 26-Feb | Video | Video | Topic Modelling Demo, Linear Regression, Leverage, Cook's Distance | |
| 28-Feb | Video | Video | Spotting Outliers: Leverage, Cook's Distance, Standardized Residuals, Bias-Variance Tradeoff | |
| 5-Mar | Video | Video | Robust Regression, Generalized Linear Models, Logistic Regression, Deviance, Classifier Thresholds & Score Distributions, ROC & PR Curves | |
| 7-Mar | Video | Video | L1 Regularization, Kernel Regression, Markov Chains | |
| 12-Mar | Video | Video | Hidden Markov Models | Simple HMM Speech Recognition |
| 14-Mar | Video | Video | Hidden Markov Models, Intro to Conditional Random Fields. Due to glitch, the videos were recorded incorrectly. Short Rerecorded video available here. |
|
| 26-Mar | Video | Video | HMM Review, Graphical Models, Variational Inference | |
| 28-Mar | Video | Video | Variational Inference, Intro to Neural Networks | |
| 2-Apr | Video | Video | Neural Networks | |
| 4-Apr | Video | Video | Convolutional Neural Networks | |
| 9-Apr | Video | Video | Neural Networks | |
| 11-Apr | Video | Video | Recurrent Neural Networks | |
| 16-Apr | Video | Video | Neural Networks, Autoencoders | |
| 18-Apr | Video | Video | Boosting | |
| 23-Apr | Video | Video | Boosting, Reinforcement Learning | |
| 25-Apr | Video | Video | Reinforcement Learning | |
| 30-Apr | Video | Video | Recommendation Systems |
Fall 2018 Movies
| Date | AM-low-res | PM-low-res | AM-hi-res | PM-hi -res |
|---|---|---|---|---|
| 28-Aug | ||||
| 30-Aug | 30 Aug | 30 Aug | 30 Aug | 30 Aug |
| 4-Sep | 4 Sep | 4 Sep | 4 Sep | 4 Sep |
| 6-Sep | 6 Sep | 6 Sep | 6 Sep | 6 Sep |
| 11-Sep | 11 Sep | 11 Sep | 11 Sep | 11 Sep |
| 13-Sep | 13 Sep | 13 Sep | 13 Sep | 13 Sep |
| 18-Sep | 18 Sep | 18 Sep | 18 Sep | 18 Sep |
| 20-Sep | 18 Sep | 20 Sep | 18 Sep | 20 Sep |
| 25-Sep | Travel | 25 Sep | Travel | 25 Sep |
| 27-Sep | Travel | 27 Sep | Travel | 27 Sep |
| 2-Oct | Travel | 2 Oct | Travel | 2 Oct |
| 4-Oct | 4 Oct | 4 Oct | 4 Oct | 4 Oct |
| 9-Oct | 9 Oct | 9 Oct | 9 Oct | 9 Oct |
| 11-Oct | 11 Oct | 11 Oct | 11 Oct | 11 Oct |
| 16-Oct | 16 Oct | 16 Oct | 16 Oct | 16 Oct |
| 18-Oct | 18 Oct | 18 Oct | 18 Oct | 18 Oct |
| 23-Oct | 23 Oct | 23 Oct | 23 Oct | 23 Oct |
| 25-Oct | 25 Oct | 25 Oct | 25 Oct | 25 Oct |
| 30-Oct | 30 Oct | 30 Oct | 30 Oct | 30 Oct |
| 1-Nov | 1 Nov | 1 Nov | 1 Nov | 1 Nov |
| 6-Nov | Travel | 6 Nov | Travel | 6 Nov |
| 8-Nov | Travel | 8 Nov | Travel | 8 Nov |
| 13-Nov | Travel | 13 Nov | Travel | 13 Nov |
| 15-Nov | 15 Nov | 15 Nov | 15 Nov | 15 Nov |
| 27-Nov | 27 Nov | 27 Nov | 27 Nov | 27 Nov |
| 1-Dec | 1 Dec | 1 Dec | 1 Dec | 1 Dec |
| 4-Dec | 4 Dec | Use am movie | 4 Dec | Use am movie |
| 6-Dec | 6 Dec | Use am movie | 6 Dec | Use am movie |
| 1/21/19 |