Applied Machine Learning (CS 441) – Spring 2024

  

  Instructor:  Derek Hoiem

 

  Lectures: Tues/Thurs 9:30-10:45, 1002 ECE Building

 

  Syllabus

  Lecture Recordings, ClassTranscribe

  Lecture Review Questions and Answers

  CampusWire Discussion (sign-up link/code: 7507)

  Canvas Submission

 

  Textbook: Applied Machine Learning by David Forsyth

                                                                                                           

  

   Assignments

HW 1 – Instance-based Methods (Feb 5)

HW 2 – PCA and Linear Models (Feb 19)

HW 3 – PDFs and Outliers (Mar 4)

HW 4 – Trees and MLPs (Mar 25)

HW 5 – Deep Learning and Applications (Apr 15)

Final Project (May 1)

 

 

 

 

 

 

  Class Schedule   (subject to change)

Week

Date

Topic

Link

Reading/Notes

1

Jan 16 (Tues)

Introduction

ppt ; pdf

Jupyter notebook tutorial vid ipynb cc

Numpy tutorial vid cc

Linear algebra tutorial vid  cc

 

 

Fundamentals of Learning

 

 

1

Jan 18 (Thurs)

Working with Data

ppt ; pdf

 

2

Jan 23 (Tues)

Clustering and Retrieval

ppt ; pdf ; recording

AML Ch 8

2

Jan 25 (Thurs)

K-NN Classification and Regression

ppt ; pdf

AML Ch 1.1-1.2

3

Jan 30 (Tues)

Dimensionality reduction: PCA, embeddings

ppt ; pdf

AML Ch 5, 19

3

Feb 1 (Thurs)

Linear regression, regularization

ppt ; pdf

 AML Ch 10-11

Feb 5 (Mon)

HW 1 (Instance-based Methods) due

 

 

4

Feb 6 (Tues)

Linear classifiers: logistic regression, SVM

ppt ; pdf

AML Ch 11.3, 2.1

4

Feb 8 (Thurs)

Probability and Naïve Bayes

ppt ; pdf

AML Ch 2

5

Feb 13 (Tues)

EM and Latent Variables

ppt ; pdf

AML Ch 9

5

Feb 15 (Thurs)

Density estimation: MoG, Hists, KDE

ppt ; pdf

AML Ch 9

6

Feb 19 (Mon)

HW 2 (PCA and Linear Models) due

 

 

6

Feb 20 (Tues)

Outliers and Robust Estimation

ppt ; pdf

recording  

Linear fit demo (Matlab)

6

Feb 22 (Thurs)

Decision Trees

ppt ; pdf

AML Ch 2

7

Feb 27 (Tues)

Ensembles and Random Forests

 

AML Ch 2

 

 

Deep Learning

 

 

7

Feb 29 (Thurs)

Stochastic Gradient Descent

 

AML Ch 2.1; Pegasos (Shalev-Shwartz et al. 2007)

8

Mar 4 (Mon)

HW 3 (PDFs and Outliers)

 

 

8

Mar 5 (Tues)

Principles of Learning + Review

 

 

8

Mar 7 (Thurs)

Exam 1 on PrairieLearn 9:30am to 10:30pm

 

 Covers through Feb 29

9

Mar 9-17

Spring Break (no classes)

 

 

10

Mar 19 (Tues)

MLPs and Backprop

 

AML 16

10

Mar 21 (Thurs)

CNNs and Computer Vision

 

AML Ch 17-18,

ResNet (He et al. 2016)

11

Mar 25 (Mon)

HW 4 (Trees and MLPs) due

 

 

11

Mar 26 (Tues)

Model Training and Tuning

 

 PyTorch Tutorial from CS444

11

Mar 28 (Thurs)

Words and Attention

 

Sub-word Tokenization (Sennrich et al. 2016)

Word2Vec (Mikolov et al. 2013)

Attention is all you need (Vaswani et al. 2017)

Transformer tutorial/walkthrough

12

Apr 2 (Tues)

Transformers in Language and Vision

 

BERT (Devlin et al. 2019)

ViT (Dosovitskiy et al. 2021)

Unified-IO (Lu et al. 2022)

12

Apr 4 (Thurs)

Foundation Models: CLIP and  GPT-3

 

 

 

Applications

 

 

13

Apr 9 (Tues)

Ethics and Impact of AI

 

13

Apr 11 (Thurs)

Bias in AI, Fair ML

 

 

15

Apr 15 (Mon)

HW 5 (Deep Learning and Applications) due

 

 

14

Apr 16 (Tues)

Building and Deploying ML: Guest speaker Daniel Kang

 

14

Apr 18 (Thurs)

Reinforcement Learning

 

 

15

Apr 23 (Tues)

Audio and 1D Signals

 

Audio Deep Learning

16

Apr 25 (Thurs)

Student ML Applications

 

 

16

Apr 30 (Tues)

Summary and Looking Forward

 

 

16

May 1 (Wed)

Final Project due (cannot be late)

 

 

 

May 3-10

Final Exam on PrairieLearn, date/time TBD