CS498 - Machine Perception

Date	Lecture	Readings	Logistics
Module 1: Introduction and Background
1/17	Lecture #1 (Shenlong): Introduction to Robot Perception [ slides \| video \| notes ]	[Tutorial] Stanford CS231n, Python/Numpy Tutorial
1/19	Lecture #2 (Shenlong): Poses, Transforms - 3D Transformations - Rotation Representations [ slides \| video \| notes ]	State Estimation on Robotics, Sec. 6.1 - Sec.6.3 Computer Vision, Sec.2.1.1 - Sec.2.1.3
Module 2: Sensing
1/24	Lecture #3 (Shenlong): Camera I - Image Formation - Perspective Geometry [ slides \| video \| notes ]	Computer Vision, Sec.2.1.1 - Sec.2.1.3
1/26	Lecture #4 (Shenlong): Camera II - Epipolar Geometry - Stereo, Event Cameras [ slides \| video \| notes ]	Computer Vision, Sec.2.1.4 State Estimation on Robotics, Sec. 6.4	Assignment 1 out
1/31	Lecture #5 (Shenlong): Camera III - Multi-view Geometry - Calibration [ slides \| video \| notes ]	Computer Vision, Chapter 12.1 - 12.3 State Estimation on Robotics, Sec. 6.4
2/2	Lecture #6 (Shenlong): Other Sensors I - LiDAR, Radar, Sonar [ slides \| video \| notes ]
2/7	Lecture #7 (Shenlong): Other Sensors II - GPS, IMU, Odometer - Touch, Tactile, etc [ slides \| video \| notes ]	State Estimation on Robotics, Sec. 6.4.4
2/9	Lecture #8 (Shenlong): Other Sensors III - Sound - Tactile [ slides \| video \| notes ]		Assignment 1 due Assignment 2 out
Module 3: State Estimation
2/14	Lecture #9 (Shenlong): State Estimation I - State Estimation Theory - Bayes Filtering, Kalman Filters - Particle Filters, Histogram Filters [ slides \| video \| notes ]	Probabilistic Robotics, Chap 2, 3, 4
2/16	Lecture #10 (Shenlong): State Estimation II - Bayes Filtering, Kalman Filters - Particle Filters, Histogram Filters [ slides \| video \| notes ]	Probabilistic Robotics, Chap 2, 3, 4
2/21	Lecture #11 (Shenlong): 3D Representations - Voxel, Mesh, Points, SDFs - Representation Learning [ slides \| video \| notes ]	Tutorial on 3D Deep Learning Slides [Demo] Open3D, Point Cloud, Mesh, RGBD Images
2/23	Lecture #12 (Shenlong): Map-based Localization - Map Representations - Registration and Matching [ slides \| video \| notes ]	Probabilistic Robotics, Ch. 9
2/28	Lecture #13 (Shenlong): SLAM I - RGBD and LiDAR SLAM [ slides \| video \| notes ]	[R] Newcombe et al., KinectFusion Real-Time Dense Surface Mapping and Tracking
3/2	Lecture #14 (Shenlong): SLAM II - Visual Odometry - Visual SLAM [ slides \| video \| notes ]	Cadena et al., Past, Present, and Future of Simultaneous Localization And Mapping	Assignment 2 due Assignment 3 out
Module 3: Learning-based Perception
3/7	Lecture #15 (Shenlong): Deep Learning I - MLP, Backprop - CNNs [ slides \| video \| notes ]	Deep Learning, Chap. 6, 9, 10, 20 CS498 Intro to DL Relavent Slides [Demo] Pytorch Tutorials Learning Basics	Project Proposal due
3/9	Lecture #16 (Shenlong): Deep Learning II - RNNs, GNNs, Transformers [ slides \| video \| notes ]	Deep Learning, Chap. 6, 9, 10, 20 CS498 Intro to DL Relavent Slides [Demo] Pytorch Tutorials Learning Basics
3/14	No class, spring break
3/16	No class, spring break
3/21	Lecture #17 (Shenlong): Motion Understanding - Optical Flow - Nonrigid Tracking [ slides \| video \| notes ]
3/23	Lecture #18 (Shenlong): Semantic Segmentation - 2D Semantic Segmentation - 3D Semantic Segmentation [ slides \| video \| notes ]
3/28	Lecture #19 (Shenlong): Object Detection - 2D and 3D Detection [ slides \| video \| notes ]		Assignment 3 due
3/30	Lecture #20 (Shenlong): Object Tracking - 2D and 3D Tracking [ slides \| video \| notes ]		Assignment 4 out
4/4	Lecture #21 (Shenlong): Object Pose Estimation - 6-DoF Pose Estimation - Articulated Pose Estimation [ slides \| video \| notes ]
4/6	Lecture #22 (Shenlong): Object Pose Estimation II - Articulated Pose Estimation [ slides \| video \| notes ]
4/11	Lecture #23 (Shenlong): Simulation I - Intro to Simulation - Sensor Simulation - Sim2Real [ slides \| video \| notes ]
4/13	Lecture #24 (Shenlong): Simulation II - Sensor Simulation - Sim2Real [ slides \| video \| notes ]
4/18	Lecture #25 (Shenlong): Multi-Modal Perception - Data Fusion - Transfer Learning [ slides \| video \| notes ]		Assignment 4 due Assignment 5 out, due 5/2
Module 4: Case Studies
4/20	Lecture #26 (TBD): Applications - Self-Driving [ slides \| video \| notes ]
4/25	Lecture #27 (TBD): Applications - Mixed Reality [ slides \| video \| notes ]
4/27	Lecture #28 : no class, preparing final project [ slides \| video \| notes ]
5/6	Final Project Report due