Course Objectives

You will learn about fundamental aspects of sensing and perception that will be of great use in robotics, augmented reality, entertainment, and more. You will also implement, debug and test machine perception algorithms on different sensory data in Python.

The core competency acquired through this course’s lectures is to gain a detailed understanding of sensing techniques (vision, motion, audio, touch), probabilistic state estimation, localization and mapping, 3D reconstruction, object detection, and scene understanding algorithms. Through this course’s programming assignments, students are expected to know how to write programs to estimate camera poses, insert 3D objects into images, reconstruct the 3D scene from sensors, detect people and follow them automatically, and more.

Prerequisites

Multi-variable calculus, linear algebra, data structures (CS 225 or equivalent), CS 361 or STAT 400. No previous exposure to machine learning, robotics or computer vision is required. However, it is recommended to complete one of the CS444, CS445, CS446 before this class. If you are not sure whether you meet the prerequisites, talk to the instructor and TAs during office hours.

We will not follow a specific textbook, but here are some recommended books that you can consult:

Grading

Grading Grading is based on participation, assignments, and the final project. Letter grades will be assigned based on the following thresholds.

A+ A A- B+ B B- C+ C C- D+ D D- F
>97 92-97 90-92 86-90 82-86 80-82 78-80 72-78 70-72 68-70 62-68 60-62 <60

Tentative grading scheme is as follows:

  • Assignments (75%). Five MPs @15% each, done individually, in Python. More details see here.
  • Final Project (25%). More details see here
  • Participation (Bonus 5%). Bonus credit for active participation in in-class discussion and on Campuswire.

Note that this class does not have any tests or exams.

Late Submission Policy:

  • Free late days for homework assignments: Each student gets a total of five free late days that apply to homework assignments throughout the whole semester. As long as you stay within your total late days budget, there is no need to request an extension and no late penalty will be assessed.

  • Free late days for projects: Each team gets a separate budget of three free late days that apply to the three project deliverables (proposal, final report).

  • Late penalty: If you are out of late days, for every day that your assignment is late, your score is multiplied by 0.6. Submissions that are more than five days late (beyond any free days) will not be accepted. You are not allowed to submit different parts of the assignment at different times to receive a late penalty on only part of the assignment.

  • Extension requests: Extension requests will be considered only after free late days are used up. Extensions beyond the free late days will be granted only in case of extraordinary circumstances. If you think that your circumstances qualify, email the instructor.

Acccommodations

To obtain disability-related academic adjustments and/or auxiliary aids, students should contact both the instructor and the Disability Resources and Educational Services (DRES) as soon as possible. You can contact DRES at 1207 S. Oak Street, Champaign, IL 61820, (217) 333-1970, or via email at disability@illinois.edu.

COVID Policy

Following University policy, all students are required to engage in appropriate behavior to protect the health and safety of the community. Students are also required to follow the campus COVID-19 protocols. Students who feel ill must not come to class. In addition, students who test positive for COVID-19 or have had an exposure that requires testing and/or quarantine must not attend class. The University will provide information to the instructor, in a manner that complies with privacy laws, about students in these latter categories. These students are judged to have excused absences for the class period and should contact the instructor via email about making up the work.

Students who fail to abide by these rules will first be asked to comply; if they refuse, they will be required to leave the classroom immediately. If a student is asked to leave the classroom, the non-compliant student will be judged to have an unexcused absence and reported to the Office for Student Conflict Resolution for disciplinary action. Accumulation of non-compliance complaints against a student may result in dismissal from the University.

All students, faculty, staff, and visitors are required to wear face coverings in classrooms and university spaces. This is in accordance with CDC guidance and University policy and expected in this class. Please refer to the University of Illinois Urbana-Champaign’s COVID-19 website for further information on face coverings. For more details about the University’s Covid-19 policy, please visit the website.

Academic Integrity

All work that you submit should be written solely by you and your group, and you should cite any significant sources of ideas. If any part of your project builds upon efforts before the semester (e.g., your ongoing research project), be sure to discuss with the instructor in advance. Plagiarism and other integrity violations will go on record at the university, and the minimum penalty will be a zero for the entire assignment. See the student code for more information on what constitutes an academic integrity violation.

Statement on anti-racism and inclusivity

The intent of this section is to raise student and instructor awareness of the ongoing threat of bias and racism and of the need to take personal responsibility in creating an inclusive learning environment.

The Grainger College of Engineering is committed to the creation of an anti-racist, inclusive community that welcomes diversity along a number of dimensions, including, but not limited to, race, ethnicity and national origins, gender and gender identity, sexuality, disability status, class, age, or religious beliefs. The College recognizes that we are learning together in the midst of the Black Lives Matter movement, that Black, Hispanic, and Indigenous voices and contributions have largely either been excluded from, or not recognized in, science and engineering, and that both overt racism and micro-aggressions threaten the well-being of our students and our university community.

The effectiveness of this course is dependent upon each of us to create a safe and encouraging learning environment that allows for the open exchange of ideas while also ensuring equitable opportunities and respect for all of us. Everyone is expected to help establish and maintain an environment where students, staff, and faculty can contribute without fear of personal ridicule, or intolerant or offensive language. If you witness or experience racism, discrimination, micro-aggressions, or other offensive behavior, you are encouraged to bring this to the attention of the course director if you feel comfortable. You can also report these behaviors to the Bias Assessment and Response Team (BART) (https://bart.illinois.edu/). Based on your report, BART members will follow up and reach out to students to make sure they have the support they need to be healthy and safe. If the reported behavior also violates university policy, staff in the Office for Student Conflict Resolution may respond as well and will take appropriate action.

Statement on CS CARES and CS Values and Code of Conduct

All members of the Illinois Computer Science department - faculty, staff, and students - are expected to adhere to the CS Values and Code of Conduct. The CS CARES Committee is available to serve as a resource to help people who are concerned about or experience a potential violation of the Code. If you experience such issues, please contact the CS CARES Committee. The Instructors of this course are also available for issues related to this class.

Statement on Mental Health

Diminished mental health, including significant stress, mood changes, excessive worry, substance/alcohol abuse, or problems with eating and/or sleeping can interfere with optimal academic performance, social development, and emotional well-being. The University of Illinois offers a variety of confidential services including individual and group counseling, crisis intervention, psychiatric services, and specialized screenings at no additional cost. If you or someone you know experiences any of the above mental health concerns, it is strongly encouraged to contact or visit any of the University’s resources provided below. Getting help is a smart and courageous thing to do – for yourself and for those who care about you.

Counseling Center: 217-333-3704, 610 East John Street Champaign, IL 61820 McKinley Health Center:217-333-2700, 1109 South Lincoln Avenue, Urbana, Illinois 61801 University wellness center: https://wellness.illinois.edu/

Acknowledgement

We would like to thank many researchers who have made their slides and course materials available.