Course Syllabus

Welcome to ECE408 / CS483 / CSE408

The aim of this course is to provide knowledge and hands-on experience in developing software for processors with massively parallel computing resources, such as Graphics Processing Units, or GPUs.  Effectively programming these processors requires in-depth knowledge about parallel programming principles, as well as the parallelism models, communication models, and resource limitations of these processors. In addition, this course also prepares the students to use the parallelism models of modern programming environments for high-performance computing and AI. The target audiences of the course are students who want to develop applications for these processors for fun, for work, and/or for research, as well as those who want to develop programming tools and future processors.

The course involves 8 programming lab assignments of steadily growing complexity.  All assignments will involve programming a massively parallel GPU system using CUDA, which is a popular commercial language extension of C/C++ for GPU programming.  Assignments involve tasks such as matrix multiplication, convolution, reduction, histogram calculation, and sparse matrix-vector multiplication.  During the final third of the semester, students work on a larger, more complex project/competition. 

Course Staff

Instructors

Prof. Volodymyr Kindratenko

Graduate Teaching Assistants

Howie Liu (hanwenl4@illinois.edu),
Yifei Song (yifeis7@illinois.edu),
Shengjie Ma (sm138@illinois.edu),
Yue Yuan (yuey10@illinois.edu),
Xiyue Zhu (xiyuez2@illinois.edu).

Undergraduate Course Assistants

Hrishi Shah (hrishis2@illinois.edu),
Colin Zeng (kelinz2@illinois.edu),
Charles Tianwei Pei (ctpei2@illinois.edu),
Vijay Beekay Daita (vijayd2@illinois.edu),
Daksh Kalley (dkalley2@illinois.edu).

Class Time and Location

AL: 1002 ECEB, Tuesdays and Thursdays, 9:30-10:50am US Central Time by Prof. Volodymyr Kindratenko. Lectures will be recorded and posted on Mediaspace.

ALL other sections: Recorded lectures posted on Mediaspace.

Reading Materials

Textbook

D. Kirk and W. Hwu, “Programming Massively Parallel Processors – A Hands-on Approach,” Morgan Kaufman Publisher, 3rd edition, 2016, ISBN 978-0123814722.  Relevant to this course chapters can be found under Files/Textbook.

NVIDIA documentation

NVIDIA, CUDA C Programming Guide (CUDA PG)
https://docs.nvidia.com/cuda/cuda-c-programming-guide Links to an external site.
https://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf Links to an external site.

NVIDIA, CUDA C++ Best Practices Guide (CUDA BPG)
https://docs.nvidia.com/cuda/cuda-c-best-practices-guide  Links to an external site.
https://docs.nvidia.com/cuda/pdf/CUDA_C_Best_Practices_Guide.pdf Links to an external site.

NVIDIA Developer Blog (NVIDIA DB)
https://developer.nvidia.com/blog Links to an external site.

Computer System Setup

We will use NCSA's Delta supercomputer for programming assignments (labs and project) in this course. We will distribute course assignments via a GitHub repository and will collect students' solutions via GitHub as well. Below are instructions on how to get started with Delta and GitHub. 

Delta Access

To get started with Delta, follow these steps:

  1. Go to https://identity.access-ci.org/new-user Links to an external site. and register yourself as a new user. If you already have an ACCESS account (some of you might already have it due to a research project, etc.) do not register as new user, just use the same username. When registering for new account, use "University of Illinois Urbana-Champaign" as your Identity Provider. This way, you can just use your UofI credentials. 
  2. Once your account is created, note your ACCESS user ID and submit this user ID to the course staff via this form: https://forms.illinois.edu/sec/1378190679.  
  3. Wait for an email from NCSA Delta team confirming completion of your account registration. This may take several days, so please be patient. After receiving the confirmation email, you should be able to login into Delta via ssh and clone the GitHub repository to get started with the labs.
    • While you are waiting for a confirmation that your account has been setup on Delta, go ahead and familiarize yourself with Delta's documentation and also setup your class GitHub repository.  

GitHub setup

To get started with GitHub:

  1. First, follow this link to establish a class repository: https://edu.cs.illinois.edu/create-gh-repo/sp25_ece408. This needs to be done only once. After this step is complete, you should be able to visit https://github.com/illinois-cs-coursework/sp25_ece408_NetID page (where NetID is your NetID) that shows the contents of your repository. Initially, this repository will be empty.
  2. After completing step 1, you should be able to access the class release repository at https://github.com/illinois-cs-coursework/sp25_ece408_.release Links to an external site.. Please read 
    README.release.md Links to an external site.file for further instructions about what to do next. You can proceed with these steps once your Delta account is setup. 

Getting Help

Office hours

Course instructor and TAs will have office hours offered in-person and on-line starting on the second week of the semester (week of 1/27). Watch this space for updates.

US Central Time Monday Tuesday Wednesday Thursday Friday
8 - 9 am
9 - 10am  
10 - 11am   Daksh
ECEB 5034
11am - noon Prof. Kindratenko
3050E NCSA
(except 2/24 and 3/31)
Hrishi
ECEB 4036
noon - 1pm Charles
ECEB 4034
1 - 2pm Shengjie
ECEB 4036
Hrishi
ECEB 4036
2 - 3pm Prof. Kindratenko
3050E NCSA
(except 3/31)
Yue
ECEB 3036
3 - 4pm Xiyue
Zoom Links to an external site.
Yifei
Zoom Links to an external site.
Colin & Howie
ECEB 4034
4 - 5pm
5 - 6pm
6 - 7pm Vijay
ECEB 4036
7 - 8pm
8 - 9pm

Discussion board

We will use Campuswire Links to an external site. for Q&A. Sign up here Links to an external site. if you are not yet enrolled. (You will be asked for a verification code, which is 1698.)

Course chatbot

Yes, we are providing a GPT-4 powered AI chatbot trained on ECE 408 course materials! You are welcome to use it to learn, not to cheat! You can start by asking the chatbot how to succeed in this class. :)

Please read this disclaimer Links to an external site. before using the chatbot, this a very experimental technology and you must understand its limitations and agree to the provisions outlined in the disclaimer. 

You can use the chatbot in a couple of ways, just follow the corresponding links:

Grading

Grads and undergrads in the same grading pool. All grades will be posted in Canvas.

  • Exam 1: 25%
  • Exam 2: 25%
  • Lab Assignments: 25%
    • Correctness (determined by the number of data sets passed and the functionality) 90%
    • Responses to the quiz 10%
    • Lab with the lowest score will be dropped from the final grade, therefore no late submissions beyond grade period will be allowed. 
    • Lab 0 and Lab 1 grades will not be counted towards the final grade. 
  • Final Project: 25%
    • Detailed Rubric will be posted with the project description

Grace Period for Assignments

All assignments are due by Friday at 8pm. However, recognizing potential issues such as Delta system maintenance, interviews, or illness, we provide a three-day grace period. You can submit an assignment by Monday at 8pm and still receive full credit without penalty. You do not need to notify us if you intend to use this default 3-day grace period. However, to extend your grace period beyond 3 days, you must notify us by Friday 8pm via email to the TA(s) responsible for the assignment you are asking about. Extensions to the grace period are given out for extenuating circumstances at our discretion. Requests after this time will not be considered. Please also note that the code in your GitHub repository at 8pm on Monday will be considered the final version, even if you had a correct version earlier. We will retrieve and grade the code at that time.

Lecture Schedule

Lecture slides can be found here. Reading materials can be found here and here Links to an external site.. Lecture recordings can be found here.

Week Date  Activity Topic Reading
1 1/21/25 Lecture 1 Introduction to the Course Kirk & Hwu: Chapter 1
CUDA PG: Introduction Links to an external site.
1/23/25 Lecture 2 Introduction to Parallel Computing and CUDA Kirk & Hwu: Chapter 2
CUDA PG: Programming Model Links to an external site.
1/24/25 Lab 0 due ACCESS and GitHub setup; Delta access and Device Query example

 

2 1/28/25 Lecture 3 CUDA Parallel Execution Model Kirk & Hwu: Chapter 3
1/30/25 Lecture 4 CUDA Memory Model Kirk & Hwu: Chapter 4
1/31/25 Lab 1 due Vector Addition
3 2/4/25 Lecture 5 Locality and Tiled Matrix Multiply Kirk & Hwu: Chapter 4
CUDA PG: Shared Memory Links to an external site.
NVIDIA DB: Shared Memory Links to an external site.
2/6/25 Lecture 6 Generalized Tiling and DRAM Bandwidth Kirk & Hwu: Chapter 5
CUDA BPG: Device Memory Spaces Links to an external site.
NVIDIA BD: Global Memory Links to an external site.
2/7/25 Lab 2 due Simple Matrix Multiplication
4 2/11/25 Lecture 7 Convolution concept (1D and 2D); 1D Basic Convolution Kernel, and constant cache Kirk & Hwu: Chapter 7
CUDA BPG: Device Memory Spaces Links to an external site.
2/13/25 Lecture 8 2D Tiled Convolution Kernel Kirk & Hwu: Chapter 7
NVIDIA DB: Shared Memory Links to an external site.
2/14/25 Lab 3 due Tiled Matrix Multiplication
5 2/18/25 Lecture 9 Shared Memory Reuse Analysis Kirk & Hwu: Chapter 7
2/20/25 Lecture 10 Introduction to Machine Learning

 

2/21/25 Lab 4 due 3D Convolution
6 2/25/25 Lecture 11 Deep Networks: Inference and Training Kirk & Hwu: Chapter 16
2/27/25 Lecture 12 Computation in CNNs and Transformers Kirk & Hwu: Chapter 16
2/28/25
7 3/4/25 Midterm 1 Lectures 1-12 and Labs 0-4
3/6/25 Lecture 13 Reductions Kirk & Hwu: Chapter 5
3/7/25 CNN-PM1 due Project Milestone 1 Code
8 3/11/25 CA lecture Overview of Project
3/13/25 Lecture 14 Parallel Scan: Kogge-Stone Kirk & Hwu: Chapter 8
3/14/25 Lab 5 due List Reduction
Spring Break
9 3/25/25 Lecture 15 Parallel Scan: Brent-Kung Kirk & Hwu: Chapter 8
3/27/25 Lecture 16 Histogramming and Atomic Operations Kirk & Hwu: Chapter 9
CUDA PG: Atomic Functions Links to an external site.
3/28/25 Lab 6 due Scan
10 4/1/25 On-line Guest lecture

Robert Searles, Senior Solutions Architect, NVIDIA

Application Profiling with Nsight Systems & Nsight Compute

C. Pearson's slides Links to an external site. and videos (part1 Links to an external site.part2 Links to an external site.part3 Links to an external site.part4 Links to an external site.)
Nsight Systems Links to an external site.Nsight Compute Links to an external site.
4/3/25 Lecture 17 Sparse Matrix I Kirk & Hwu: Chapter 10
4/4/25 Lab 7 due Histogram
11 4/8/25 Lecture 18 Sparse Matrix II Kirk & Hwu: Chapter 10
4/10/25 Lecture 19 GPU Systems Architecture
4/11/25 CNN-PM2 due Project Milestone 2 Code & Report
12 4/15/25 Lecture 20 Tasks Parallelism and Asynchronous Data Transfer Kirk & Hwu: Chapter 18
4/17/25

Guest Lecture 21

Sanjay Patel, Distinguished Senior Compute Platforms Architect, NVIDIA

Tensor Processing APIs

NVIDIA Technical Blog Links to an external site.
CUDA PG: Warp Matrix Functions Links to an external site.
4/18/25 Lab 8 due Sparse Matrix Multiplication
13 4/22/25 Lecture 22 Other Accelerations APIs
4/24/25 Guest Lecture TBD
14 4/29/25 Guest Lecture TBD
5/1/25 Lecture 23 Course Retrospective

5/2/25

CNN-PM3 due Project Milestone 3 Code & Report
15 5/6/25 Midterm 2  
5/9/25 Project Competition due Project Competition

Labs (aka MPs)

Lab (aka MP) assignments will be done using NCSA's Delta system for coding and PrairieLearn Links to an external site. for quiz submission. Programming part of each lab will be distributed via GitHub repository. Q&A part will be done via PrairieLearn. Do not start working on the quiz part before completing the programming part as the quiz will require providing info from the actual working lab. Labs are individual assignments; they must be carried out and submitted individually. 

Lab assignments are due on Fridays at 8:00pm US Central Time, unless otherwise noted or explicitly extended by the teaching staff. 

Date released Due Date Notes
Lab 0: Device Query 1/21/25 1/24/25 not counted toward the final grade
Lab 1: Vector Addition 1/24/25 1/31/25 not counted toward the final grade
Lab 2: Simple Matrix Multiplication 1/31/25 2/7/25
Lab 3: Tiled Matrix Multiplication 2/7/25 2/14/25
Lab 4: 3D Convolution 2/14/25 2/21/25
Lab 5: List Reduction 3/7/25 3/14/25
Lab 6: Scan 3/14/25 3/28/25
Lab 7: Histogram 3/28/25 4/4/25
Lab 8: Sparse Matrix Multiplication 4/11/25 4/18/25

Lab TA: Yue Yuan and Xiyue Zhu

Project-CNN

We release the project in the same GitHub repository as the labs. Use "git pull" if you have checked out the project before. We will update instructions one milestone at a time. Projects are individual assignments, they must be carried out and submitted individually. We will use Delta for project as well.

Milestones are due on Fridays at 8:00pm US Central Time, unless otherwise noted or explicitly extended by the teaching staff.

Date Released Due Date Notes
Milestone 1 2/21/25 3/7/25
Milestone 2 3/12/25 4/11/25
Milestone 3 4/15/25 5/2/25
Final Competition 5/5/25 5/9/25 optional

Project-CNN TAs: Shengjie Ma and Yifei Song

Project-GPT

This semester, we are excited to launch a new project where we implement GPT-2 from scratch! Our goal is to build a fully functional version of the model, diving deep into the transformer architecture.

Stay tuned for updates, behind-the-scenes insights, and technical deep dives as we publish this project! 🚀✨

Project-GPT TA: Howie Liu

Exams

There are two midterm exams, there is no final exam.

Midterm 1 Midterm 2

When: 3/4/25, 7-10pm

Where: TBD

What: Lectures 1-12 and Labs 0-4

How: paper-based 

Alternative Exam Time: as arranged  

Study Materials: see Files/MT1

Review Session: TBD

Last-minute Office Hours: TBD

The cover page and the reference sheet will be provided.

When: 5/6/25, 7-10pm

Where: TBD

What: All lectures, including all guest lectures, all Labs, all project milestones. With a focus on everything after Exam 1.

How: paper-based

Alternative Exam Time: as arranged  

Study Materials: see Files/MT2

Review Session: TBD

Last-minute Office Hours: TBD

The cover page and the reference sheet will be provided.

Cheating and Academic Integrity

We take cheating very seriously. If we catch you cheating, or helping someone cheat, the penalty will not be less than a zero on the applicable assignment and the incident will be reported to the College.

The Student Code defines cheating as:

Using or attempting to use in any academic exercise materials, information, study aids, or electronic data that the student knows or should know is unauthorized (§1-402(a))

and notes that:

Substantial portions of the same academic work may not be submitted for credit more than once or by more than one student without authorization. (§1-402(a)(4))

The student code also disallows facilitation of cheating, prohibiting

Helping or attempting to help another to commit an infraction of academic integrity, where one knows or should know that through one’s acts or omissions such an infraction may be facilitated. (§1-402(c))

Cheating is an offense to the entire academic ecosystem: you, your classmates, and the course staff. It creates extra work for the course staff, it creates grading imbalances for your classmates, and it prevents you from learning the material that the rest of your academic career is built on.