CS 598: Machine Learning Algorithms for Large Language Models

Fall 2024

Course description:
This course is a general overview of machine learning algorithms used in the current development of large language models (LLMs), with special emphasis on models and model training. It covers a relatively broad range of topics, starting with mathematical models for sequence generation, and important neural network architectures with a focus on transformers. We will then investigate variants of transformer based language models, along with algorithms for prompt engineering and improving reasoning capability. Other topics include ML techniques used in studying LLM safety, hallucination, fine-tuning of LLMs, alignment (reinforcement learning from human feedback), multimodal LLMs, and common methods for accelerating training and inference.
Prerequisites:
This course focuses on the understanding of machine learning algorithms and techniques in the development of LLMs. Therefore students are expected to have a solid foundation in machine learning (especially deep learning), and programming in python. Prior project experience with machine learning, natural language processing, and PyTorch is also needed. The students should also have solid background in mathematics, and are expected to be able to understand abstract notations in advanced probability and linear algebra without any difficulty.
Class time:
Tues, Thu, 11:00am–12:15pm, Siebel 2406
Instructor:
Prof. Tong Zhang (tozhang@illinois.edu)
- Office: Siebel Center 2118
- Office hour: Tues, Thu 10am-11am
TA:
Sayantani Basu (basu9@illinois.edu)
- Office hour: Weds 9-10am in Siebel Center 2124 (exception: Rm 3124 on Sept 18)
Course format:
- I will give the first three lectures and distribute lecture slides.
- Other lectures will be a mixture of two student paper presentations followed by an overall summary of the topic by me.
- The last two lectures are student project presentations.
Grading:
- individual assignments: six small programming assignments (48%), one paper presentation (12%), class participation (10%)
- group assignments (12 groups with 3-4 students in each group)
  - final project (25%) + presentation (5%)
Course Material:
- Lecture slides and papers

Lectures

Lecture Number	Topic
1.	Introduction
2.	Training and Optimization
3.	Sequence Modeling
4.	Transformer
5.	State Space Model
6.	Extensions of Transformer
7.	Encoder-Only and Encoder-Decoder Model
8.	Decoder-Only Model (GPT)
9.	Scaling Law and Emergent Abilites
10.	Prompt Engineering
11.	LLM Safety
12.	Hallucination
13.	Retrieval Augmented Generation
14.	Instruction Tuning - Methods
15.	Instruction Tuning - Data Aquisition
16.	Fine-Tuning and Evaluation
17.	Resource Efficient Finetuning
18.	RLHF Basics
19.	RLHF Algorithms
20.	Open-source LLMs
21.	LLM with Tools
22.	Planning and Agents
23.	LLM Data Generation and Distillation
24.	Coding LLM
25.	Math LLM
26.	Multimodal Embedding
27.	Multimodal LLMs
28.	GPU Acceleration Techniques (Self Study)
29.	Probing and Interpretability (Self Study)