ECE 498 AI Systems & Engineering

In this course, we focus on teaching students practical system-building skills and toolchains for implementing large language models (LLMs) from scratch. We cover the entire lifecycle of developing an LLM and its enabled applications, including LLM architecture implementation, model training, fine-tuning, inference, and agentic applications. We will not only discuss the basic building blocks of foundation models, but also learn the essential computing techniques and engineering skills needed for enabling functionality, efficiency, and scalability.

Students will have multiple programming assignments to strengthen their understanding of the basic concepts and practice their systems-building skills. Specifically, lecture topics include PyTorch and development tools, CUDA implementation of tensor operators, AI infrastructure, the basics of foundation models (e.g., tokens, embeddings, attention mechanism, and mixtures of experts), LLM training methods, training parallelism, data engineering, fine-tuning methods, inference engine development and deployment, prompt engineering, agent development, RAG-based LLM, and applications.

It is worth noting that this course focuses on exploring problem-solving approaches for LLM development. Due to limited computing resources in academia, some programming assignments of this course may not reach industrial scale; however, the development approaches are generic. Students will obtain hands-on experience of developing and deploying LLMs throughout this course.

Prerequisites: Python programming. Prior exposure to CUDA programming (ECE408) and machine learning (CS446/ECE449, CS440/ECE448, ECE479) will be beneficial.