Class calendar

Talk title: Codesigning Computing Systems for Artificial Intelligence

Abstract: The rapid advancement of artificial intelligence has ushered in an era of unprecedented computational demands, necessitating continuous innovation in computing systems. This talk highlights how codesign has enabled innovative solutions and state-of-the-art performance in Google's AI computing systems, namely Tensor Processing Units (TPUs).

It presents several codesign case studies across hardware, systems, software, algorithms, and the datacenter. The discussion examines how TPUs have made judicious, yet opinionated, design choices that have both kept pace with the rapid rate of change and enabled many breakthroughs in AI.

Speaker bio: Suvinay Subramanian is a Senior Staff Engineer at Google DeepMind, where he works on architecture and codesign for Google's ML supercomputers, Tensor Processing Units. His work has directly influenced innovative architecture and systems features in multiple TPU generations, supporting high-performance training and serving for Google's research and production AI workloads. He received a Ph.D. from MIT and a B.Tech. from the Indian Institute of Technology Madras. He is a recipient of the ACM SIGMICRO Early Career Award in Computer Architecture and co-hosts the Computer Architecture podcast, which spotlights developments in computer architecture and systems.

Talk title: The CPU Is the Agentic Critical Path: The Case for Heterogeneous Core Profiles

Abstract: Serving a language model is a forward pass. Serving an agent is a loop: reason, retrieve, act, admit, context, commit. Five of those six phases run on general-purpose cores. The CPU in this workload is not the host attached to an accelerator; it is the critical path. This talk argues that the six phases have CPU signatures different enough that no single core profile serves them all, and presents the measurements behind that claim.

Starting with what happens when a message is sent to an AI assistant, the talk traces where a single pass becomes a loop. It then examines how to count what an agent system actually delivers: trials per minute, turns per second, sandbox work seconds, and agents held inside an SLO. Aggregate throughput is the wrong axis to buy on. A capacity model decomposes delivered capacity into routing, balance, and offload efficiency, drawing on familiar operational laws including Amdahl's law, Little's law, the universal scalability law, and roofline.

The empirical core is phase-level measurement on server CPUs, including two counterintuitive findings: under agent density, the scheduler binds before memory bandwidth does; and moving tool execution to another node costs almost nothing. Both point toward racks whose nodes are matched to phases rather than uniform. The talk closes with implications for core profiles, on-die accelerators, and a memory hierarchy organized around sessions; a short critique session on three public capacity claims, including Intel's; and a set of open problems suitable for a semester project.

Speaker bio: Mrittika Ganguli is a Senior Principal Engineer and Platform Software Architect in Intel's Data Center Group. She leads Xeon platform software optimization and strategy, as well as software-enablement workstreams for Xeon 7 and 8. Her work spans accelerator enablement across QAT, DSA, and IAA, workload characterization, and cross-organization prioritization of software investments. She also works on agentic AI architecture, focusing on how CPU platforms, accelerators, and system software can deliver efficient, scalable infrastructure for emerging agentic AI workloads. She holds a master's degree in computer science and has published more than 80 patents.

Talk title: Building Distributed Inference at Scale: Lessons from NVIDIA Dynamo

Abstract: As AI models evolve toward longer contexts, reasoning, and agentic workloads, the demands on inference systems are changing. Serving these models efficiently requires coordinating computation, memory, and communication across a distributed system while meeting latency, throughput, and reliability goals.

This lecture explores how these trends shape inference architecture through lessons from building NVIDIA Dynamo. It covers major components including disaggregated prefill and decode, KV-cache-aware routing and management, data transfer, resource planning, and orchestration, using illustrative examples to explain how they work together. It also examines practical lessons in fault tolerance, recovery, load balancing, and adapting to changing workloads, connecting those design decisions to core machine learning and systems concepts and the tradeoffs of serving a model at scale.

Speaker bio: Dr. Vikram Sharma Mailthody is a Senior Research Scientist at NVIDIA Research and a co-architect of NVIDIA Dynamo. His work focuses on foundational systems-level challenges in emerging data center workloads, with an emphasis on scalable GPU memory and storage system architectures. He earned his Ph.D. in Electrical and Computer Engineering at UIUC, advised by Professor Wen-mei Hwu.

Talk title: From the Chip to the Rack and Beyond: Architecture for Large Language Models

Abstract: Frontier LLMs no longer fit on a single GPU, and the systems that run them have grown from a chip to a rack, and now to several racks working as one. The interconnect that ties those GPUs together has become one of the central problems in system design.

This talk examines how LLM training and inference shape the hardware underneath them, from the way parallelism appears as traffic on the fabric to the architectural tradeoffs that follow. It discusses how questions of bandwidth, latency, semantics, and energy are being answered differently in the LLM era, and what that means for building rack-scale systems and beyond. The talk closes with open problems in this space and work underway at NVIDIA Research.

Speaker bio: Benjamin Klenk joined NVIDIA in 2017 after completing his Ph.D. in Computer Engineering at Heidelberg University, Germany. His doctoral research was among the first to explore direct GPU-to-GPU communication models for GPU-accelerated high-performance computing. At NVIDIA, he contributed to several GPU on-chip networks and NVSwitch architectures, including in-network computing for collective operations. He now leads the Networking Research Group, whose work focuses on network and system architectures for energy-efficient, large-scale AI systems, heterogeneous accelerators, and multi-rack NVLink-class domains.