The class is conducted in a seminar format. The first set of classes will mainly be lectures aimed at introducing core concepts and techniques that will be useful in understanding state-of-the-art research in this area. After that the class will transition into reading papers, paper presentations, online discussions and a final class project. For Fall 2021, we are planning on meeting in person. However, if the circumstances change we may use Zoom and go fully virtual. Submission links and other infrastructure will be announced in subsequent lectures and will be posted here in advance.
Following is a tentative grading rubric we are going to use.
Activity | Grade | Details |
---|---|---|
Paper Reviews and Discussion | 25% |
|
Presentation and Discussion Lead | 25% |
|
Project | 50% |
|
We will first discuss the core concepts behind compiler construction and computer organization. We will go into details about optimization and design decisions compiler engineers and computer architects face. Next, we will discuss core Machine Learning concepts that are needed to understand research done in this area. We will cover black-box optimizations, neural network and basics of sequential decision making. We will use this knowledge to read latest research papers that are published in this area covering compiler auto-tuning, cost models, domain specific optimizations, design space exploration for domain specific architectures and data-driven simulation etc.
Date | Topic | Presenter | Notes |
---|---|---|---|
8/24 |
IntroductionIntroduction to compilers, architecture and logistics |
Charith Slides |
|
8/26 |
CompilersQuick overview of Compiler Construction + Optimizations |
Charith Slides |
|
8/31 |
Compiler OptimizationsAnatomy of a Compiler Optimization Pass, DSLs, Domain Specific Optimizations |
Charith Slides |
|
9/02 |
DSLs + ML in ArchitectureContinuation of discussion on DSLs and examples of ML in architecture |
Charith Slides |
|
9/07 |
Machine Learning TechniquesQuick overview of ML techniques: Neural Networks |
Charith Slides |
|
9/09 |
Machine Learning Techniques (Contd.) and Auto-tuningQuick Overview of ML techniques: Genetic Algorithms, Simulated Annealing, Sequential Decision Making; Introduction to Auto-tuning |
Charith Slides |
|
9/14 |
Autotuning: Empirical Autotuning |
Charith Discussion Pathirikt |
|
9/16 |
Autotuning: Languages for exposing choices |
Vincent |
|
9/21 |
Autotuning: Techniques |
Edward |
|
9/23 |
Autotuning: Frameworks |
Rafae | AutoTVM: Learning to Optimize Tensor Programs (NeurIPS 2018) |
9/28 |
Autotuning: Scaling Up |
Archit | Portable Performance on Heterogeneous Architectures (ASPLOS 2013) |
9/30 |
Autotuning: Diverging Workloads |
Garvita | Autotuning Algorithmic Choice for Input Sensitivity (PLDI 2015) |
10/05 |
Guest Lecture: Autotuning in the industry (Google) |
||
10/07 |
Data-driven Cost Models: Part 1 |
Ashitabh | Learning execution through neural code fusion (ICLR 2020) |
10/12 |
Data-driven Cost Models: Part 2 |
Kartik | A Deep Learning based cost model for automatic code optimization (MLSys 2021) |
10/14 |
Program Embeddings: Part 1 |
Damitha | Blended, precise semantic program embeddings (PLDI 2020) |
10/19 |
Program Embeddings: Part 2 |
Jiawei | IR2Vec: LLVM IR Based Scalable Program Embeddings (TACO 2020) |
10/21 |
Learned Optimizations: Traditional Compiler Optimizations 1 |
Leon |
|
10/26 |
Learned Optimizations: Traditional Compiler Optimizations 2 |
Stefanos | AutoPhase: Juggling HLS Phase Orderings in Random Forests with Deep Reinforcement Learning (MLSys 2020) |
10/28 |
Learned Optimizations: DSLs Part 1 |
Hyoungwook | Ansor: Generating High-Performance Tensor Programs for Deep Learning (OSDI 2020) |
11/02 |
Learned Optimizations: DSLs Part 2 |
Vishnu | The case for learned index structures (SIGMOD 2018) - databases |
11/04 |
Learned Optimizations: Tensor Programs |
Yong Zhi | Transferable Graph Optimizers for ML Compilers (NeurIPS 2020) |
11/09 |
Architecture Design Space Exploration: Part 1Project Clinic |
Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search (ASPLOS 2021) |
|
11/11 |
Architecture Design Space Exploration: Part 2 |
Ajay | A Full-stack Accelerator Search Technique for Vision Applications |
11/16 |
Guest Lecture: Chris Cummins from Facebook |
||
11/18 |
Recap + Project Clinic |
||
11/23 |
Break |
||
11/25 |
Break |
||
11/30 |
Learned Architecture Simulation |
Jovan | SimNet: Computer Architecture Simulation using Machine Learning |
12/02 |
Learned Systems: Caches |
Benjamin |
|
12/07 |
Student Presentations |
You are required to propose and complete a class project by the end of the semester (Groups of 2). The goal is to pursue an idea in a novel research direction. At the end of the semester you should present your findings (negative results are fine!) and submit a write up describing the problem specification, methodology, results and key takeaways. Even though it is encouraged to do a project exploring an idea in a novel direction, following are all the project types you can pursue.
Please feel free (not required) to setup a time before you submit your proposal to discuss interesting ideas that can be your class project.
Please use this form to submit your proposal, presentation, report and artifact link (if applicable).