Yafan Huang

CMSC828G: Systems for Machine Learning

This course focuses on computer systems and parallel computing research for advancing machine learning, in particular, large language model (LLM) systems. Topics include GPU data compression, KV cache and memory management, profiling and parallel scaling of LLM workloads, kernel optimizations, optimizers, parallel training and inference, and hardware-software co-design.

Course Information

Prerequisites For graduate students: Familiarity with basic systems and machine learning concepts is recommended. Students may find prior coursework such as CMSC412, CMSC416, or CMSC422 helpful. For undergraduate students: Permission of the instructor is required.

Course Overview

The goal of this course is to understand the systems challenges behind modern LLM training and inference, how high-performance and parallel computing techniques can address these challenges, and what open systems problems remain. Students will learn to analyze the performance of LLM workloads and reason about the design tradeoffs across algorithms, software systems, and hardware. For each topic, the instructor will first provide one or two lectures introducing the key concepts and background, followed by student-led presentations and discussions of selected research papers.

This course will cover below topics (subject to change):

• Introduction to High-Performance Computing (HPC).
• Programming with Triton and CUDA.
• Introduction to LLM systems.
• Challenges in high-performance LLM inference and training.
• Distributed parallel training: data, model, and hybrid parallelism.
• On-node optimizations: GPU kernel optimizations, ML compilers, memory offload.
• KV cache and memory management: long context, quantization, data movement.
• Hardware and software co-design for efficient LLM systems.

Throughout the semester, each student will participate in paper readings, paper presentation, two midterm exams, and final project. Students are expected to attend class regularly and actively engage with lectures and presentation discussions.

Textbook

There is no required textbook for this course. Most lectures will be accompanied by readings from top-tier conference research papers, and a reading list with links to the relevant papers will be provided. The following books may be useful as optional references:

• HPC: The Art of HPC book by Victor Eijkhout.
• Deep Learning (DL): Deep Learning book by Ian Goodfellow et al.
• MLSys: Introduction to Machine Learning System books by Vijay Janapa Reddi.
• GPU Programming: Modern GPU Programming for MLSys by MLC Community.

Grading

Grade for each student will be determined as follows:

More details for each component can be found in later sections.

Tentative Schedule (Subject to Change)

Note: Classes with papers listed in the Readings column include student paper presentations and require a paper reading report. Specific presentation assignments will be determined after the semester begins through a student sign-up process.

Academic Conduct

All students are expected to adhere to the University of Maryland’s standards of academic integrity. Academic dishonesty, including cheating, plagiarism, fabrication, and facilitation, is a violation of university policy and may result in disciplinary action. Students are responsible for understanding and following the UMD Academic Code of Conduct.

Collaboration. Discussion and collaboration with classmates are encouraged, particularly when reading papers and exploring course topics during lectures. However, paper readings, paper presentations, and midterm exams must be completed independently and reflect each student’s own understanding. Collaboration within your project team is only expected for the final project.

Use of Generative AI. Generative AI tools (e.g., ChatGPT and Claude) may be used as learning aids (e.g. to clarify concepts). However, submitted work must reflect your own understanding and intellectual contribution. Students must disclose substantive use of generative AI in submitted coursework and briefly describe how it was used. Generative AI tools are not permitted during midterm exams.

Paper Readings

For each class with assigned readings, students should choose one of the two assigned papers and submit a one-page reading report. The report should briefly summarize the paper’s motivation, key ideas, main results, and your own thoughts or questions. The same set of assigned papers will also be used for the individual paper presentations described in the next section. Reports are due at 10:00 PM the day before class (Monday for Tuesday classes and Wednesday for Thursday classes). Each student may skip up to three reading reports without penalty. No make-up submission is required for skipped reports. Please note that reading report submission is only required for the lectures that are marked with assigned readings in the course schedule.

Paper Presentation

Each student will give one individual paper presentation on one of the assigned papers. Paper presentations will take place during the class meetings marked with assigned readings in the tentative course schedule. Specific presentation assignments will be determined after the semester begins through a student sign-up process. The presentation should be approximately 20 minutes, followed by a 5–7 minute Q&A session. Presentations will be evaluated based on technical understanding, clarity and organization, presentation quality, and Q&A performance. Audience feedback will also be collected and considered, particularly regarding the clarity and effectiveness of the presentation.

Midterm Exam

There will be two in-class written midterm exams. Midterm 1 primarily covers LLM training, while Midterm 2 primarily covers LLM inference. Both will also cover relevant HPC, GPU, and ML concepts. Both exams will be based on material covered in lectures and class discussions.

Final Project

Students will work in groups of 2–3 on a course project related to systems for machine learning. Individual projects are allowed with instructor permission. Important project dates are listed in the tentative course schedule. Each group will give a Final Project Pitch Presentation during the semester to introduce the project motivation, proposed approach, and project plan. At the end of the semester, each group is expected to give a Final Project Presentation. The final deliverables include:

• Final Presentation Slides: Submit the slides for Final Project Presentation.
• Implementation: Submit the implementation developed for the project.
• Final Report: Submit a report written in the MLSys paper format, including Introduction, Background, Methodology, Results, and Conclusion.

Some example project topics (including projects from past offerings) are:

• Customized GPU Compression Kernels for KV Cache Management
• Anatomy and Performance Analysis of RadixAttention in SGLang
• Understanding and Characterizing KV Connector in vLLM
• Explore Lossless Compression for Communication in LLM Training
• Profiling and Optimizing KV Caching and Prefix Caching in LLM Inference
• Efficient Implementation of LLMs on Handheld Edge Devices
• Efficient LLM Inference on Heterogeneous GPUs
• Accelerating Distributed LLM Inference on FPGAs
• Modeling Error Proapgation in LLM Inference or Training

University and Course Policies

Students are responsible for reviewing the University of Maryland’s Course-Related Policies and Resources, including policies regarding academic accommodations, excused absences, religious observances, academic integrity, and other course-related matters.

Web Accessibility

Powered by Jekyll and Minimal Light theme.