CS 181AI: Machine Learning System Design, Spring 2023


Instructor: Prof. Arthi Padmanabhan

Lectures: MW 9:35 - 10:50 (Section 1), 11 - 12:15 (Section 2) SHAN 2460

Office Hours: McGregor 321, M 4:00-5:00; F 2:00-4:00

Course Overview

This course presents the steps and considerations for deploying machine learning models in real-world use cases. Students will become skilled at designing machine learning models and analyzing their model's tradeoff between performance and resource usage. They will also read and discuss relevant machine learning systems papers. Topics include creating a good dataset, designing a model, assessing your model's computational/memory/energy needs, when and how to use edge computing, and ways of lowering a model's resource usage.

Masking: You and your classmate's health is a priority in this class. Given the significant caseload of COVID still present in LA county, as well as the risks posed by travel and outside activities, masks will be required in class for the foreseeable future. Please arrive in class with a well-fitting mask on your face (while a surgical, KN95, or N95 mask is preferable, cloth is also fine as long as it actually fits). Though it is permissible to take short sips of water during class to avoid dehydration, you should return your mask to its proper position between sips. Other food or drink must be consumed outside the classroom. If circumstances change, this policy may be revisited later in the semester; in the meantime, if you have concerns, please let me know and I'll be happy to address them.

Lectures

# Title Link Files Reading
1 Overview Watch
2 Intro to ML Models Watch Lecture 2 Files
3 Training (intro) Watch Lecture 3 Files
4 Training (debugging low accuracy) Watch Lecture 4 Files Visualizing and Understanding Convolutional Networks (Group A)
5 Training (cont.) + Object Detectors Watch Lecture 5 Files Deep Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images (Group B)
6 Paper Discussion You Can't Escape Hyperparameters and Latent Variables: Machine Learning as a Software Engineering Enterprise (Group C)
7 Paper Discussion Closing the AI Accountability Gap: Defining the End-to-End Framework for Internal Algorithmic Auditing (Group D)
8 Performance + Parallelism Watch NoScope: Optimizing Neural Network Queries over Video at Scale (Group A)
9 Locking + Synchronization Focus: Querying Large Video Datasets with Low Latency and Low Cost (Group B)
10 Synchronization cont. Watch Cracking open the DNN black-box: Video Analytics with DNNs across the Camera-Cloud Boundary (Group C)
11 Intro to GPUs Watch Bridging the Edge-Cloud Barrier for Real-time Advanced Vision Analytics (Group D)
12 Intro to GPU Programming Watch Lecture 12 Files Scalable Adaptation of Video Analytics (Group A)
13 GPUs Matrix Multiplication: Behind the Scenes Watch Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices (Group B)
14 Overview: ML Resource Usage Watch Lecture 14 Files A Method to Estimate the Energy Consumption of Deep Neural Networks (Group C)
15 Final Projects Overview Watch Mainstream: Dynamic Stem-Sharing for Multi-Tenant Video Processing (Group D)
16 ML Resource Usage: Memory Watch Lecture 16 Files
17 Working Session: Project Proposal
18 Scheduling Watch MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters (Group A)
19 Scheduling: In-Class Project Themis: Fair and Efficient GPU Cluster Scheduling (Group B)
20 Working Session Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (Group C)
21 Distributed Training Pt. 1 Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs (Group D)
22 Distributed Training Pt. 2 Watch Group A's Paper Choice
23 Profiling Lecture 23 Files Group B's Paper Choice
24 Model Compression Lecture 24 Files Group C's Paper Choice
25 Working Session Group D's Paper Choice
26 ML Systems Roles

Assignments

Title Additional Resources
Assignment 1 Assignment 1 Resources
Assignment 2 Assignment 2 Resources
Assignment 3 Assignment 3 Resources
Assignment 4 Assignment 4 Resources
Assignment 5 Assignment 5 Resources

Resources

Syllabus
Gradescope
How To Read A Paper
Feedback form
Project Proposal Template
Project Report Template