AI-Sys Spring 2019
- When: Mondays and Wednesdays from 9:30 to 11:00
- Where: Soda 405
- Instructors: Ion Stoica and Joseph E. Gonzalez
- Announcements: Piazza
- Sign-up to Present: Google Spreadsheet
- Project Ideas: Google Spreadsheet
- If you have reading suggestions please send a pull request to this course website on Github by modifying the index.md file.
Course Description
The recent success of AI has been in large part due in part to advances in hardware and software systems. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. These new hardware and software systems include a new generation of GPUs and hardware accelerators (e.g., TPU and Nervana), open source frameworks such as Theano, TensorFlow, PyTorch, MXNet, Apache Spark, Clipper, Horovod, and Ray, and a myriad of systems deployed internally at companies just to name a few. At the same time, we are witnessing a flurry of ML/RL applications to improve hardware and system designs, job scheduling, program synthesis, and circuit layouts.
In this course, we will describe the latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems. The format of this course will be a mix of lectures, seminar-style discussions, and student presentations. Students will be responsible for paper readings, and completing a hands-on project. Readings will be selected from recent conference proceedings and journals. For projects, we will strongly encourage teams that contains both AI and systems students.
Course Syllabus
This is a tentative schedule. Specific readings are subject to change as new material is published.
Week | Date (Lec.) | Topic |
---|---|---|
1/23/19 ( 1 ) |
Introduction and Course OverviewThis lecture will be an overview of the class, requirements, and an introduction to what makes great AI-Systems research. Slide Links |
|
1/28/19 ( 2 ) |
Convolutional Neural Network ArchitecturesMinor Update: We have moved the reading on auto-encoders to Wednesday. Reading notes for the two required readings below must be submitted using this google form by Monday the 28th at 9:30AM. We have asked that for each reading you answer the following questions:
If you find some of the reading confusing and want a more gentle introduction, the optional reading contains some useful explanatory blog posts that may help. Links
|
|
1/30/19 ( 3 ) |
More Neural Network ArchitecturesLinks
|
|
2/4/19 ( 4 ) |
Deep Learning FrameworksLinks
|
|
2/6/19 ( 5 ) |
RL Systems & AlgorithmsLinks
|
|
2/11/19 ( 6 ) |
Application: Data Structure and AlgorithmsLinks
|
|
2/13/19 ( 7 ) |
Distributed Systems for MLLinks
|
|
2/18/19 ( 8 ) |
Administrative Holiday (Feb 18th) |
|
2/20/19 ( 9 ) |
Hyperparameter searchLinks
|
|
2/25/19 ( 10 ) |
Auto ML & Neural Architecture Search (1/2)Links
|
|
2/27/19 ( 11 ) |
Auto ML & Neural Architecture Search (2/2)Links
|
|
3/4/19 ( 12 ) |
Autonomous VehiclesLinks
|
|
3/6/19 ( 13 ) |
Deep Learning CompilersLinks
|
|
3/11/19 ( 14 ) |
Project Presentation Checkpoints |
|
3/13/19 ( 15 ) |
Application: Program synthesisLinks
|
|
3/18/19 ( 16 ) |
Distributed Deep Learning (Part 1)Links
|
|
3/20/19 ( 17 ) |
Distributed Deep Learning (Part 2)Links
|
|
3/25/19 ( 18 ) |
Spring Break (March 25th) |
|
3/27/19 ( 19 ) |
Spring Break (March 27th) |
|
4/1/19 ( 20 ) |
Application: NetworkingLinks
|
|
4/3/19 ( 21 ) |
Dynamic Neural NetworksLinks
|
|
4/8/19 ( 22 ) |
Model CompressionLinks
|
|
4/10/19 ( 23 ) |
Applications: SecurityLinks
|
|
4/15/19 ( 24 ) |
Application: Prediction ServingLinks
|
|
4/17/19 ( 25 ) |
Natural Language Processing SystemsLinks
|
|
4/22/19 ( 26 ) |
Explanability & InterpretabilityLinks
|
|
4/24/19 ( 27 ) |
Scheduling for DL WorkloadsLinks
|
|
4/29/19 ( 28 ) |
Cortical Learning and Stoica Course SummaryLinks
|
|
5/1/19 ( 29 ) |
Neural Modular Networks and Gonzalez Course SummaryLinks
|
|
5/6/19 ( 30 ) |
RRR Week (May 6th) |
|
5/8/19 ( 31 ) |
Poster Session from 9:00 to 11:00 |
|
5/13/19 ( 32 ) |
Final Reports Due
|
Projects
Detailed candidate project descriptions will be posted shortly. However, students are encourage to find projects that relate to their ongoing research.
Grading
Grades will be largely based on class participation and projects. In addition, we will require weekly paper summaries submitted before class.
- Projects: 60%
- Weekly Summaries: 20%
- Class Participation: 20%