CS 59200: Datacenter Memory Systems

Fall 2026, Department of Computer Science, Purdue University

Instructor: Midhul Vuppalapati
Days/Time: Tue, Thu 4:30pm-5:45pm
Location: SCHM 307

Course Overview

Main memory has emerged as a critical bottleneck resource in modern datacenters. Memory capacity and bandwidth bottlenecks have resulted in unsustainable costs for datacenter operators and severe performance impact for modern applications (e.g., key-value stores, data analytics, graph processing). Memory bottlenecks are even more acute for AI applications running on GPUs (e.g., LLM inference). In this course, we will explore approaches to mitigate and manage datacenter memory bottlenecks across the hardware and software stack spanning operating systems, distributed systems and computer architecture.

This seminar-style course will cover a wide range of topics, including but not limited to: OS memory management, memory offloading, tiering and disaggregation, address translation optimizations, memory bandwidth management, LLM inference memory management, and new memory technologies. Topic discussions will primarily be guided by recent research papers in top-tier OS, distributed systems and computer architecture conferences.

Target Audience

This course is primarily intended for PhD students in computer science who are interested in pursuing research in one or more of the operating systems, distributed systems and computer architecture areas.

MS students and undergraduates interested in the course material are also welcome to join. The following prerequisites are required for undergrads: a grade of B or higher in CS 35400.

Schedule

Note that the below schedule and reading list is subject to change. Please keep an eye on the course website and Brightspace/email announcements for updates.

Date	Topic	Readings
Warmup
Aug 25	No class
Aug 27	No class
Sep 1	Course introduction and preliminaries [slides]	Optional How to read a paper, You and your research
Sep 3	Memory technology trends	Required Memory Wall Optional AI and Memory Wall
Classical OS memory management
Sep 8	Linux memory management	Required OSTEP ch. 13, 15, 17, 18, 19, 20, 21, 22, 23.2 Optional Understanding the Linux Virtual Memory Manager
Sep 10	NUMA memory management [slides]	Required Verghese Optional AutoNUMA
Sep 15	VM memory management [slides]	Required Memory Resource Management in VMware ESX Server
Memory offloading, tiering and disaggregation
Sep 17	Saving memory capacity by offloading [slides]	Required TMO
Sep 22	Offloading policies	Required MDK
Sep 24	Memory disaggregation or pooling	Required Infiniswap, Pond Optional Mage
Sep 29	OS memory tiering [slides]	Required HeMem, TPP Optional Colloid
Oct 1	Hardware-managed memory tiering	Required JohnnyCache Optional Memstrata
Oct 6	Interconnects for memory disaggregation	Required Octopus
Virtual to physical address translation
Oct 8	Pages vs. hugepages	Required Ingens Optional MEMTIS
Oct 9	Initial project proposal due
Oct 13	Fall break
Oct 15	TLB compression	Required Mosaic pages
Oct 20	Mid-term exam
Oct 22	Address translation for VMs	Required DMT
Memory bandwidth problems
Oct 27	Industry observations on memory bandwidth bottlenecks	Required Limoncello, Necro-reaper
Oct 29	Interplay between host interconnects	Required Understanding the Host Network Optional hostCC
Nov 3	DDR and DRAM architecture	Required Memory Systems and Memory-Centric Computing
Nov 5	Memory controller scheduling	Required STFM
GPU/AI memory bottlenecks
Nov 10	Memory capacity and bandwidth limits for LLM inference	Required Brief background, LIMINAL
Nov 12	Inference memory management: Paged attention	Required vLLM, vAttention Optional vLLM anatomy
Nov 13	Mid-term report due
Nov 17	Inference memory management: Long context	Required Strata Optional ECHO
Nov 19	Inference memory management: Heterogeneous models	Required Jenga
Nov 24	Project feedback (optional class)
Nov 26	Thanksgiving break
Final project presentations
Dec 1	Project presentations
Dec 3	Project presentations
Looking into the future
Dec 8	Processing in memory for inference	Required CENT
Dec 10	New memory technologies	Required MRAM, Lt/St-RAM
Dec 15	Final report due

Logistics

The course is structured around student-led presentations and discussion held during weekly sessions, with the instructor providing guidance and facilitating exploration of the material. Course evaluation is based on in-class presentations, one assignment, one midterm exam, and a research project.

Paper presentations

Each student is required to select a single class during which they will present the paper(s) listed as required readings for that class. Presenting for more than one class is welcome, but not required.

A signup sheet for selecting a presentation slot will be shared via Brightspace.
Length: target 45-50 minutes, leaving enough time for discussion.
Must cover: the motivation and key technical contributions of the paper(s).
Should ideally also cover: brief background necessary for understanding the paper, and a brief overview of related work.
Nice to have (optional): if time permits, briefly summarize the optional readings toward the end of your presentation and say how they relate to the required readings.
Important: meet with the instructor at least one week before your presentation to go over a draft of your slides. Please email the instructor to schedule this, well in advance.
Draft slides do not need to be polished. A rough first draft is sufficient. In fact, an overly polished draft can be counter-productive, since it makes any suggested changes harder to incorporate.

The exact assignments and their details will be announced during the course of the class. Below is a tentative description to give students a high-level idea.

Assignment

Students can select one of three options

Option 1: Students will work on an implementation exercise that will require programming in the Linux kernel and interacting with the kernel's memory management subsystem. The exercise will be sufficiently self-contained and will not require intricate knowledge of kernel internals. Students will test their implementations for correctness and conduct basic performance measurements.
Option 2: Implement paged attention in nanochat's (https://github.com/karpathy/nanochat) inference engine. Students will test their implementations for correctness and conduct basic performance measurements.
Option 3: Implement a given memory controller scheduling policy in a DRAM simulator (Ramulator 2.0). Students will evaluate implemented policies with synthetic memory access traces and report results.

Midterm

The midterm will be an in-class oral examination. Please see the Brightspace announcement for details.

Final project

Students can choose one of two options, described below. In both cases, a proposal, a mid-term report, a presentation, and a final report are required.

Option A: Research project. Motivate a research problem, identify limitations of related work, propose a solution or approach, and make progress toward implementing and evaluating the proposed solution or approach. This option can be done individually or in groups of two.

Given the limited time frame of the class, it is not expected that students execute a full research project end-to-end. Instead, reasonable preliminary progress is expected.
It is acceptable to use parts of a project that you are already working on with faculty or other collaborators as long as you can connect it to aspects relevant to the course. For example, if you are designing an algorithm as part of your existing research project, you could explore ways to make the algorithm more memory efficient.
- Important: If you plan to take this route, please talk to the instructor before the initial proposal submission (also happy to brainstorm).

Option B: Survey. Identify a topic of interest relevant to the course, do a literature search for papers related to the topic, create a taxonomy or categorization of the existing literature, identify limitations of existing literature and discuss remaining open problems. This option is done individually.

Suggestion: Try to focus on papers from top-tier systems conferences (e.g., SOSP, OSDI, NSDI, SIGCOMM, ISCA, MICRO, ASPLOS, Eurosys, ATC).

Checkpoints. All the following checkpoints are required for both research project and survey. The proposal, mid-term report and final report will be submitted via Brightspace.

Due	Checkpoint	Research project	Survey
Oct 9	Proposal1 page	Motivation: What problem you want to solve and why it is important. Related work: Key related papers and why they fall short of solving the problem. Does not have to be exhaustive; just a starting point (even 1-2 papers is fine). Proposed solution or approach: High-level description of what you plan to investigate. Does not have to be perfect or 100% clear; high-level ideas or intuition are sufficient. Implementation and evaluation plan: Rough idea of how you plan to implement and evaluate your solution, including what resources (e.g., servers) you anticipate needing. A detailed implementation plan or experimental setup is not required.	Motivation: What topic you plan to focus on and why it is important. References: Papers you plan to incorporate into your survey. Does not have to be exhaustive, but you should have a reasonable initial list. Categorization: Rough idea of how you anticipate categorizing the papers (e.g., what the dimensions may be).
Nov 13	Mid-term report2-3 pages	Motivation: Expand and refine the motivation from the proposal; showing experimental data to substantiate your claims is a great way to do this. Related work: Expand and refine the discussion. Include additional papers you found and articulate how each work relates to your project. Proposed solution or approach: A more detailed description; ideally the key components should be there. It is ok if some details are not yet figured out. Implementation: Brief description of the implementation plan and the current status of the implementation effort. Evaluation plan: What experiments you plan to run and what results you expect to see. Feel free to include any preliminary results you already have.	Expanded and refined motivation from the proposal. Draft of the taxonomy/categorization of papers. Discussion of limitations of the papers. Proposed next steps.
Dec 1, 3	Presentation15-20 minutes	Goal: share what you have learned so far with the class. It is not expected that your project is fully complete by this point---just share your current progress
Dec 15	Final report5-6 pages	Refine and expand on the mid-term report and incorporate evaluation results Incorporate any feedback from the presentations Polish text and figures Should ideally look like a short conference or workshop paper

Learning Objectives

Develop critical thinking in computer systems
Analyze and present research papers
Lead and participate in academic discussions
Write research papers in computer systems / architecture
Implementation experience with Linux kernel memory management, LLM inference memory management and/or architectural memory simulators

Grading Criteria

In-class presentations: 15%
Assignments: 25%
Midterm exam: 25%
Final project/paper: 35%