ECE 5765

Building AI Infrastructure: From Chip to Grid

Course Description

Course information provided by the 2026-2027 Catalog.

Modern AI runs on a new class of machine: clusters of tens of thousands of accelerators drawing megawatts of power. This course examines how that infrastructure is built and operated, layer by layer, from chip to grid. Topics include AI accelerator architecture (GPUs, TPUs, and custom silicon); memory systems and high-bandwidth interconnects; scale-up servers and scale-out clusters; distributed training and inference serving systems; datacenter design, power delivery, and cooling; and the interface between datacenters and the electric grid, including energy efficiency, embodied and operational carbon, and sustainability. Students analyze real systems through performance modeling, case studies, and hands-on projects, and learn to reason about cost, performance, and energy across the computing stack.

Last 4 Terms Offered (None)

Learning Outcomes

• Analyze the performance, power, and cost of AI hardware and systems — from individual accelerators to multi-thousand-GPU clusters — using quantitative techniques such as roofline analysis, performance modeling, and workload characterization.
• Evaluate trade-offs across the AI infrastructure stack — accelerator and memory-system choices, distributed training and inference serving strategies — and justify design decisions under performance, cost, and energy constraints.
• Design and defend a datacenter-scale deployment plan for a given AI workload, sizing compute, networking, power delivery, and cooling against capacity, cost, and reliability requirements.

View Enrollment Information