Overview

Graphics Processing Units (GPUs) have become the backbone of modern machine learning systems. Understanding GPU architecture is essential for anyone working in the MLSys field.

A High-Level View

Compute Architecture

CUDA Core

Tensor Core

Ray Tracing Core

Memory Hierarchy

On-Chip Memory

L1 Cache/Shared Memory

L2 Cache

Off-Chip Memory

Adavace Feature

Spatial Sharing