Part VII — Synthesis

The real stacks

For seventeen lessons you derived a pipeline from nothing but pressure: capture a graph because optimization needs the whole program (02), give it a typed SSA IR so rewrites are safe (03), fold and fuse to kill HBM round trips (04–05), plan memory and layout (06–07), schedule and codegen the loop nest (08–09), autotune the config (10), then handle dynamic shapes, autodiff, and sharding (11–13) — and finally made the result shippable: cover the long tail of ops (14), spend precision through quantization (15), launch it at run time (16), and debug what you can't trust (17). That is one chain of forced decisions, end to end from graph to a model you can build, ship, and verify. The field ships exactly that chain — but it ships it five times, under a dozen product names, each with its own IR, its own bets, and a marketing page that makes it sound unique. The through-line is easy to lose. This lesson is a Rosetta stone: line up torch.compile, XLA, TVM, MLIR, and TensorRT against the same five stages and watch the same pipeline appear, with the differences reduced to a handful of honest trade-offs.

1 · The five columns

The whole map rests on the five-stage frame from lesson 00. Restate it as the fixed columns every stack will be read against — these are the same stages, just lifted up to product names:

Two cross-cutting concerns thread through all five and are not separate columns — they are decisions each stack makes inside these stages: dynamic shapes (lesson 11) live in capture+represent (symbolic dims) and optimize+codegen (guarded specialization), and autodiff (lesson 12) and sharding (lesson 13) are graph-to-graph passes that run inside represent+optimize. Keep them in mind as the rows differ; the biggest differences between stacks are exactly where and how well they handle these. And remember the recurring trade from lesson 03: a high IR is portable but slow, a low IR is fast but locked — every stack picks a different rung of that ladder to do its heavy optimization on, and that choice is most of its personality.

2 · torch.compile — JIT, Python-native

IR: FX graph (high) → ATen/Prims (decomposed) → Inductor's internal loop-level IR → Triton/C++. Niche: training and research inside eager PyTorch, with the lowest possible friction — one decorator, your debugger still works.

The defining bet is JIT: nothing compiles until the function runs on real tensors, so Dynamo gets exact dtypes and shapes for free and can graph-break back to eager on anything it can't trace — the program never fails to run, it only fails to fully optimize. That is the Python-native niche: you keep arbitrary Python in the loop and lose only the unoptimizable slices. The cost is the flip side of the same coin — first-call latency (seconds to minutes under max-autotune), recompiles on shape changes (lesson 11), and a speedup that silently erodes if a stray print or numpy call inserts a graph break. Inductor's deliberate choice to emit Triton rather than raw CUDA (lesson 09) trades a few points of peak for portability and a tiny codegen surface. For serving, the same stack has an AOT escape hatch — torch.export → AOTInductor emits a Python-free .so, and mode="reduce-overhead" wraps the launch sequence in a CUDA graph — both squarely lesson 16's runtime concerns. The full mechanics of Dynamo/AOTAutograd/Inductor and how to read graph breaks live in gpu_kernels · 23 torch.compile; functionalization and the fwd/bwd partitioner are lesson 12, the fusion grouping it does is lesson 05.

3 · XLA / HLO — AOT, whole-program

IR: HLO (High-Level Optimizer IR) — a single, medium-level, functional op set, lowered to LLVM IR → PTX/SASS (GPU) or direct codegen (TPU). Niche: whole-program AOT compilation for JAX and TensorFlow, especially on TPU, where the entire step function is one compiled artifact.

XLA's bet is the mirror image of torch.compile's: AOT, whole-program. It traces the entire function to HLO before running anything, so it sees the whole graph at once and can fuse and schedule globally — but tracing means it shares tracing's traps from lesson 02 (Python control flow is unrolled or must be a lax.cond/scan) and it leans toward static shapes, with dynamic dimensions a comparatively newer and more limited feature. Two things flow from "single medium-level IR": optimization is powerful but monolithic — one big pass pipeline on HLO rather than a ladder of dialects — and sharding is first-class, because GSPMD (and now Shardy) is just another HLO pass that propagates shardings and inserts collectives (exactly lesson 13). On TPU this is the only game and it is superb; on GPU it competes with Inductor. The honest verdict: best-in-class for static-shape, whole-program training on TPU; less flexible than torch.compile for the messy dynamic Python of research on GPU.

4 · TVM — Relay → Ansor, deployment & edge

IR: Relay (high-level graph) → TE/TIR (tensor-expression / loop-level schedule IR) → LLVM/CUDA/C. Niche: ahead-of-time deployment across a wide hardware zoo — phones, microcontrollers, accelerators, CPUs — where you compile once on a build machine and ship a small artifact.

TVM is where the lessons-08-and-10 machinery is most visible as a named product. Its TE/TIR layer is the algorithm/schedule split from lesson 08, and Ansor (the auto-scheduler) is the search-plus-learned-cost-model from lesson 10, made into the headline feature. Because the niche is deployment, TVM is overwhelmingly AOT: you pay a long autotuning search once on a build host (minutes to hours per target), then ship a tiny, dependency-light runtime — exactly the economics that justify expensive search (amortize over a million edge-device runs). The trade is that it is import-and-deploy oriented, not a drop-in for an evolving training loop; its strength is breadth of targets and a self-contained artifact, not eager-mode ergonomics. Verdict: the strongest answer when "run this fixed model fast on that odd chip" matters more than developer iteration speed.

5 · MLIR — the infrastructure layer

IR: not one IR — a framework for many coexisting IRs called dialects (e.g. tosa, mhlo, torch, linalg, affine, vector, gpu, llvm) with shared infrastructure for passes, pattern rewrites, and progressive lowering between them. Niche: the compiler-construction substrate the other stacks increasingly build on, not a user-facing framework.

MLIR is the odd row: it doesn't compete with the others, it is the thing they're written in. It is the lesson-03 idea — multiple levels of IR, each able to express optimizations the others can't, with lowering as the move down a level — promoted from a concept to reusable infrastructure. The pattern-rewrite engine from lesson 04 is literally MLIR's PatternRewriter; the dialect ladder is the ir-ladder from lesson 03 made concrete. Concrete consumers: IREE is an end-to-end MLIR-based AOT compiler+runtime (a deployment story like TVM's), Torch-MLIR bridges PyTorch into the dialect world, and Triton itself is implemented as MLIR dialects under the hood — so when Inductor emits Triton (§2), it is feeding an MLIR-based compiler. The verdict: you rarely "use MLIR" directly, but it is quietly becoming the common middle of the whole field, which is exactly why understanding the five stages transfers — they are MLIR's stages.

6 · TensorRT — inference-only, vendor-optimal

IR: an internal network graph (built via the TensorRT builder or imported from ONNX) lowered to a serialized, hardware-specific engine. Niche: squeeze maximum inference throughput/latency out of NVIDIA GPUs, accepting a closed, vendor-locked, training-incapable artifact.

TensorRT is the most specialized row and it makes the abstraction↔specialization trade (the motif from lessons 03–09) maximally extreme: it gives up everything portable — no training, no other vendor, a closed engine you can't introspect — to buy peak inference speed on the exact GPU you built for. Its "autotuning" (the tactic selection in the build phase) is lesson 10 in spirit but draws from a library of NVIDIA's own hand-written kernels rather than generating Triton, which is why it often beats open compilers on peak but only on NVIDIA. Two passes are distinctively its own emphasis: aggressive layer/tensor fusion tuned for NVIDIA's kernels, and precision calibration — folding INT8/FP8 quantization into the build with a calibration dataset (exactly lesson 15's PTQ, made vendor-native), which the general-purpose compilers treat more cautiously. Its serialized engine plus an execution context that owns the buffers and dispatches the layers is its lesson-16 runtime, baked into one artifact. The engine is shape-specialized (it bakes in optimization profiles for shape ranges — lesson 11's bucketing made vendor-native). Verdict: when the job is "lowest-latency inference of a frozen model on NVIDIA," it is usually the fastest; for anything else (training, portability, openness) it is the wrong tool by design.

7 · The Rosetta table

The same five stages, five times. Read across a row to see one stack's bets; read down a column to see how the field disagrees on one decision. The verdicts are honest, not promotional.

Notice the columns that don't vary: every stack has all five stages, and four of the five are open source. The real disagreements are concentrated in three columns — JIT vs AOT (when do you commit to shapes?), training vs inference (do you need a backward graph?), and openness/target (portable or vendor-optimal?). Pick a stack by answering those three; the rest of the pipeline is shared.

8 · Drive it: the stack Rosetta

Pick a stack and watch the five-stage pipeline light up with that stack's component at each stage, the trait it bets on, and the lesson (FILE#) that derived the stage. The flags above the pipe summarize the three decisions that actually distinguish the row. The point the widget proves: the boxes never change, only their contents — every stack is the same chain with different fills.

Flip through all five. The exercise that lands the lesson: name, for each stack, which single column of §7's table is its reason to exist — torch.compile's is JIT/Python-native, XLA's is whole-program AOT, TVM's is the target zoo, MLIR's is being the substrate, TensorRT's is vendor-optimal inference. Five products, one pipeline, five different bets on the same five stages.

Failure modes & checklist

Checkpoint

Where this points next

You now have the map: torch.compile, XLA, TVM, MLIR, and TensorRT are the same capture → represent → optimize → lower → codegen pipeline you derived link by link, distinguished only by their IR, their niche, and three bets (JIT/AOT, train/infer, open/vendor). But a map is not a journey. You have seen every pass in isolation across seventeen lessons, and now seen them aligned across five products — yet you have never once watched a single concrete program flow through the whole chain from Python source to a running kernel, with each transform firing as the forced consequence of the last and the scoreboard (kernels, HBM bytes, peak memory) collapsing stage by stage. Seeing the map is exactly what invites doing the whole thing once, end to end. That is the capstone — one block, y = gelu(x @ w + b) with a dynamic batch and its backward, source to kernel — lesson 19, Capstone — one block, source to kernel.

Interview prompts

• Map torch.compile onto the five compiler stages. (§2 — Dynamo = capture (bytecode → FX), AOTAutograd = represent + autodiff (functionalize, decompose, build backward), Inductor = optimize (fuse/plan/layout) + lower (schedule), Triton = codegen; JIT, with graph breaks as the eager fallback.)
• What is the core bet that distinguishes XLA from torch.compile? (§3 — AOT whole-program vs JIT per-call: XLA traces the entire function to HLO before running, fusing/scheduling globally and leaning static-shape (great on TPU); torch.compile compiles lazily on real tensors and graph-breaks on what it can't trace, keeping dynamic Python alive at the cost of recompiles.)
• Why is MLIR not really comparable to the other four? (§5 — it isn't a framework but compiler infrastructure: a system of coexisting dialects with shared pass/rewrite machinery and progressive lowering — the lesson-03 multi-level-IR idea made reusable. Triton, IREE, and Torch-MLIR are built on it, so it sits under the others rather than beside them.)
• Which stack would you reach for to deploy a fixed vision model to a phone, and why? (§4 — TVM: AOT, the widest target zoo, exposes TE/TIR scheduling and Ansor autotuning to squeeze an odd chip, and ships a tiny dependency-light runtime — you pay an expensive offline search once and amortize over device runs.)
• What does TensorRT give up, and what does it buy by giving it up? (§6 — gives up training, portability, and openness (NVIDIA-only, closed engine); buys peak inference latency via aggressive NVIDIA-tuned fusion, INT8/FP8 precision calibration, and tactic selection from a hand-written kernel library — the abstraction↔specialization trade at its extreme.)
• "GSPMD," "Ansor," "epilogue fusion," "tactic selection" — which stage and which stack does each belong to? (§3–6 — GSPMD = optimize/sharding in XLA (L13); Ansor = codegen/autotune in TVM (L10); epilogue fusion = optimize in Inductor/torch.compile (L05); tactic selection = lower/codegen in TensorRT (L08–10). Every named feature is one of the five stages.)
• Given an unfamiliar new compiler, how would you understand it in five minutes? (§1, §7 — fill the five columns (which component captures, represents, optimizes, lowers, codegens), identify its single IR spine and its niche, then answer JIT/AOT, train/infer, portable/vendor — those facts predict its behavior and place it next to the stacks you know.)