Realistic rasterization — PBR, shadows, and global illumination

Lesson 07 gave us local shading and nothing more: a couple of point lights, an ad-hoc Blinn–Phong highlight, a normal per fragment — and then it stopped. No shadows. No bounced light. No principled materials. But a real photograph carries all of it — the soft shadow under a chair, the red of a wall bleeding onto a white floor, gold that looks like gold under any sky. This lesson climbs the realism ladder that turns a flat-shaded model into a photoreal render: physically-based materials, shadow mapping, global illumination, and the linear-light pipeline that ties them together. All of it still rides the same hard rasterizer from lessons 06–07 — non-differentiable, and entirely hand-authored — which is exactly the ceiling that will force the inverse turn to NeRF in lesson 09.

1 · The target: the rendering equation

Before we add tricks, name the thing they approximate. All of physically-correct rendering is a single equation — Kajiya's rendering equation — that says how much light leaves a surface point in a given direction. The outgoing radiance L_o at a point x in direction ω_o is what the surface emits plus everything it reflects:

Read it left to right. L_e is emitted radiance (zero unless the surface is a light). The integral runs over the hemisphere Ω of all incoming directions above the surface. Inside it: the BRDF f_r — the bidirectional reflectance distribution function from lesson 07 §3, "how does this material scatter light arriving from ω_i out toward ω_o?" — times the incoming radiance L_i from that direction, times a cosine foreshortening factor (ω_i · n).

That cosine is Lambert's cosine law, and it is worth pausing on: light arriving at a grazing angle spreads its energy over a larger patch of surface, so each point receives less of it. A beam hitting head-on (ω_i = n, cosine 1) delivers full power; the same beam at 60° off-normal (cosine 0.5) delivers half. This is the exact n·l term that showed up in lesson 07's diffuse formula — now revealed as one factor inside the full integral.

Every renderer is an approximation of this one integral. The three we care about differ only in how they attack it:

So lesson 07's local shading is not "a different model" — it is the crudest possible truncation of this equation: keep the cosine, replace the BRDF integral with a couple of point-light evaluations, and silently set the visibility to 1 everywhere. The rest of this lesson is a guided tour of putting the dropped pieces back — the BRDF (§2), the visibility term (§3), and the bounced-light term (§4) — while staying inside the real-time budget of the rasterizer.

2 · PBR materials — the microfacet BRDF

Lesson 07 shaded with an ad-hoc Blinn–Phong lobe: a diffuse color, a specular color, and a shininess exponent you hand-tuned until it "looked right" under your test light. Move the object to a different scene and it looks wrong — too shiny, too matte, the wrong tint on the highlight. The fix is to stop guessing and use a BRDF derived from physics. That is physically-based rendering (PBR), and its workhorse is the microfacet model.

The idea. A real surface is not smooth at the scale of light. It is millions of tiny mirrors — microfacets — each perfectly reflective but oriented randomly. A polished surface has facets nearly all aligned with the macroscopic normal n (so reflections are sharp); a rough surface has facets scattered in orientation (so reflections are blurred). The microfacet specular BRDF, in its Cook–Torrance form, is:

Three factors, each answering a physical question about those microfacets. Let h = normalize(ω_i + ω_o) be the half-vector (the microfacet normal that would mirror the light straight into the eye):

• D — the normal distribution function (NDF). "What fraction of microfacets are oriented to reflect the light toward me?" — i.e. how many face along h. The modern choice is GGX / Trowbridge–Reitz, controlled by roughness α. Small α concentrates the facets near n → a tight, bright mirror highlight; large α spreads them → a broad, dim, matte sheen. This is the single knob that replaces Blinn–Phong's shininess exponent, and it does so with a physically meaningful long tail that Blinn–Phong lacks.
• F — the Fresnel term. "How reflective is the surface at this viewing angle?" The physical fact: everything becomes a mirror at grazing angles. Look straight down into calm water and you see the bottom; look across it at a shallow angle and it is a mirror of the sky. The Schlick approximation captures it cheaply:
  F = F₀ + (1 − F₀)(1 − (h·ω_o))⁵
  where F₀ is the reflectance at head-on incidence and the (1−·)⁵ drives it toward 1 at the silhouette.
• G — the geometry (shadowing–masking) term. "At this angle, how many microfacets shadow or occlude each other?" At grazing angles a facet's neighbors block its light coming in (shadowing) and its light going out (masking). The Smith formulation splits this into the two directions and multiplies them. It keeps the BRDF from over-brightening at grazing angles and is what makes energy add up.

The 4(ω_o·n)(ω_i·n) in the denominator is a normalization from the change of variables between microfacet and macroscopic geometry. In code you guard it with a small ε so a grazing fragment doesn't divide by zero.

The metallic–roughness workflow

PBR would be no improvement if it needed a physicist to author. Its triumph is a tiny, intuitive parameter set — the metallic–roughness workflow (glTF, Unreal, Unity all use it):

The physics collapses cleanly onto these three because of a real dividing line in nature — metals versus everything else (dielectrics: plastic, wood, skin, stone, paint):

Two worked constants to memorize. A dielectric has F₀ ≈ 0.04 — that is why a plastic ball has a small white highlight regardless of its body color. A metal has F₀ = base color and no diffuse — which is the whole reason gold reflects gold: its specular highlight and reflections are tinted by the metal's own color, not white. In shader code this is one mix:

Energy conservation. A surface cannot reflect more light than it receives, so diffuse + specular must not exceed the incoming energy: diffuse + specular ≤ 1. The metallic term enforces the most important half of this automatically — as a material goes metallic, its diffuse fades to zero exactly as its tinted specular takes over. Ad-hoc Blinn–Phong had no such guarantee; you could (and people did) author materials that emitted more light than fell on them, which reads as "fake" the moment the lighting changes.

3 · Shadows — shadow mapping

Look again at the rendering equation's integrand: L_i(x, ω_i) is the light actually arriving from direction ω_i. If something blocks the path between x and the light, L_i is zero from that direction — the point is in shadow. Lesson 07's local shading quietly assumed every point sees every light (visibility always 1). Real light is occluded constantly, and shadows are one of the strongest cues the eye uses to read a scene's geometry. So we need a visibility term. The rasterizer's answer is beautifully economical: reuse the z-buffer from lesson 06, run from a second viewpoint.

Shadow mapping — two passes.

1. Depth pass, from the light. Put the camera at the light and render the scene, but keep only the depth buffer — the distance to the nearest surface along each of the light's rays. Store it as a depth texture: the shadow map. This is exactly the z-buffer of lesson 06 §4, just rendered from a different eye point. It answers, for every direction the light looks, "how far is the closest thing I can see?"
2. Shading pass, from the camera. Now render normally from the real camera. For each fragment at world point x, transform it into the light's clip space to find (a) where it lands in the shadow map and (b) its depth as seen from the light. Compare: if the fragment's light-space depth is greater than the stored nearest depth at that texel, something else was closer to the light — the fragment is occluded, so it is in shadow (skip its contribution from this light). If the depths match, the fragment is the closest surface, so it is lit.

That comparison — "am I farther from the light than the nearest thing the light could see?" — is the entire visibility test, and it is one texture lookup plus one subtraction per fragment per light.

Artifacts and their fixes

The idea is clean; the finite-resolution reality has three classic failure modes, and knowing the fix for each is a standard interview beat:

For a large view frustum a single shadow map can't be sharp everywhere — nearby shadows want high resolution, distant ones don't. Cascaded shadow maps (CSM) split the view frustum into depth slices and give each its own shadow map, so resolution follows where you're looking. The whole apparatus is, once more, the same rasterizer run a second time — no new machinery, just a clever reuse of coverage + z-buffer to recover the visibility term the rendering equation demanded.

4 · Global illumination — the bounce rasterization can't do in one pass

Even with a principled BRDF and correct shadows, one thing is still missing, and it is the term that makes rendered images feel lit rather than painted. In the rendering equation, L_i — the light arriving at a point — is not just light straight from a lamp. It includes light that bounced off other surfaces first. A red wall bleeds red onto a white floor beside it (color bleeding). The inside of a crevice is darker than the open surface because ambient light from the environment is blocked from reaching it (ambient occlusion). A mirror shows the room. That entire recursive, scene-wide transport is global illumination (GI).

The honest integral of §1 is expensive because the bounced L_i is itself an outgoing radiance from somewhere else — the equation is recursive. Real-time rendering approximates the bounce term with a toolbox of cheats, each trading a slice of correctness for speed:

Notice the throughline: IBL and SSAO both approximate the ambient (indirect diffuse) slice; SSR and probes approximate the specular slice; lightmaps bake the static part offline; and RTX stops faking and traces the real thing where the budget allows. Hybrid ray tracing is the telling one — it is the two renderers from lesson 06 (rasterizer + ray tracer) finally sharing a frame, rasterizing primary visibility for speed and tracing secondary rays for the transport a raster pass structurally cannot do. Frame the whole section this way: the honest integral of §1 is expensive, and every technique here is an approximation of its bounce term.

5 · The pipeline reality: linear light, HDR, tone mapping, AA/post

You can have a perfect BRDF, correct shadows, and good GI, and the image can still look subtly wrong — muddy midtones, dirty edges — because of the color space the math ran in. Two facts about the pipeline are non-negotiable.

HDR and tone mapping. Lighting in linear space produces high dynamic range radiance — a sunlit highlight or a bright bulb has a value well above 1.0, because real luminances span many orders of magnitude. But a display shows only [0, 1]. Clamping (min(color, 1)) destroys all detail in the brights — a window blows out to a flat white rectangle. Instead you tone-map: apply a curve (Reinhard, or the filmic ACES curve, scaled by an exposure control) that gently compresses the HDR range into [0, 1], preserving highlight detail the way film does. Tone mapping happens after all linear lighting and before the sRGB encode.

Antialiasing and post. Finally, the modern pipeline layers on the filmic polish: temporal antialiasing (TAA, which reuses samples across frames to smooth edges — the successor to lesson 07's MSAA), then post-process effects — bloom (glow around HDR highlights), depth of field, motion blur, and color grading. These don't add physical correctness; they add the final layer of realism the eye reads as "photographic." Order matters throughout: decode → light in linear/HDR → tone-map → encode to sRGB → antialias + grade.

6 · Where forward rendering ends → the inverse pivot

Stand back and look at what we have built across lessons 06–08. We can turn a hand-modeled scene into a photorealistic image: coverage and a z-buffer (06), perspective-correct interpolation and per-fragment shading (07), and now principled materials, shadows, GI, and a linear-HDR pipeline (08). This is the summit of forward rendering. And at the summit, two hard limits come into focus — the same two the last lesson foreshadowed.

• It is still the non-differentiable hard rasterizer. Nothing in this lesson touched the core machinery of lesson 07 §5: coverage is a hard argmax, the z-test a hard min, so ∂(pixel color)/∂(vertex position) is zero almost everywhere and undefined at edges. A prettier BRDF doesn't change the fact that you cannot backpropagate an image loss into the geometry through this pipeline.
• It is entirely hand-authored. Every input we've assumed came from a human: an artist modeled the mesh, painted the albedo/normal/roughness maps, and placed the lights. Photorealism here is a authoring achievement, not a capture one.

Now flip the goal. Suppose you don't want to author a scene — you want to capture a real one, from nothing but photographs. No modeling, no painting, no light placement: just images of a real object, and out the other end a renderable 3D scene. You cannot get there by running this forward pipeline harder. You have to run it backward — invert the renderer: define a scene representation with free parameters, render it, compare to the photos, and use the gradient of that photometric loss to update the parameters by gradient descent until the renders match. That loop is exactly the inverse-rendering loop lesson 01 promised — and it demands a differentiable renderer, which the hard rasterizer is not.

// Per-pixel Cook-Torrance on a sphere disk of radius R centred on the canvas.
// v = view dir (toward camera); l = light dir from the angle slider; h = half-vector.
var v = [0, 0, 1];
var l = norm([Math.sin(theta), 0.40, Math.cos(theta)]);   // theta = light-angle slider
var a = rough * rough;                                     // GGX uses alpha = roughness^2
// F0: dielectric = 0.04 (white); metal = base colour (tinted). Diffuse fades with metallic.
var F0  = [ mix(0.04, base[0], metal), mix(0.04, base[1], metal), mix(0.04, base[2], metal) ];
var kd  = 1 - metal;                                       // no diffuse for pure metal

for (each pixel (px,py) with r2 = px*px + py*py <= R*R) {
  var nx = px / R, ny = -py / R, nz = Math.sqrt(1 - r2 / (R*R));   // reconstruct normal
  var n = [nx, ny, nz];
  var h  = norm([l[0]+v[0], l[1]+v[1], l[2]+v[2]]);
  var nl = Math.max(dot(n,l), 0), nv = Math.max(dot(n,v), 1e-4);
  var nh = Math.max(dot(n,h), 0), hv = Math.max(dot(h,v), 0);

  // D — GGX / Trowbridge-Reitz
  var dn = nh*nh*(a*a - 1) + 1;
  var D  = (a*a) / (Math.PI * dn * dn + 1e-7);
  // F — Schlick (per channel);  F = F0 + (1-F0)(1 - h·v)^5
  var fp = Math.pow(1 - hv, 5);
  // G — Smith with Schlick-GGX, k = (rough+1)^2 / 8
  var k  = (rough + 1)*(rough + 1) / 8;
  var G  = (nv/(nv*(1-k)+k)) * (nl/(nl*(1-k)+k));

  for (each RGB channel c) {
    var F    = F0[c] + (1 - F0[c]) * fp;
    var spec = D * F * G / (4 * nv * nl + 1e-4);            // Cook-Torrance specular
    var diff = kd * base[c] / Math.PI;                      // Lambert diffuse
    var col  = (diff + spec) * LIGHT[c] * nl + 0.03*base[c]; // + tiny ambient
    col = col / (1 + col);                                  // Reinhard tone-map
    out[c] = Math.pow(col, 1/2.2) * 255;                   // gamma encode to sRGB
  }
  setPixel(px, py, out);
}

Where this points next

We have reached photorealism on the rasterizer — a principled microfacet BRDF, shadows from a reused z-buffer, faked global illumination, and a linear-light HDR pipeline with tone mapping. But we did it by authoring everything (mesh, materials, lights) and running a non-differentiable forward pass. Same rendering equation, but only ever in the forward direction. Lesson 09 takes the opposite turn: instead of authoring a scene and rasterizing it, capture a real scene from photographs by inverting the renderer. This needs a representation that is differentiable by construction — colored, semi-transparent density integrated along each ray, so an image loss flows all the way back into the scene by gradient descent. That is the radiance field of NeRF, and it finally closes the represent → render → compare → backprop loop lesson 01 promised and this forward pipeline structurally cannot. Put crisply: forward realism = author + rasterize; inverse realism = capture + optimize. Lesson 10 (3D Gaussian Splatting) will then bring that inverse idea back onto a — now soft, now differentiable — rasterizer.

Interview prompts

• Write the rendering equation and locate lesson 07's local shading inside it. (§1 — L_o = L_e + ∫_Ω f_r·L_i·(ω_i·n) dω_i; local shading drops the hemisphere integral, sums a few point lights with an ad-hoc f_r, and assumes visibility = 1. The cosine is Lambert's law.)
• Why does gold's highlight look gold but a plastic ball's look white? (§2 — metals have F₀ = base color and no diffuse (tinted specular); dielectrics have F₀ ≈ 0.04 (achromatic specular) plus a colored diffuse. Metallic selects between them.)
• Name the three terms of the Cook–Torrance BRDF and what each controls. (§2 — D (GGX) = microfacet distribution, driven by roughness (tight vs broad highlight); F (Schlick) = grazing-angle reflectance rim; G (Smith) = shadowing/masking; over 4(ω_o·n)(ω_i·n).)
• How does shadow mapping work, and what causes shadow acne vs peter-panning? (§3 — render depth from the light, then in shading compare the fragment's light-space depth to the stored nearest depth; acne = self-shadowing from depth quantization, fixed with a bias; peter-panning = over-bias detaching the shadow; jaggies → PCF, big frusta → CSM.)
• Why can't a single rasterization pass do global illumination, and how is the bounce faked? (§4 — a raster pass shades a fragment knowing only itself + the lights, so it gets direct lighting only; the recursive bounced L_i is approximated by SSAO, IBL, baked lightmaps/probes, SSR, or hybrid RTX ray tracing.)
• Why must lighting be computed in linear space, and what happens if you don't? (§5 — light adds linearly but sRGB is gamma-encoded (L≈V^2.2, CV lesson 01); decode textures to linear, light, re-encode. Skipping it gives too-dark midtones and muddy/wrong-colored blends. Lighting also produces HDR, which must be tone-mapped, not clamped.)