all lessons / computer_graphics / 14 · color, HDR & tone mapping lesson 14 / 15

Color, HDR & tone mapping色彩、HDR 与色调映射

Lessons 07–11 computed radiance in linear light — an unbounded HDR quantity, where a lit floor might read 0.3 and the sun in frame reads 50000. But your monitor emits only values it can represent as [0,1] in sRGB. Something has to carry physical light down to a displayable pixel without lying about it. That something is the color pipeline, and this lesson is it: keep all light linear while you compute, choose an exposure, tone-map the highlights that exceed 1 instead of clipping them, and encode to sRGB only at the very last step. This closes the promise of lesson 01 — that "an image is measured light." 第 07–11 课在线性光中计算辐射亮度——一个无上界的 HDR 量:被照亮的地面也许读作 0.3,而画面里的太阳读作 50000。可你的显示器只能发出它能表示为 sRGB [0,1] 的值。总得有某样东西,把物理的光一路送到可显示的像素,又不歪曲它。那样东西就是色彩流水线,本课就是它:计算时把所有光保持为线性,选定一个曝光,对超过 1 的高光做色调映射而非硬裁剪,并只在最后一步编码到 sRGB。这兑现了第 01 课的承诺——“图像是被测量的光”。

The plan本课计划

Five moves. (1) Insist that all lighting math happens in linear light, because adding, averaging, filtering, and blending are only physically correct on linear radiance — encode to sRGB only at output. (2) Accept that radiance is HDR: unbounded, so we store float framebuffers and cannot merely clip. (3) Pick an exposure — a single multiplier, the digital aperture/shutter/ISO — that decides the image's key before mapping. (4) Apply a tone-mapping operator that squeezes [0,∞) into [0,1]: clipping, Reinhard, or a filmic/ACES S-curve. (5) Encode & display: the sRGB OETF last, plus a word on gamut, white point, bloom, and true-HDR (PQ/HLG) displays. The golden rule falls out: light in linear, tone-map between, encode at output.

五步走。(1) 坚持所有光照运算都在线性光中进行,因为加法、平均、滤波与混合只有对线性辐射亮度才物理正确——只在输出时才编码到 sRGB。(2) 承认辐射亮度是 HDR 的:无上界,所以要用浮点帧缓冲存储,且不能只是裁剪。(3) 选一个曝光——一个乘数,数字世界里的光圈/快门/ISO——它在映射前定下图像的基调。(4) 施加一个色调映射算子,把 [0,∞) 压进 [0,1]:硬裁剪、Reinhard,或一条胶片风/ACES 的 S 曲线。(5) 编码与显示:sRGB OETF 放在最后,再讲一句色域、白点、辉光(bloom)以及真 HDR(PQ/HLG)显示器。黄金法则随之浮现:光在线性里算,中途做色调映射,输出时才编码。

1 · Render in linear light, not gamma在线性光里渲染,而不是在 gamma 里

The rendering equation of lesson 01 is physics: Lo = Le + ∫ fr Lii·n) dωi. Every operation in it — adding two light contributions, integrating over a hemisphere, multiplying by a reflectance — is arithmetic on radiance. Radiance adds linearly: two lamps of intensity a and b together deliver a + b photons, always. That linearity is the one non-negotiable assumption of the whole light-transport machine.

第 01 课的渲染方程是物理:Lo = Le + ∫ fr Lii·n) dωi。其中的每一步——把两份光贡献相加、在半球上积分、乘以一个反射率——都是对辐射亮度的算术。辐射亮度线性相加:强度为 ab 的两盏灯合起来总是送来 a + b 份光子,恒真。这条线性性,是整台光传输机器唯一不可让步的前提。

But an 8-bit image file, and the number your monitor's electronics expect, are not linear. Recall the CV track's lesson 01: displays and image formats store values through a gamma curve, roughly Vencoded = L1/2.2, because human vision is more sensitive in the darks and this allocates code values where the eye can see them. The stored number is perceptual, not physical. Do lighting math on those perceptual numbers and you are adding square-rootish quantities that do not represent photons — the sum is simply wrong.

但一个 8 位图像文件、以及你显示器电路所期待的数,都不是线性的。回忆 CV 课的第 01 课:显示器与图像格式通过一条 gamma 曲线存值,大致是 V编码 = L1/2.2,因为人眼在暗部更敏感,这样能把码值分配到眼睛看得见的地方。存下来的数是感知的,不是物理的。在这些感知数上做光照运算,你就是在把一些带开方味道的、并不代表光子的量相加——那个和干脆就是错的。

linear L  ⇄  sRGB encode ·   V ≈ L1/2.4 (piecewise)    L ≈ V2.4 · add / filter / blend ONLY on L

Concretely: mid-gray sRGB 0.5 decodes to a linear reflectance of about 0.214, not 0.5. So averaging two pixels in sRGB is not averaging light. Two worked traps that bite everyone:

具体地说:中灰 sRGB 0.5 解码出的线性反射率约为 0.214,而非 0.5。所以在 sRGB 里对两个像素取平均,并不是在对光取平均。两个人人都会中招的实例陷阱:

The recurring bug: wrong color space反复出现的 bug:色彩空间搞错

"Too dark / haloed / muddy" is almost always a linearity bug. The discipline is simple and absolute: decode textures to linear on read, do every light computation in linear, encode to sRGB once at the final write. Modern GPUs do the decode/encode for free via sRGB texture and framebuffer formats, so the cost is zero if you flip the right flags. The moment you average, filter, blur, or blend in sRGB, you have introduced an error that no later tweak fully removes.

“太暗 / 有光晕 / 发脏”几乎总是线性性的 bug。纪律简单而绝对:读取时把纹理解码到线性,一切光照计算都在线性中做,只在最终写出时编码一次到 sRGB。现代 GPU 通过 sRGB 纹理与帧缓冲格式免费完成解码/编码,只要把正确的开关打开,代价为零。你一旦在 sRGB 里做了平均、滤波、模糊或混合,就引入了一个后续任何微调都无法完全消除的误差。

2 · HDR — radiance is unbounded, a display is notHDR——辐射亮度无上界,显示器却有

In linear light there is no maximum. A comfortably lit interior wall might sit near 0.2. A bright sky is a few units. A lightbulb filament, a specular glint off chrome, or the sun are hundreds, thousands, or more. Nothing in the physics says "stop at 1" — the 1.0 ceiling is a property of the display, not of light. This mismatch defines the problem: the scene has high dynamic range (HDR); the output device has low dynamic range.

在线性光里没有上限。一面照明舒适的室内墙壁也许在 0.2 附近。明亮的天空是几个单位。灯丝、铬件上的镜面高光、或太阳,则是几百、几千甚至更多。物理里没有任何东西说“到 1 为止”——那个 1.0 上限是显示器的属性,不是光的。这种错配定义了问题:场景有高动态范围(HDR),而输出设备是低动态范围。

Two consequences. First, you cannot store the framebuffer in 8-bit integers [0,255] while lighting — a specular highlight of 50 would saturate to 255 and lose all the information you need to shape it later. So intermediate render targets are floating point: 16-bit half-float (fp16) is the standard HDR framebuffer — about 11 bits of mantissa, a huge exponent range, half the memory and bandwidth of fp32. Second, because the display physically cannot emit values above its peak, those bright values must be mapped down into [0,1]. The only question is how.

两个后果。其一,光照期间不能用 8 位整数 [0,255] 存帧缓冲——一处 50 的镜面高光会饱和成 255,把你之后塑形所需的信息全部丢失。所以中间渲染目标是浮点的:16 位半精度浮点(fp16)是标准的 HDR 帧缓冲——约 11 位尾数、巨大的指数范围、只占 fp32 一半的内存与带宽。其二,因为显示器物理上无法发出超过其峰值的值,那些明亮的值必须被映射[0,1]。唯一的问题是怎么映射

Why not just clip?为什么不直接裁剪?

The naive answer — min(L, 1) — throws away every distinction above 1. A sky at 4 and a sun at 4000 both become pure white, so a scene loaded with bright content flattens into featureless blobs of 1.0. Worse, clipping is per-channel, so it shifts hue: an orange highlight (6, 3, 1) clips to (1, 1, 1) — the red and green pin first, and the fireball turns white instead of staying warm. Highlights must roll off gracefully, which is exactly what tone mapping (§4) provides.

最朴素的答案——min(L, 1)——把 1 以上的一切区别都扔掉。4 的天空和 4000 的太阳都变成纯白,于是充满明亮内容的场景被压成一团团毫无细节的 1.0。更糟的是,裁剪是逐通道的,因此会偏色:一处橙色高光 (6, 3, 1) 被裁成 (1, 1, 1)——红与绿先被顶到上限,火球于是发白,而不再是暖色。高光必须优雅地滚落,这正是色调映射(第 4 节)所提供的。

3 · Exposure — the digital aperture that sets the key曝光——定下基调的数字光圈

Before choosing how to compress highlights, you choose how much light to admit. A real camera does this with aperture, shutter speed, and ISO; in rendering it collapses to one scalar exposure multiplier applied to linear radiance before the operator:

在选定如何压缩高光之前,你要先选让多少光进来。真实相机用光圈、快门速度和 ISO 来做这件事;在渲染里,它坍缩成一个标量曝光乘数,在算子之前施加到线性辐射亮度上:

Lexposed = e · Llinear     y = operator( Lexposed )     (exposure BEFORE the operator · 曝光在算子之前)

Doubling e is one photographic "stop" — twice the light. Turn it up and a dim night scene lifts into view; the price is that midtones climb toward the ceiling and more of the frame clips. Turn it down and a blazing daylight scene tames its highlights at the cost of crushing the shadows. Exposure is a slider along the scene's dynamic range, choosing which slice the [0,1] output will resolve — the key of the image (dark/low-key vs bright/high-key).

e 翻倍就是摄影里的一“档”(stop)——光量加倍。调高它,昏暗的夜景就被提亮到可见;代价是中间调向上限攀升、画面里更多地方被裁剪。调低它,烈日场景的高光被驯服,代价是压死暗部。曝光是沿着场景动态范围滑动的一个滑块,选择 [0,1] 输出将解析出哪一段——也就是图像的基调(暗调/低调 vs 亮调/高调)。

Auto-exposure automates the choice: measure the scene's average luminance — usually as the geometric mean, i.e. the average of log(luminance), because brightness is perceived logarithmically — then pick e so that average lands on a target middle-gray (around 0.18, the classic "18% gray card"). This is why stepping from a dark cave into daylight in a game blooms white for a moment, then settles: the metering is chasing a new average, exactly like your own pupils.

自动曝光把这个选择自动化:测量场景的平均亮度——通常取几何平均,即 log(亮度) 的平均,因为明暗是按对数被感知的——再选 e,使该平均落到目标中灰(约 0.18,经典的“18% 灰卡”)。这就是游戏里从黑暗洞穴走进日光时会先泛白一瞬、再稳定下来的原因:测光正在追逐一个新的平均值,跟你自己的瞳孔一模一样。

4 · Tone-mapping operators — fitting [0,∞) into [0,1]色调映射算子——把 [0,∞) 塞进 [0,1]

A tone-mapping operator is a function y = f(x) that takes exposed linear radiance x ≥ 0 and returns a displayable y ∈ [0,1]. A good operator is 0 at 0, monotonically increasing (brighter input never darkens output), roughly linear for small x (dark tones keep their contrast), and asymptotic to 1 (arbitrarily bright input approaches, but never exceeds, white). Three you must know:

色调映射算子是一个函数 y = f(x),它接收曝光后的线性辐射亮度 x ≥ 0,返回可显示的 y ∈ [0,1]。一个好算子在 0 处为 0、单调递增(更亮的输入绝不使输出变暗)、在小 x 处近似线性(暗调保住其对比度),并渐近于 1(任意亮的输入趋近但永不超过白)。三个必须掌握的:

OperatorFormulaBehavior
Clipf(x) = min(x, 1)Linear then a hard wall; blows out and shifts hue above 1.
Reinhardf(x) = x / (1 + x)Simple, soft; asymptotes to 1 but never truly reaches it — whites look slightly gray.
ACES (approx)f(x) = clamp( x(2.51x+0.03) / (x(2.43x+0.59)+0.14), 0, 1 )Filmic S-curve; toe + shoulder, preserves highlight detail and contrast — the "film look."
算子公式行为
硬裁剪 Clipf(x) = min(x, 1)先线性、再撞上一堵硬墙;在 1 以上过曝并偏色。
Reinhardf(x) = x / (1 + x)简单、柔和;渐近于 1 却永不真正到达——白色看着略灰。
ACES(近似)f(x) = clamp( x(2.51x+0.03) / (x(2.43x+0.59)+0.14), 0, 1 )胶片风 S 曲线;带趾部与肩部,保住高光细节与对比度——即“胶片感”。

Read the shapes. Reinhard x/(1+x) maps 1→0.5, 4→0.8, 9→0.9 — it compresses everything smoothly, which is why nothing clips but also why bright whites never quite reach 1.0 and the whole image reads a touch desaturated and low-contrast. ACES adds a toe (gentle lift near black for richer shadows) and a shoulder (a steeper-then-flattening roll-off that keeps highlight separation while heading to white), matching how film stock responds and why it "looks cinematic." Clip is the null operator: correct below 1, catastrophic above it.

读它们的形状。Reinhard x/(1+x)1→0.54→0.89→0.9——它平滑地压缩一切,这既是它不裁剪的原因,也是明亮的白永远差一点到 1.0、整幅图看着略微发灰、对比偏低的原因。ACES 加了一段趾部(近黑处轻微抬升,让暗部更醇厚)与一段肩部(先陡后平的滚落,在奔向白色的同时保住高光间的区分度),契合胶片的响应方式,也是它“看着很电影”的原因。Clip 是空算子:1 以下正确,1 以上灾难。

A subtlety: applied per channel, a strong operator can still desaturate bright colors (all three channels get pushed toward 1 together). Production pipelines often tone-map on luminance and re-apply the original chroma, or use full ACES with its own color transforms, to keep saturated highlights saturated. And operators split into global (one curve for every pixel — everything above) versus local (the curve adapts to a pixel's neighborhood, like the eye's local adaptation) — local operators can reveal more detail but risk halos and are costlier.

一个微妙之处:逐通道施加时,强算子仍可能让明亮的颜色去饱和(三个通道被一起推向 1)。生产流水线常在亮度上做色调映射、再把原始色度贴回去,或使用带自有色彩变换的完整 ACES,以让饱和的高光保持饱和。算子又分全局(每个像素用同一条曲线——以上皆是)与局部(曲线随像素邻域自适应,像眼睛的局部适应)——局部算子能揭示更多细节,但有光晕风险且更昂贵。

Tone mapping: fitting HDR into a display色调映射:把 HDR 塞进显示器
The input is an HDR intensity ramp spanning 0 to ~8 (values above 1 are present and cannot be shown directly). Top: the transfer curve y = f(e·x) mapping input radiance x (horizontal, 0..8) to display value y (vertical, 0..1) for the chosen operator and exposure; the dashed line is the y=1 ceiling. Bottom: the ramp tone-mapped into a preview strip. Watch Clip flat-line to a blown-out white band, while Reinhard and ACES roll the highlights off smoothly. Drag exposure; toggle the operator.输入是一条从 0~8 的 HDR 强度斜坡(存在超过 1 的值,无法直接显示)。上方:传递曲线 y = f(e·x),把输入辐射亮度 x(横轴,0..8)映射到显示值 y(纵轴,0..1),对应所选算子与曝光;虚线是 y=1 上限。下方:被色调映射后的斜坡预览条。看 Clip 拉平成一条过曝的白带,而 ReinhardACES 把高光平滑地滚落。拖动曝光;切换算子。
Exposure
1.00×
Operator
ACES
Ramp clipped to 1.0
Maps x=1 →
Show the core JS查看核心 JS
// Apply exposure BEFORE the operator; each maps x in [0,inf) to y in [0,1].
function clip(x){ return x < 1 ? x : 1; }
function reinhard(x){ return x / (1 + x); }
function aces(x){                                   // Narkowicz fit
  var y = (x*(2.51*x + 0.03)) / (x*(2.43*x + 0.59) + 0.14);
  return y < 0 ? 0 : (y > 1 ? 1 : y);
}
function toneMap(x, e, op){
  var xe = e * x;                                   // exposure first
  return op === 'clip' ? clip(xe) : op === 'reinhard' ? reinhard(xe) : aces(xe);
}
// "% clipped" = fraction of the ramp whose mapped value hits the 1.0 ceiling.

5 · Encode & display — the OETF goes last编码与显示——OETF 放在最后

Tone mapping produced a value in [0,1] that is still linear — proportional to light. The final step turns it into the code value the display expects by applying the OETF (opto-electronic transfer function), which for a standard monitor is the sRGB curve — very close to x^{1/2.4}, defined piecewise with a tiny linear segment near black to avoid an infinite slope:

色调映射产出的是一个 [0,1] 内、但仍是线性的值——与光成正比。最后一步把它变成显示器所期望的码值,办法是施加 OETF(光电传递函数),对标准显示器而言就是 sRGB 曲线——非常接近 x^{1/2.4},分段定义,近黑处有一小段线性段以避免斜率无穷:

sRGB OETF:   V = 12.92·L  (L ≤ 0.0031308),   V = 1.055·L1/2.4 − 0.055  (otherwise)

Order is everything, and it is the golden rule of this lesson: textures decoded to linear → all lighting in linear → tone-map → encode with the OETF, once, at the very end. Swap tone-mapping and encoding, or encode twice, and the image is visibly wrong. Now the remaining vocabulary of "display" falls into place:

顺序就是一切,也是本课的黄金法则:纹理解码到线性 → 一切光照在线性中 → 色调映射 → 用 OETF 编码,只一次,在最末尾。把色调映射与编码调换顺序、或编码两次,图像就会明显出错。至此,“显示”余下的术语也各归其位:

Where this points next接下来指向何处

We have now closed the loop opened in lesson 01: an image is measured light, and this lesson is the bridge from the unbounded linear radiance the transport lessons compute to the bounded, encoded pixels a display can emit — linear math throughout, exposure to set the key, a tone-mapping curve to tame highlights, and the OETF applied exactly once at output. But everything from lesson 04 onward has quietly assumed a machine fast enough to run these per-pixel operations millions of times per frame: the shading, the interpolation, the tone map, the encode. Lesson 15 opens that machine — the GPU — and follows a single frame through its parallel pipeline, then looks at where real-time rendering is heading: hardware ray tracing, neural methods, and the convergence of the two visibility loops from lesson 01.

我们现在合上了第 01 课打开的环:图像是被测量的光,而本课正是那座桥——从传输课所计算的无上界线性辐射亮度,通往显示器能发出的、有界的、已编码的像素——全程线性运算,用曝光定基调,用一条色调映射曲线驯服高光,并在输出处只施加一次 OETF。但从第 04 课起的一切,都悄悄假设了一台足够快、能在每帧内把这些逐像素运算跑上百万次的机器:着色、插值、色调映射、编码。第 15 课打开这台机器——GPU——跟随单独一帧走过它的并行流水线,然后眺望实时渲染的去向:硬件光线追踪、神经方法,以及第 01 课那两个可见性循环的融合。

Takeaway要点

Radiance from the rendering equation is linear and unbounded (HDR); a display shows only [0,1] encoded through the sRGB gamma curve. So: do all lighting — adding, filtering (lesson 06 mips), interpolation and blending (lesson 05) — in linear light, or you get the "too dark / haloed" bug. Store float (fp16) framebuffers because highlights exceed 1. Pick an exposure e (the digital aperture) to set the key, then tone-map e·x into [0,1]: clip blows out and shifts hue, Reinhard x/(1+x) is soft but slightly gray, ACES is a filmic S-curve that keeps highlight detail. Bloom bleeds energy from >1 pixels before mapping. Apply the sRGB OETF last, and remember colors carry a gamut and white point. Golden rule: light in linear, tone-map between, encode at output.

来自渲染方程的辐射亮度是线性且无上界(HDR)的;显示器只显示经 sRGB gamma 曲线编码的 [0,1]。所以:一切光照——相加、滤波(第 06 课 mip)、插值与混合(第 05 课)——都在线性光中做,否则就会得到“太暗 / 有光晕”的 bug。用浮点(fp16)帧缓冲存储,因为高光超过 1。选一个曝光 e(数字光圈)定下基调,再把 e·x 色调映射[0,1]Clip 过曝并偏色,Reinhard x/(1+x) 柔和但略灰,ACES 是保住高光细节的胶片风 S 曲线。辉光在映射前把 >1 像素的能量渗出去。最后才施加 sRGB OETF,并记住颜色带有色域白点。黄金法则:光在线性里算,中途做色调映射,输出时才编码。

Interview prompts面试题