all lessons / computer_graphics / 09 · materials & texturing lesson 9 / 15

Materials & texturing in depth材质与纹理进阶

Lesson 08 shaded surfaces with per-object constants: one albedo, one roughness, one metallic value smeared over the whole mesh. But a real brick wall is red here and grey in the mortar there; a rusted panel is glossy on the metal and matte on the rust. Real surfaces vary point-to-point. So we stop feeding the BRDF constants and start feeding it texture maps — one lookup per pixel, per BRDF input — and then we learn to fake fine surface detail (bumps, grooves, pores) without adding geometry, because triangles are expensive and detail is cheap when it is only a normal. 第 08 课用逐物体的常量给表面着色:一个反照率、一个粗糙度、一个金属度,糊满整个网格。但真实的砖墙这里是红的、砖缝那里是灰的;一块生锈的板材,金属处发亮、锈处发哑。真实表面逐点变化。于是我们不再给 BRDF 喂常量,而开始喂它纹理贴图——每个像素、每个 BRDF 输入各查一次表——然后学会不加几何地伪造精细表面细节(凹凸、沟槽、毛孔),因为三角形昂贵,而当细节只是一个法线时,它很便宜。

The plan本课计划

Five moves. (1) Lay out the PBR map set — base-color, metallic, roughness, normal, AO, height, emissive — each map handing the BRDF one input per texel, and nail the trap that sinks beginners: color-space correctness (albedo/emissive are sRGB, must decode to linear; roughness/metallic/normal/AO are raw linear data, must not). (2) Normal mapping: store a fake normal per texel so flat geometry lights up bumpy — and see the silhouette that gives the lie away. (3) Tangent space & the TBN matrix: normal maps live in a per-texel frame built from UV gradients; transform the sampled normal into world space to light it, and understand why tangent space (it survives animation). (4) Parallax vs displacement: offset the UV by height×view for cheap fake depth, or actually move vertices for a correct silhouette at real cost. (5) Procedural texturing & noise: value/Perlin/simplex noise and the fractal sum (fbm), resolution-independent detail from a formula. The widget drives normal mapping live; lesson 10 then leaves surfaces behind for rays.

五步走。(1) 铺开 PBR 贴图组——base-color、metallic、roughness、normal、AO、height、emissive——每张贴图给 BRDF 每个纹素一个输入,并钉死那个坑杀新手的陷阱:色彩空间正确性(albedo/emissive 是 sRGB,须解码到线性;roughness/metallic/normal/AO 是原始线性数据,不能解)。(2) 法线贴图:每个纹素存一个假法线,让平坦几何被照得凹凸不平——并看穿那条露馅的轮廓。(3) 切线空间与 TBN 矩阵:法线贴图活在由 UV 梯度搭出的逐纹素坐标系里;把采样到的法线变换到世界空间去照亮它,并理解为什么要切线空间(它经得起动画)。(4) 视差 vs 位移:用 height×视线偏移 UV 来廉价伪造深度,或真的移动顶点、以真实开销换正确轮廓。(5) 程序化纹理与噪声:value/Perlin/simplex 噪声与分形求和(fbm),从一条公式生出分辨率无关的细节。控件现场演示法线贴图;随后第 10 课离开表面,转向光线。

1 · The PBR map set — one texel, one BRDF inputPBR 贴图组——一个纹素,一个 BRDF 输入

Recall lesson 08: the shading of a point needs a handful of numbers — a base color, whether the surface is metal or dielectric, how rough it is, which way it faces. In lesson 08 those were constants for the whole object. The single idea of texturing is to promote each constant to a function of surface position, stored as an image indexed by the mesh's (u, v) coordinates. Each texel of each map supplies exactly one BRDF input at that point. The maps are not decoration bolted on afterward; they are the arguments to the same shading equation, now varying per pixel.

回想第 08 课:给一个点着色需要若干数——一个基础颜色、表面是金属还是电介质、有多粗糙、朝向何方。在第 08 课这些是整个物体的常量。纹理化的唯一思想,就是把每个常量升格为表面位置的函数,存成一张按网格 (u, v) 坐标索引的图像。每张贴图的每个纹素,恰好在该点提供一个 BRDF 输入。这些贴图不是事后拴上去的装饰,而是同一个着色方程的实参,只是如今逐像素变化。

MapFeeds which BRDF inputChannelsColor space
Base color / albedodiffuse & metal reflectance colorRGBsRGB → decode
Metallicmetal vs dielectric switch (0 or 1)1 (grey)linear
Roughnessmicrofacet spread α (glossy↔matte)1 (grey)linear
Normalperturbed surface normal (tangent space)RGB → xyzlinear
Ambient occlusion (AO)fraction of ambient light not self-shadowed1 (grey)linear
Height / displacementsurface offset along the normal1 (grey)linear
Emissiveself-emitted radiance LeRGBsRGB → decode
贴图喂给哪个 BRDF 输入通道色彩空间
Base color / albedo漫反射与金属反射的颜色RGBsRGB → 解码
Metallic金属 vs 电介质开关(0 或 1)1(灰度)线性
Roughness微表面散布 α(光滑↔粗哑)1(灰度)线性
Normal被扰动的表面法线(切线空间)RGB → xyz线性
Ambient occlusion (AO)未被自遮挡的环境光比例1(灰度)线性
Height / displacement沿法线的表面偏移1(灰度)线性
Emissive自发射辐射亮度 LeRGBsRGB → 解码

The one column you cannot get wrong is color space. Two of these maps store perceptual color — base-color and emissive — and image files save color in sRGB, a nonlinear encoding that spends more bits where the eye is sensitive (the darks). But the shading math from lesson 08 is only valid in linear light, where adding two radiances means adding photons. So albedo and emissive must be decoded from sRGB to linear before lighting. The other maps — metallic, roughness, normal, AO, height — do not store a color at all; they store data that already means exactly what its number says. A roughness of 0.5 is 0.5. Running an sRGB decode over them corrupts the number.

唯一不能搞错的一列是色彩空间。其中两张贴图存的是感知颜色——base-color 与 emissive——而图像文件把颜色存成 sRGB,一种非线性编码,它在人眼敏感处(暗部)花更多比特。但第 08 课的着色数学只在线性光下成立,那里“两个辐射亮度相加”意味着“光子相加”。所以 albedo 与 emissive 必须在光照之前从 sRGB 解码到线性。其余贴图——metallic、roughness、normal、AO、height——根本不存颜色;它们存的是数据,其数值已就是字面意思。粗糙度 0.5 就是 0.5。对它们做 sRGB 解码会破坏这个数。

The transfer functions, with the standard sRGB curve (a near-gamma-2.4 with a small linear toe). Worked number: a mid-grey texel stored as 0.5 in an sRGB base-color map decodes to linear 0.52.4 ≈ 0.214 — less than half. Skip the decode and every albedo is far too bright, and the error compounds through every bounce.

传递函数,采用标准 sRGB 曲线(接近 gamma 2.4,带一小段线性起趾)。算个数:sRGB base-color 贴图里存为 0.5 的中灰纹素,解码到线性0.52.4 ≈ 0.214——不到一半。跳过解码,每个反照率都会亮得离谱,而误差会在每一次弹射中累积放大。

linear ≈ sRGB2.4  (decode, on load)   ·   sRGB ≈ linear1/2.4  (encode, at the very end)
The classic bug经典 bug

sRGB-decoding a normal or roughness map. A normal map's blue-ish RGB is not a color — it is a packed vector n = 2·RGB − 1. Decode it as sRGB and every component is skewed, so lit surfaces tilt subtly wrong and roughness reads too smooth. The rule is mechanical: if the map means a color you would show a human, it is sRGB and you decode it; if it means a number the shader consumes, it is linear and you leave it alone. Most asset pipelines tag this per-texture; getting the tag wrong is the number-one texturing bug, and lesson 14 (the full color-management pipeline) returns to it in earnest.

对 normal 或 roughness 贴图做 sRGB 解码。法线贴图偏蓝的 RGB 不是颜色——它是打包的向量 n = 2·RGB − 1。当成 sRGB 解码,每个分量都被扭曲,于是受光表面朝向微妙地错、粗糙度读起来太光滑。规则很机械:若贴图表示你会拿给人看的颜色,它是 sRGB、要解码;若它表示着色器要消费的数,它是线性、别去动它。多数素材管线逐纹理打这个标;标错就是头号纹理 bug,第 14 课(完整的色彩管理管线)会正式回到它。

2 · Normal mapping — bumps that aren't there法线贴图——并不存在的凹凸

Here is the highest-leverage trick in real-time graphics. In lesson 08, the only thing the diffuse term cares about is n · l — the angle between the surface normal and the light. That dot product is what makes a sphere look round: the normal swings smoothly, so brightness falls off toward the terminator. Normal mapping exploits this ruthlessly. Instead of building real bumps out of thousands of tiny triangles, we keep the geometry flat and coarse and store, per texel, a perturbed normal — a fake "which way this speck of surface faces." When we light the pixel, we use the map's normal in n · l instead of the geometry's. The lighting responds as if the surface were bumpy, at the cost of one texture lookup and zero extra triangles.

这是实时图形学里杠杆最大的一招。第 08 课里,漫反射项唯一在乎的是 n · l——表面法线与光线的夹角。正是这个点积让球看起来是圆的:法线平滑地摆动,于是亮度朝明暗界线渐弱。法线贴图把这一点用到极致。我们不用成千上万个小三角形堆出真凹凸,而是让几何保持平坦、粗糙,并逐纹素存一个被扰动的法线——一个假的“这一小片表面朝哪”。给像素着色时,我们在 n · l 里用贴图的法线,而不是几何的。光照的反应就仿佛表面真是凹凸的,代价只是一次纹理查表、零额外三角形。

The map itself is just an RGB image where each texel packs a unit vector: red↔x, green↔y, blue↔z, remapped from [−1, 1] to the [0, 1] pixel range by RGB = (n + 1)/2. A "flat" texel (normal straight out) is (0, 0, 1) → RGB(0.5, 0.5, 1.0) — which is why normal maps are that characteristic periwinkle blue: most of the surface is unperturbed, and only the grooves and rivets deviate.

贴图本身只是一张 RGB 图像,每个纹素打包一个单位向量:red↔x、green↔y、blue↔z,由 RGB = (n + 1)/2[−1, 1] 重映射到 [0, 1] 的像素范围。“平坦”纹素(法线笔直朝外)是 (0, 0, 1) → RGB(0.5, 0.5, 1.0)——这正是法线贴图那标志性的长春花蓝的由来:大部分表面未被扰动,只有沟槽与铆钉处才偏离。

The silhouette tell轮廓露馅

Normal mapping fakes shading, never shape. The geometry is still flat, so three things give it away: the silhouette stays a clean straight edge (real bumps would make it ragged); the surface casts no self-shadows from its bumps; and at a grazing angle the illusion collapses because you can see the flatness edge-on. This is exactly why the field also has parallax mapping (§4, moves the texture to fake occlusion) and displacement (§4, moves the actual vertices to fix the silhouette). Normal mapping is the cheapest rung of a ladder, not the whole ladder.

法线贴图伪造的是着色,从不是形状。几何仍是平的,于是三点露馅:轮廓仍是一条干净的直边(真凹凸会让它参差);表面的凹凸不投自阴影;在掠射角下幻觉崩塌,因为你侧看就看见了那份平坦。这正是本领域还需要视差贴图(§4,移动纹理来伪造遮挡)与位移(§4,移动真实顶点来修正轮廓)的原因。法线贴图是一架梯子最便宜的一级,而不是整架梯子。

3 · Tangent space & the TBN matrix切线空间与 TBN 矩阵

A subtlety sits inside §2 that we glossed. The map stores a normal like (0.1, −0.2, 0.97) — but relative to what frame? Not world space: if it were, the map would only be correct for one fixed orientation, and the moment the mesh rotates, deforms, or the same map tiles onto a differently-facing face, the "bumps" would light up wrong. The fix is to store the perturbation in tangent space — a local coordinate frame glued to the surface at each texel, whose z axis is the geometric normal. Then "mostly (0,0,1)" always means "mostly along the real surface normal," wherever and however that surface is oriented.

§2 里藏着一个我们略过的微妙之处。贴图存的法线形如 (0.1, −0.2, 0.97)——但相对哪个坐标系?不是世界空间:若是,这张贴图就只对某个固定朝向正确,一旦网格旋转、形变,或同一张贴图平铺到朝向不同的面上,“凹凸”就会照错。办法是把扰动存在切线空间里——一个逐纹素粘在表面上的局部坐标系,其 z就是几何法线。这样“大体是 (0,0,1)”就永远意味着“大体沿真实表面法线”,无论那表面在哪、朝向如何。

The three axes of that frame are the Tangent, Bitangent, and Normal — hence TBN. N is the interpolated geometric normal. T and B are derived so they align with the texture's u and v directions on the surface: T points the way u increases, B the way v increases. Concretely you recover them from two triangle edges and their UV deltas — the gradients of position with respect to (u,v):

这个坐标系的三条轴是 Tangent(切线)、Bitangent(副切线)、Normal(法线)——故名 TBN。N 是插值出的几何法线。T 与 B 的推导使它们在表面上对齐纹理的 uv 方向:T 指向 u 增大的方向,B 指向 v 增大的方向。具体地,你从三角形的两条边及其 UV 增量恢复它们——即位置对 (u,v) 的梯度:

[ e1 ; e2 ] = [ Δu1 Δv1 ; Δu2 Δv2 ] · [ T ; B ]   ⟹   solve the 2×2 for T, B

Stack the three unit axes as columns and you get the TBN matrix, a rotation that carries a tangent-space vector into world space. Sample the map, unpack nt = 2·RGB − 1, then transform and light:

把三条单位轴按列堆起来,就得到 TBN 矩阵,一个把切线空间向量送入世界空间的旋转。采样贴图,解包 nt = 2·RGB − 1,然后变换并照亮:

nworld = TBN · ntangent = [ T | B | N ] · nt   ·   then use nworld · l in shading

Why go to this trouble instead of baking normals straight into world space? Because tangent space is reusable. One tangent-space map tiles across a whole surface, works on every instance of a mesh regardless of its model matrix, and — critically — stays correct as a skinned character deforms: each frame you rebuild the (cheap) TBN from the animated geometry, and the same stored bumps ride along, always relative to the current surface. A world-space map would have to be re-authored for every pose. Same reasoning as lesson 02's insistence that directions carry their own transform rule — the normal lives in a frame, and you must respect the frame.

为什么费这个劲,而不把法线直接烘到世界空间?因为切线空间可复用。一张切线空间贴图能平铺整个表面,对一个网格的每个实例都有效、无论其模型矩阵如何,而且——关键地——当蒙皮角色形变时依然正确:每帧你从动画后的几何重建(廉价的)TBN,同一批存好的凹凸随之而行,永远相对当前表面。世界空间贴图则要为每个姿势重新制作。这与第 02 课强调“方向自带变换规则”是同一个道理——法线活在一个坐标系里,你必须尊重这个坐标系。

4 · Parallax & displacement — faking, then fixing, depth视差与位移——先伪造、再修正深度

Normal mapping fixed the shading of a bumpy surface but not its parallax — the way near bumps should slide over far ones as your eye moves, and near bumps should occlude the valleys behind them. Two techniques climb further up the ladder, trading cost for correctness.

法线贴图修好了凹凸表面的着色,却没修好它的视差——即当你眼睛移动时,近处的凸起应当在远处的凸起上滑动,近处的凸起应当遮住其后的凹谷。两种技术沿梯子往上爬,用开销换正确性。

Read them as three rungs on one ladder, each fixing what the previous faked: normal map fakes shading, parallax fakes intra-surface depth and occlusion, displacement makes the depth real. You pick the lowest rung that survives how close the camera gets.

把它们读作一架梯子的三级,每一级修正上一级伪造的东西:法线贴图伪造着色,视差伪造表面内的深度与遮挡,位移让深度成真。你选的是“能扛住相机凑多近”的最低那一级。

5 · Procedural texturing & noise程序化纹理与噪声

Everything so far assumed a hand-painted image sitting in memory. But some detail — marble veins, wood grain, terrain, clouds, rust — is better generated from a formula than stored. That is procedural texturing, and its raw material is noise: a function that looks random but is smooth, repeatable, and defined everywhere. The workhorses, in increasing sophistication:

迄今为止的一切都假设内存里躺着一张手绘图像。但有些细节——大理石纹、木纹、地形、云、锈——与其存储,不如从公式生成更好。这就是程序化纹理,其原料是噪声:一个看似随机、却平滑、可重复、且处处有定义的函数。三员主力,复杂度递增:

One octave of noise is bland. Nature has detail at every scale — big hills carry medium rocks carry small pebbles — so we sum copies of noise at doubling frequency and halving amplitude. That fractal sum is fbm (fractional Brownian motion):

单个八度的噪声很平淡。自然界在每个尺度都有细节——大山驮着中石、中石驮着小卵石——所以我们把频率翻倍、振幅减半的噪声副本叠加起来。这个分形求和就是 fbm(分数布朗运动):

fbm(p) = Σi=0..n−1   (½)i · noise( 2i · p )    (each layer = an "octave"; gain ½, lacunarity 2)

Why bother, when you could just store a big texture? Three reasons that a lookup can never match. Procedural textures are resolution-independent — zoom in forever and new detail keeps appearing, no pixelation. They are tileable and seamless by construction, so they cover infinite terrain with no visible repeat. And they are nearly free in memory — a formula and a seed instead of megabytes of image. The trade is compute per sample and less direct art control. This is why noise underlies terrain heightfields, cloud and smoke volumes, water, and countless material masks (where does the rust go? where is the surface worn?).

既然可以直接存一张大贴图,何必费事?三个查表永远比不上的理由。程序化纹理分辨率无关——无限放大,新细节不断涌现,绝不马赛克。它们天生可平铺、无缝,因而覆盖无限地形也看不出重复。而且它们几乎不占内存——一条公式加一颗种子,取代数兆字节的图像。代价是每次采样都要计算、且美术的直接掌控更弱。这就是为什么噪声撑起了地形高度场、云与烟的体积、水面,以及无数材质遮罩(锈往哪长?表面哪里磨损?)。

Normal mapping: bumps from a texture, not geometry法线贴图:凹凸来自纹理,而非几何
One perfectly flat quad, shaded per pixel. A procedural height field h(u,v) — a fixed lattice of smooth round Gaussian bumps — defines a perturbed normal N = normalize(−∂h/∂u, −∂h/∂v, 1) by finite differences. Toggle the normal map: OFF uses the flat normal (0,0,1), so the whole quad is one uniform tone (it is flat). ON lights each pixel by N·l, and the flat quad springs into embossed relief. Drag the light around and watch the highlights track the bumps — yet the geometry never changed.一块完全平坦的四边形,逐像素着色。一个程序化高度场 h(u,v)——固定格点上一批平滑圆润的高斯凸起——用有限差分定义被扰动的法线 N = normalize(−∂h/∂u, −∂h/∂v, 1)。切换法线贴图:用平坦法线 (0,0,1),整块四边形是一个均匀色调(它本来就平)。N·l 逐像素着色,平坦四边形立刻浮起浮雕感。拖动光源,看高光追随凸起——可几何从未改变。
Light angle
Mode
map ON
Geometry
flat quad
Cost vs bumps
1 lookup
Show the core JS查看核心 JS
// Deterministic height field: a fixed lattice of Gaussian bumps (no RNG).
function height(u, v){
  var h = 0;
  for (var i=0; i<centers.length; i++){
    var du = u - centers[i][0], dv = v - centers[i][1];
    h += centers[i][2] * Math.exp(-(du*du + dv*dv) / (r*r));  // Σ exp(−dist²/r²)
  }
  return h;
}
// Perturbed normal from the height gradient (finite differences), then normalize.
var e = 0.004;
var dhdu = (height(u+e,v) - height(u-e,v)) / (2*e);
var dhdv = (height(u,v+e) - height(u,v-e)) / (2*e);
var N = normalize(-dhdu, -dhdv, 1.0);          // map OFF ⇒ N = (0,0,1)
// Lambert shading: the flat quad looks bumpy purely through N·l.
var diff = Math.max(0, N[0]*lx + N[1]*ly + N[2]*lz);
var shade = 0.12 + 0.88*diff;                  // ambient + diffuse

Where this points next接下来指向何处

We can now feed the lesson-08 BRDF a different set of inputs at every pixel — color, roughness, metalness, and a faked normal — all from images or formulas, and we know the two ways this goes wrong: mixing up sRGB and linear data, and mistaking normal mapping's shading trick for real shape. Everything in lessons 04–09 has lived on the rasterization side: for each triangle, which pixels, shaded locally. But local shading with a single light ignores the recursive term of the rendering equation from lesson 01 — the light bouncing between surfaces, the sharp reflections, the true shadows. To get those we must be able to ask "what does an arbitrary ray hit?", cheaply, millions of times. Lesson 10 builds exactly that: ray tracing and the BVH, the acceleration structure that turns the naive per-ray O(T) triangle test into O(log T) — and reopens the other loop from lesson 01.

现在我们能在每个像素给第 08 课的 BRDF 喂一套不同的输入——颜色、粗糙度、金属度、以及一个伪造的法线——全都来自图像或公式,并且知道这条路会怎样出错:混淆 sRGB 与线性数据,以及把法线贴图的着色戏法误当成真实形状。第 04–09 课的一切都活在光栅化这一侧:对每个三角形,覆盖哪些像素、就地着色。但单光源的局部着色忽略了第 01 课渲染方程的递归项——在表面之间弹射的光、锐利的反射、真正的阴影。要得到它们,我们必须能廉价地、上百万次地问“任意一条光线命中什么?”。第 10 课正是造这个:光线追踪与 BVH,这个加速结构把朴素的每条光线 O(T) 三角形测试变成 O(log T)——并重新打开第 01 课的另一个循环。

Takeaway要点

Texturing promotes each per-object BRDF constant to a per-texel function: base-color, metallic, roughness, normal, AO, height, emissive — one lookup, one input. The non-negotiable rule is color space: albedo and emissive are sRGB and must be decoded to linear before lighting (mid-grey 0.5 → 0.214); metallic/roughness/normal/AO/height are linear data and must not be decoded — the top texturing bug. Normal mapping stores a perturbed normal per texel and lights the flat surface by N·l, faking bumps for one lookup and zero triangles — but it fakes shading, not shape, so the silhouette, missing self-shadows, and grazing angles give it away. Those normals live in tangent space, a per-texel (T,B,N) frame from the UV gradients; the TBN matrix carries them to world space, and tangent space is chosen because it is reusable across tiling and deformation. Up the ladder: parallax offsets UVs by h·view to fake intra-surface depth/occlusion (silhouette still flat); displacement moves real (tessellated) vertices to make it correct and costly. And procedural noise (value/Perlin/simplex, summed as fbm) generates resolution-independent, tileable, near-free detail from a formula.

纹理化把每个逐物体的 BRDF 常量升格为逐纹素函数:base-color、metallic、roughness、normal、AO、height、emissive——一次查表,一个输入。不可商量的规则是色彩空间:albedo 与 emissive 是 sRGB,光照前须解码到线性(中灰 0.5 → 0.214);metallic/roughness/normal/AO/height 是线性数据,不能解码——头号纹理 bug。法线贴图逐纹素存被扰动的法线,用 N·l 照亮平坦表面,以一次查表、零三角形伪造凹凸——但它伪造的是着色而非形状,故轮廓、缺失的自阴影、掠射角都会露馅。这些法线活在切线空间,一个由 UV 梯度得来的逐纹素 (T,B,N) 坐标系;TBN 矩阵把它们送到世界空间,选切线空间是因为它能跨平铺与形变复用。往梯子上走:视差h·视线 偏移 UV 伪造表面内的深度/遮挡(轮廓仍平);位移移动真实(细分出的)顶点,使之正确而昂贵。而程序化噪声(value/Perlin/simplex,叠加为 fbm)从一条公式生成分辨率无关、可平铺、几乎免费的细节。

Interview prompts面试题