Evaluation, deployment, and CV system design

评估、部署与计算机视觉系统设计

Lessons 17–18 gave us models that both generate and restore pixels — and a warning that even a brilliant checkpoint can be harmful or useless in the wild. Every lesson so far produced a model. This capstone produces a system: the data, evaluation, compression, serving, monitoring, and ethics that decide whether the model survives contact with production. It also closes the track's loop — because the first thing that breaks in deployment is the very contract lesson 01 set up: preprocessing.

第 17–18 课给了我们既能生成又能复原像素的模型——以及一个警告：即便是一个出色的 checkpoint，在真实世界里也可能有害或无用。到目前为止每一课产出的都是一个模型。这门收官课产出的是一个系统：由数据、评估、压缩、服务、监控与伦理共同决定模型能否经受住与生产环境的正面碰撞。它也让整门课的环路闭合——因为部署中最先出问题的，正是第 01 课立下的那份契约：预处理。

1 · Evaluation that survives contact with production

1 · 能经受住生产环境碰撞的评估

A single aggregate number lies. A detector at 92% mAP (mean Average Precision, lesson 09) can be 98% on bright daytime frames and 60% at night — and your users at night experience the 60%. The discipline is slicing: report the task metric broken down by the conditions that change image statistics. For detection, slice by object size, occlusion, class, camera, lighting, geography. For OCR (optical character recognition), slice by font, resolution, language, blur, rotation, document type. The aggregate is a weighted average that hides exactly the failures that hurt.

单一的聚合数字会撒谎。一个 92% mAP（mean Average Precision，平均精度均值，第 09 课）的检测器，在明亮的白天画面上可能是 98%、在夜间只有 60%——而你的夜间用户体验到的就是那个 60%。这门功夫叫切片：把任务指标按那些会改变图像统计特性的条件拆开来报告。对检测，按目标尺寸、遮挡、类别、相机、光照、地域来切。对 OCR（optical character recognition，光学字符识别），按字体、分辨率、语言、模糊、旋转、文档类型来切。聚合数字是一个加权平均，它恰好掩盖了那些真正伤人的失败。

Three evaluation failure modes recur across the whole track, and a strong interview answer names them before being asked:

三种评估失效模式贯穿整门课反复出现，而一个出色的面试回答会在被问之前就点出它们：

• Leakage — train/test contamination that inflates offline metrics. In CV this is structural: near-duplicate images (the same product shot twice), adjacent video frames (lesson 16), or augmentations of the same source landing on both sides of the split. Fix with entity/time/source-level splits, not random ones.泄漏（leakage）——训练/测试集互相污染，从而虚抬了离线指标。在计算机视觉里这是结构性的：近似重复的图像（同一件商品拍了两次）、相邻的视频帧（第 16 课），或同一来源的数据增强版本落到了划分的两侧。修复办法是按实体/时间/来源级划分，而不是随机划分。
• Robustness / domain shift — the test set is clean; production isn't. New cameras, compression artifacts (lesson 01's JPEG blocks), weather, motion blur. Evaluate on corrupted and shifted slices deliberately, not just the curated benchmark.鲁棒性 / 域偏移（domain shift）——测试集是干净的，生产环境却不是。新相机、压缩伪影（第 01 课的 JPEG 块效应）、天气、运动模糊。要刻意地在被损坏和被偏移的切片上评估，而不只是在精心整理过的基准上。
• Fairness — unequal performance across skin tone, age, geography, or device. This is a slice audit with a product constraint attached: you must measure per-group metrics and set a floor, because an aggregate can be excellent while a subgroup is failed.公平性——在不同肤色、年龄、地域或设备上的表现不均等。这是一次附带产品约束的切片审计：你必须度量每个分组的指标并设定一个下限，因为聚合结果可以很出色，而某个子群体却被辜负了。

1.5 · Adversarial and corruption robustness

1.5 · 对抗鲁棒性与损坏鲁棒性

The §1 slices ask "where is the model weak on natural data?" This section asks a sharper question: where is it weak on data an adversary or a sensor chooses? Both attacks exploit the same blind spot, so it pays to see it from the model's perspective first. A trained classifier's decision boundary is shaped only by the directions the training data happened to constrain. In the ~150,000-dimensional input space of an image, the data lies on a thin manifold (lesson 17, §1); the vast majority of directions away from it were never seen, so the boundary there is essentially arbitrary. Anything that nudges an input along those unconstrained directions — a malicious perturbation or an everyday corruption — can flip the prediction even though the image looks unchanged to a human.

§1 的切片问的是"模型在自然数据上哪里弱？"本节问一个更尖锐的问题：它在对手或传感器挑选出来的数据上哪里弱？两类攻击利用的是同一个盲点，所以值得先从模型的视角把它看清。一个训练好的分类器，其决策边界只被训练数据恰好约束到的那些方向所塑造。在一张图像约 15 万维的输入空间里，数据落在一个很薄的流形上（第 17 课，§1）；离开它的绝大多数方向从未被见过，所以边界在那里本质上是任意的。任何把输入沿着这些无约束方向轻推的东西——恶意扰动，或日常的损坏——都能翻转预测，即便这张图在人眼看来毫无变化。

(a) Adversarial examples. A tiny, often imperceptible perturbation, deliberately crafted to push an input across the decision boundary. The perturbation is kept small by bounding it in an L∞ ball — every pixel may move by at most ε, a common setting being ε = 8/255 ≈ 0.031 (about 3% of the full 0–255 range per pixel, well under the threshold of visible change). Why such a small budget suffices: a near-linear high-dimensional model accumulates many small, coordinated nudges into one large change in the logit — each pixel barely moves, but thousands of them moving in the gradient's direction add up. The cheapest construction is FGSM (fast gradient sign method): take one step along the sign of the loss gradient with respect to the input,

(a) 对抗样本。一个微小、通常人眼不可察觉的扰动，被刻意构造出来以把输入推过决策边界。扰动通过被约束在一个 L∞ 球内而保持很小——每个像素最多移动 ε，一个常见设置是 ε = 8/255 ≈ 0.031（约为每像素完整 0–255 范围的 3%，远在可见变化的阈值之下）。为什么这么小的预算就够了：一个近似线性的高维模型会把许多微小、协调一致的轻推累积成 logit 上的一次大改变——每个像素几乎没动，但成千上万个像素朝梯度方向一起动就叠加起来了。最廉价的构造是 FGSM（fast gradient sign method，快速梯度符号法）：沿着损失对输入的梯度符号方向走一步，

Each pixel is shoved by exactly ±ε in whichever direction raises the loss — the maximal damage inside the L∞ ball for a linear approximation. The standard defense is adversarial training: generate these attacked inputs during training and include them in the loss, so the boundary is explicitly pushed away from the manifold's neighborhood. It costs accuracy on clean data and never fully closes the gap, but it is the strongest general defense known.

每个像素都被恰好推动 ±ε，方向取那个能抬高损失的方向——这是线性近似下 L∞ 球内的最大破坏。标准的防御是对抗训练：在训练期间生成这些被攻击的输入并把它们纳入损失，从而把边界明确地推离流形的邻域。它会牺牲干净数据上的精度，而且永远无法完全弥合差距，但它是已知最强的通用防御。

(b) Corruption / common-perturbation robustness. The same blind spot, exploited by nature rather than an attacker. ImageNet-C-style benchmarks apply a fixed catalogue of non-adversarial shifts — Gaussian/shot noise, several blurs, weather (snow, fog), digital (JPEG compression, pixelation) — each at several severity levels, and measure graceful degradation rather than a single accuracy. The headline metric is mCE — mean corruption error: error averaged across corruption types and severities, usually normalized to a baseline model so 100 = baseline and lower is better. Unlike an adversarial attack, no gradient is involved; the point is whether the model degrades smoothly under the shifts it will actually meet in production (§1's domain-shift row, made measurable).

(b) 损坏 / 常见扰动鲁棒性。同一个盲点，只是被自然而非攻击者利用。ImageNet-C 风格的基准施加一份固定的非对抗偏移清单——高斯/散粒噪声、若干种模糊、天气（雪、雾）、数字类（JPEG 压缩、像素化）——每种都有几个严重程度等级，度量的是优雅降级而不是单一精度。头号指标是 mCE——mean corruption error（平均损坏错误率）：在各损坏类型与严重程度上平均的错误率，通常相对一个基线模型归一化，使得 100 = 基线、越低越好。与对抗攻击不同，这里不涉及任何梯度；重点在于模型在它将来真正会在生产中遇到的那些偏移下，是否平滑地退化（把 §1 中域偏移那一行变得可度量）。

The unifying lesson, from the model's perspective: the decision boundary is constrained only where the training data put pressure on it; everywhere else it is free for an attacker or a corruption to exploit. Robustness is therefore an evaluation axis in its own right — report adversarial accuracy at a stated ε and an mCE, not just clean accuracy.

从模型视角出发的统一结论是：决策边界只在训练数据对它施加了压力的地方才受约束；在其他任何地方，它都任由攻击者或损坏去利用。因此鲁棒性本身就是一条独立的评估轴——要报告在给定 ε 下的对抗精度和一个 mCE，而不只是干净精度。

2 · Compression — how to fit the latency and cost budget

2 · 压缩——如何塞进延迟与成本预算

A model that hits the metric but misses the latency or cost budget does not ship. Four families shrink it, and you should know the mechanism of each, not just the name.

一个达到了指标却超出延迟或成本预算的模型是无法上线的。有四大类方法能把它缩小，而你应该知道每一类的机制，而不只是名字。

Quantization stores and computes weights/activations in lower precision than FP32 — typically INT8 (8-bit integers) or FP8 (8-bit float, common on recent accelerators). The "4×" is just bit math: FP32 is 32 bits per weight, INT8 is 8, so 32/8 = 4× smaller weights (and 2–4× faster matmuls on hardware with INT8/FP8 units, plus lower memory-bandwidth pressure — often the real bottleneck). The mapping itself is affine: q = round(x / scale) + zero_point, sending a float x to an integer code q. Worked example, symmetric INT8 over weights in [−1, 1] using the 255 levels [−127, 127] with zero_point = 0: the step is scale = 1/127 ≈ 0.00787, so a weight x = 0.5 maps to q = round(0.5 / (1/127)) + 0 = round(0.5 · 127) = round(63.5) = 64 — i.e. 0.5 lands at integer level 64 of 127, and dequantizing 64 · scale ≈ 0.504 recovers it to within one step. That rounding is the entire source of quantization error. Two flavors:

量化用比 FP32 更低的精度存储和计算权重/激活——通常是 INT8（8 位整数）或 FP8（8 位浮点，在近期加速器上常见）。所谓"4×"只是位数的算术：FP32 每个权重 32 位，INT8 是 8 位，所以 32/8 = 权重小 4 倍（在带 INT8/FP8 计算单元的硬件上矩阵乘还快 2–4 倍，外加更低的显存带宽压力——那往往才是真正的瓶颈）。映射本身是仿射的：q = round(x / scale) + zero_point，把一个浮点数 x 送到一个整数码 q。举个具体例子，对 [−1, 1] 内的权重做对称 INT8，用 [−127, 127] 这 255 级、取 zero_point = 0：步长是 scale = 1/127 ≈ 0.00787，于是权重 x = 0.5 映射为 q = round(0.5 / (1/127)) + 0 = round(0.5 · 127) = round(63.5) = 64——也就是 0.5 落在 127 级中的整数级 64，反量化 64 · scale ≈ 0.504 把它还原到一个步长之内。那次取整就是量化误差的全部来源。有两种风味：

• PTQ — post-training quantization. Take a trained FP32 model and quantize it directly, calibrating the scale factors on a small unlabeled sample. Cheap, no retraining, minutes to apply — but can lose accuracy if activations have outliers.PTQ——post-training quantization（训练后量化）。拿一个训练好的 FP32 模型直接量化，在一小份无标注样本上标定缩放因子。廉价、无需重训、几分钟就能用上——但如果激活里有离群值，可能会掉精度。
• QAT — quantization-aware training. Simulate the rounding during fine-tuning so the network learns weights robust to low precision. More expensive (a training run) but recovers most of the accuracy PTQ loses. Rule of thumb: try PTQ first; reach for QAT when PTQ drops accuracy past your floor.QAT——quantization-aware training（量化感知训练）。在微调期间模拟取整，让网络学到对低精度鲁棒的权重。更贵（要跑一次训练），但能把 PTQ 损失掉的精度大部分挽回。经验法则：先试 PTQ；当 PTQ 把精度掉到你的下限以下时，再动用 QAT。

Pruning removes weights. Unstructured pruning zeros individual weights (high sparsity, but needs special hardware/kernels to actually speed up). Structured pruning removes whole channels or filters — yielding a genuinely smaller dense network that any hardware runs faster. Distillation (lesson 14's teacher-student idea, generalized — but note the difference: lesson 14 was self-distillation, where the teacher is an EMA copy of the student itself for representation learning; here it is model-compression distillation, where a small student learns from a large, separate, already-trained and frozen teacher to shrink the model) trains a small student to mimic a large teacher's soft outputs; the student reaches accuracy it could not learn from hard labels alone, at a fraction of the cost. The mechanism in one sentence: the student trains on the teacher's full soft probability vector — the dark knowledge in the non-argmax classes (that a husky is a little dog-like and very little truck-like) — which carries more information per example than a one-hot label. Cascades / early-exit exploit that most inputs are easy: a tiny cheap model handles the obvious cases and only escalates uncertain ones to a big model. If the cheap stage confidently resolves 90% of inputs at 5 ms and the expensive stage (50 ms) sees only 10%, the average cost is 0.9·5 + 0.1·(5+50) = 4.5 + 5.5 = 10 ms — a 5× speedup over always running the big model, with little accuracy loss because escalation catches the hard cases. Early-exit is the same idea inside one network: attach classifier heads at intermediate layers and stop as soon as one is confident.

剪枝移除权重。非结构化剪枝把单个权重置零（稀疏度高，但要真正提速需要专门的硬件/内核）。结构化剪枝移除整条通道或整个滤波器——得到一个货真价实更小的稠密网络，任何硬件都能跑得更快。蒸馏（第 14 课师生思想的推广——但请注意区别：第 14 课是自蒸馏，教师是学生自身的 EMA 副本，用于表示学习；这里是模型压缩蒸馏，一个小学生向一个大的、独立的、已经训练好并被冻结的教师学习，以缩小模型）训练一个小学生去模仿一个大教师的软输出；学生达到了它单靠硬标签学不到的精度，而代价只是一小部分。一句话讲清机制：学生在教师完整的软概率向量上训练——即非 argmax 类别里的暗知识（dark knowledge）（哈士奇有点像狗、非常不像卡车）——它每个样本携带的信息比一个 one-hot 标签更多。级联 / 提前退出利用了大多数输入都很简单这一点：一个小而廉价的模型处理显而易见的情形，只把不确定的上交给大模型。如果廉价阶段在 5 ms 内自信地解决了 90% 的输入，而昂贵阶段（50 ms）只见到 10%，那平均成本就是 0.9·5 + 0.1·(5+50) = 4.5 + 5.5 = 10 ms——相比始终运行大模型提速 5 倍，且精度损失很小，因为上交机制接住了那些难例。提前退出（early-exit）是同一思想在单个网络内部的版本：在中间层挂上分类头，一旦某个头足够自信就停下来。

3 · The serving math — batching, throughput, and p99

3 · 服务背后的数学——批处理、吞吐与 p99

Two numbers govern serving, and they pull in opposite directions. Throughput is requests served per second; latency is time per request. Accelerators are most efficient on large batches (one big matmul beats many small ones), so batching raises throughput — but it raises latency too, because a request must wait for the batch to fill (or for a max-wait timeout) before it runs.

有两个数字主宰着服务，而它们朝相反的方向拉扯。吞吐是每秒服务的请求数；延迟是每个请求的耗时。加速器在大批量上效率最高（一次大矩阵乘胜过许多次小的），所以批处理抬高吞吐——但它也抬高延迟，因为一个请求必须等批量填满（或等到一个最大等待超时）之后才能运行。

The deeper trap is which latency you optimize. The mean latency hides the tail. p99 is the 99th-percentile latency — 99% of requests finish faster than this; 1% are slower. It is the right target because a user issuing many requests, or a page making several model calls, is very likely to hit that slow 1%, and the tail is what they remember. Worked example: say 99% of requests take 20 ms and 1% take 500 ms (a GC pause, a cold cache, an oversized image). The mean is 0.99·20 + 0.01·500 = 19.8 + 5 = 24.8 ms — looks fine. But a web page that fires 10 independent model calls has a 1 − 0.99¹⁰ ≈ 9.6% chance that at least one call hits the 500 ms tail, so nearly 1 in 10 page loads stalls for half a second. The mean said 25 ms; the user feels 500 ms. Optimize the tail. Levers on it: cap input resolution, set a batch max-wait timeout, isolate slow paths, and use cascades (§2) so the common case is fast.

更深的陷阱在于你优化的是哪一个延迟。均值延迟掩盖了尾部。p99 是第 99 百分位延迟——99% 的请求比它快，1% 比它慢。它是正确的目标，因为一个发出许多请求的用户、或一个要做好几次模型调用的页面，非常可能撞上那慢的 1%，而尾部正是他们记住的东西。算个具体例子：假设 99% 的请求耗时 20 ms，1% 耗时 500 ms（一次 GC 暂停、一次冷缓存、一张超大图像）。均值是 0.99·20 + 0.01·500 = 19.8 + 5 = 24.8 ms——看起来没问题。但一个触发 10 次独立模型调用的网页，有 1 − 0.99¹⁰ ≈ 9.6% 的概率至少一次调用撞上 500 ms 的尾部，于是差不多每 10 次页面加载就有 1 次卡上半秒。均值说是 25 ms；用户感受到的是 500 ms。要优化尾部。可用的杠杆有：限制输入分辨率、设定批量的最大等待超时、隔离慢路径，并使用级联（§2）让常见情形保持快速。

3.5 · Interactive · batching, throughput, and the p99 tail

3.5 · 交互 · 批处理、吞吐与 p99 尾部

This is the most counterintuitive trade in the lesson: a bigger batch buys throughput but charges latency, and a healthy mean can hide a brutal tail. Move the sliders and watch both prices move at once — then watch the tail compound across a page that makes several calls.

这是本课中最反直觉的权衡：更大的批量买来了吞吐，却要付出延迟的代价，而一个健康的均值可能藏着一条残酷的尾部。拖动滑块，看这两个代价同时变动——然后看尾部如何在一个做多次调用的页面上层层放大。

// Closed-form serving model. One batched matmul of size B takes a fixed
// overhead plus a per-item cost: service(B) = OVERHEAD + PER_ITEM*B  (ms).
// Throughput = items per second when the box runs back-to-back batches.
function tput(B){ return 1000 * B / service(B); }            // req/s

// A request's latency = time waiting for the batch to fill (bounded by the
// max-wait timeout) + the service time of the batch it rides.
function meanLatency(B, maxWait){
  const fillWait = Math.min(maxWait, 0.5*(B-1)*ARRIVAL_GAP); // avg wait to fill
  return fillWait + service(B);
}
// The tail: a fraction p of requests are slow (GC pause, cold cache, big image).
// p99 ≈ the slow branch once p ≥ 1%. Across n independent calls, the chance
// that AT LEAST ONE is slow is the key formula — the mean never shows this:
function pTailAcrossCalls(p, n){ return 1 - Math.pow(1 - p, n); }

Notice what the right panel proves: hold the per-call tail at p = 1% and a single call looks safe, but a page firing n = 10 calls hits 1 − 0.99¹⁰ ≈ 9.6% — the same arithmetic as the worked example above, now as a curve you can watch bend toward certainty as n grows. That is why the target is p99, not the mean: the mean is an average over requests, but the user is an and over their requests.

注意右侧面板证明了什么：把每次调用的尾部固定在 p = 1%，单次调用看起来很安全，但一个触发 n = 10 次调用的页面会命中 1 − 0.99¹⁰ ≈ 9.6%——和上面那个具体例子是同样的算术，现在成了一条曲线，你能看着它随 n 增大而弯向必然。这正是为什么目标是 p99 而不是均值：均值是对各个请求取平均，而用户是对他的各个请求取与（and）。

Edge vs cloud. Cloud serving gives big accelerators, easy batching, and elastic scale — at the cost of network round-trip latency and per-request bandwidth (sending full-resolution images). Edge serving (phone, camera, car) gives near-zero network latency and privacy-by-default (data never leaves the device), but is constrained by memory, thermal throttling, battery, and offline operation — which is exactly where the §2 compression toolkit (quantize, prune, distill, early-exit) becomes mandatory rather than optional.

边缘 vs 云。云端服务提供大加速器、便捷的批处理和弹性伸缩——代价是网络往返延迟和每请求带宽（要传送全分辨率图像）。边缘服务（手机、相机、汽车）提供近乎零的网络延迟和默认的隐私（数据从不离开设备），但受限于内存、热降频、电池以及离线运行——而这恰恰是 §2 那套压缩工具箱（量化、剪枝、蒸馏、提前退出）从"可选"变成"必需"的地方。

4 · Closing the loop — drift, humans, privacy

4 · 闭合环路——漂移、人、隐私

A deployed model degrades even if its weights never change, because the world changes. Drift monitoring watches for this. Data drift: input statistics move (a new phone camera, a seasonal lighting shift) — detectable without labels by tracking input feature distributions and confidence histograms. Concept drift: the input-to-label relationship itself shifts (fashion changes what "trending" looks like) — needs fresh labels to catch. The classic production puzzle: offline metrics unchanged, but the product regressed. The cause is almost always outside the weights — a preprocessing mismatch (train resized with bilinear, serving with nearest; train normalized with one mean, serving another), a threshold or NMS change shipped separately from the model, or a new input distribution the static test set never saw. That mismatch is the train/serve skew we close the track on below.

一个已部署的模型，即便权重从不改变也会退化，因为世界在变。漂移监控就是在盯着这件事。数据漂移（data drift）：输入统计特性移动了（一款新手机相机、一次季节性光照变化）——无需标签即可检测，方法是跟踪输入特征分布和置信度直方图。概念漂移（concept drift）：输入到标签的关系本身变了（时尚改变了"流行"长什么样）——需要新鲜标签才能捕捉。经典的生产谜题：离线指标没变，产品却退化了。原因几乎总在权重之外——一次预处理不匹配（训练用 bilinear 缩放、服务用 nearest；训练用一个均值归一化、服务用另一个）、一次与模型分开上线的阈值或 NMS 改动，或一个静态测试集从未见过的新输入分布。那种不匹配就是我们在下文收尾整门课时要讲的训练/服务偏斜（train/serve skew）。

Human-in-the-loop routes uncertain or high-stakes cases to people: a confidence threshold sends low-confidence predictions to review (the cascade pattern again, with a human as the final stage), and those reviewed cases become labeled training data — closing the loop via active learning. For safety-critical systems (medical triage, content moderation) the human is a required gate, not an optimization. Privacy is a first-class design constraint in CV because the data is faces, plates, locations, and bodies: minimize retention, redact/blur at ingestion, keep audit-logged access controls, and prefer on-device processing when the data is sensitive.

人机协同（human-in-the-loop）把不确定或高风险的情形交给人：一个置信度阈值把低置信度的预测送去复核（又是级联模式，只是把人放在最后一级），而那些被复核过的情形又变成有标注的训练数据——通过主动学习闭合环路。对安全攸关的系统（医疗分诊、内容审核），人是一道必需的关卡，而不是一项优化。隐私在计算机视觉里是一等的设计约束，因为数据是人脸、车牌、位置和身体：最小化留存、在采集入口就脱敏/模糊、保留带审计日志的访问控制，并在数据敏感时优先在设备端处理。

5 · The 6-step CV system-design template

5 · 六步计算机视觉系统设计模板

This is the spine of a strong system-design interview answer. Move from product to architecture — never the reverse. Architecture names are the last thing you say, justified by the constraints above them.

这是一个出色的系统设计面试回答的脊柱。从产品走向架构——绝不能反过来。架构的名字是你最后才说的东西，并由它上方的那些约束来给出理由。

1. Clarify the task output. Class, box, mask, text, embedding, track, or decision? This alone selects half the stack (lessons 08–12).厘清任务输出。是类别、边界框、掩码、文本、嵌入、轨迹，还是一个决策？单是这一点就选定了半个技术栈（第 08–12 课）。
2. Define the metric and business cost. What does a false positive cost vs a false negative? What is the latency SLA and the human-review budget? (A missed tumor and a missed ad-click are not the same error.)定义指标与业务成本。一次假阳性与一次假阴性各自的代价是多少？延迟 SLA 和人工复核预算是多少？（漏掉一个肿瘤和漏掉一次广告点击，不是同一种错误。）
3. Design the data. Collection, annotation + QA, splits (entity/time-level — §1), the slices you'll audit, and privacy at ingestion.设计数据。采集、标注 + 质检（QA）、划分（实体/时间级——§1）、你将审计的切片，以及采集入口处的隐私。
4. Choose model and pipeline. Preprocessing, backbone, head, post-processing, thresholds — and version them together (§1, §4).选择模型与流水线。预处理、骨干网络、任务头、后处理、阈值——并把它们一起做版本管理（§1、§4）。
5. Plan serving. Hardware, edge vs cloud, batching and the p99 target (§3), compression to fit the budget (§2), fallbacks, observability, staged/canary rollout.规划服务。硬件、边缘 vs 云、批处理与 p99 目标（§3）、为塞进预算而做的压缩（§2）、回退方案、可观测性、分阶段/金丝雀灰度发布。
6. Close the loop. Monitor drift, sample and triage errors, route uncertain cases to humans, retrain, re-audit slices.闭合环路。监控漂移，采样并分诊错误，把不确定的情形交给人，重训，重新审计切片。

Where this points next — the whole track, closing on lesson 01

这将指向何处——整门课，收束于第 01 课

There is no lesson 19; the arrow turns back on itself. Trace the spine we built: an image is a sampled, quantized, noisy grid with a preprocessing contract (01) → convolution extracts edges (02) → repeatable features match across views (03) → the camera model turns matches into geometry (04) → multiple views recover 3D (05) → CNNs learn the features end-to-end (06) → which needs data, loss, and optimization to train (07) → the simplest task is naming the image (08) → then locating objects (09) → then labeling every pixel (10) → then joints, identities, motion (11) → then embedding spaces for retrieval (12) → transformers scale past the CNN's locality (13) → self-supervision frees us from labels (14) → text supervises images (15) → time turns images into video (16) → and the arrow reverses to synthesis (17). This lesson (18) wraps all of it in the system that makes it real.

没有第 19 课；箭头转回到它自己身上。回溯我们搭起的这条脊柱：图像是一个带预处理契约的、经采样、量化、含噪声的网格（01）→ 卷积提取边缘（02）→ 可重复的特征在多视图间匹配（03）→ 相机模型把匹配变成几何（04）→ 多视图恢复三维（05）→ CNN 端到端地学出特征（06）→ 而这需要数据、损失和优化来训练（07）→ 最简单的任务是给图像命名（08）→ 然后定位物体（09）→ 然后给每个像素打标签（10）→ 然后是关节、身份、运动（11）→ 然后是用于检索的嵌入空间（12）→ Transformer 越过 CNN 的局部性去扩展（13）→ 自监督把我们从标签中解放出来（14）→ 文本监督图像（15）→ 时间把图像变成视频（16）→ 箭头反转，走向合成（17）。这一课（18）把这一切包进那个让它成为现实的系统里。

And the loop closes precisely where it began. The single most common production regression in computer vision — offline metrics green, users unhappy — is train/serve preprocessing skew: the resize interpolation, the channel order (RGB vs BGR), the normalization constants, the color space, the JPEG decode path differ between the pipeline that trained the model and the pipeline that serves it. Lesson 01 called preprocessing "part of the model contract." Eighteen lessons later, that is the line that decides whether the whole stack works. The model is never just the weights; it is the weights plus the exact pixel pipeline that produced its inputs — versioned together, monitored together, shipped together.

而环路恰好在它开始的地方闭合。计算机视觉中最常见的那一种生产回归——离线指标一片绿、用户却不满——就是训练/服务预处理偏斜：缩放插值、通道顺序（RGB vs BGR）、归一化常数、色彩空间、JPEG 解码路径，在训练模型的那条流水线和服务它的那条流水线之间不一致。第 01 课把预处理称为"模型契约的一部分"。十八课之后，正是这一条决定了整个技术栈能否工作。模型从来不只是权重；它是权重加上那条产生其输入的、确切的像素流水线——一起做版本管理、一起监控、一起上线。

Interview prompts

面试题

• Design a real-time license-plate detection and OCR system. (§5: task = box + text; metric weighs FN heavily, hard latency SLA; detect → rectify → OCR; edge/cloud per privacy; quantize + cascade for the budget; monitor by region/lighting slices.)设计一个实时车牌检测与 OCR 系统。(§5：任务 = 边界框 + 文本；指标高度看重假阴性（FN），有硬性延迟 SLA；检测 → 矫正 → OCR；按隐私决定边缘/云；用量化 + 级联塞进预算；按地区/光照切片监控。)
• Design a product-image duplicate-detection system for a marketplace. (§5 + lesson 12: embed + ANN index; dedup by perceptual hash to avoid leakage; threshold + human review for borderline.)为一个电商平台设计一个商品图像去重系统。(§5 + 第 12 课：嵌入 + ANN 索引；用感知哈希去重以避免泄漏；对临界情形用阈值 + 人工复核。)
• Design a medical-image triage model with human review. (§4–5: FN-dominant cost, human-in-the-loop as a required gate, strict privacy/retention, per-demographic slice floors.)设计一个带人工复核的医学影像分诊模型。(§4–5：假阴性主导的成本，人机协同作为必需关卡，严格的隐私/留存，按人群切片的下限。)
• A model regressed after launch but offline metrics are unchanged — what do you inspect? (§4: preprocessing/train-serve skew, threshold/NMS shipped separately, input distribution drift — not the weights.)一个模型上线后退化了，但离线指标没变——你会检查什么？(§4：预处理/训练-服务偏斜、单独上线的阈值/NMS、输入分布漂移——而不是权重。)
• PTQ vs QAT — when do you use each? (§2: PTQ first for a fast INT8/FP8 win; QAT when PTQ drops accuracy past the floor.)PTQ vs QAT——各自何时使用？(§2：先用 PTQ 快速拿下 INT8/FP8 的收益；当 PTQ 把精度掉到下限以下时用 QAT。)
• Why optimize p99 rather than mean latency? (§3: the tail is what multi-call pages and heavy users hit; worked example shows mean 25 ms hiding a 500 ms tail.)为什么优化 p99 而不是均值延迟？(§3：尾部才是多调用页面和重度用户会撞上的东西；具体例子表明 25 ms 的均值藏着一条 500 ms 的尾部。)
• Explain cascades / early-exit and estimate the speedup. (§2: cheap stage resolves easy inputs, escalates hard ones; worked 10 ms avg vs 50 ms always-big.)解释级联 / 提前退出并估算提速。(§2：廉价阶段解决简单输入，把难的上交；算得平均 10 ms，对比始终用大模型的 50 ms。)
• Why do imperceptible adversarial perturbations fool a CV model, and what's the standard defense? (§1.5: the boundary is constrained only where training data put pressure on it, so the many unconstrained high-dim directions accumulate small coordinated nudges — bounded in an L∞ ball, e.g. ε=8/255 — into a logit flip; FGSM steps along sign(∇_xℒ); defense is adversarial training on the attacked inputs. Distinct from non-adversarial corruption robustness, measured as mCE on ImageNet-C.)为什么人眼不可察觉的对抗扰动能骗过计算机视觉模型，标准防御是什么？(§1.5：边界只在训练数据对它施压的地方受约束，于是众多无约束的高维方向把微小、协调一致的轻推——约束在一个 L∞ 球内，例如 ε=8/255——累积成一次 logit 翻转；FGSM 沿 sign(∇_xℒ) 走一步；防御是在被攻击输入上做对抗训练。这与非对抗的损坏鲁棒性不同，后者在 ImageNet-C 上以 mCE 度量。)