Classification and recognition

分类与识别

Lesson 07 gave us a trained backbone and a recipe that does not lie. The simplest task we can hand it is also the most fundamental: turn one image into one label. To do that we need the loss that every CV reader must be able to derive — cross-entropy from softmax — and then the harder question of whether to believe the probability the model prints.

第 07 课给了我们一个训练好的骨干网络，以及一套不会骗人的配方。我们能交给它的最简单的任务，同时也是最根本的任务：把一张图像变成一个标签。要做到这一点，我们需要每一位学计算机视觉的人都必须会推导的那个损失——由 softmax 导出的交叉熵——然后是更棘手的问题：模型打印出来的那个概率，到底该不该相信。

1 · From logits to a loss, derived

1 · 从 logit 到损失的推导

The backbone emits a vector of real-valued logits z = (z₁, …, z_K), one per class — unbounded scores, not probabilities. We need three things from a probability: each entry in [0,1], the entries summing to 1, and a monotone, differentiable map from logits. The standard choice meeting all three requirements is the softmax:

骨干网络输出一个由实数值 logit 组成的向量 z = (z₁, …, z_K)，每个类别一个——它们是无界的分数，而不是概率。我们对概率有三点要求：每一项都落在 [0,1] 内、所有项之和为 1，以及一个从 logit 出发单调、可微的映射。同时满足这三条要求的标准选择就是 softmax：

Exponentiation forces positivity; dividing by the sum forces normalization; the ratio is invariant to adding a constant to all logits (which is why implementations subtract max(z) for numerical stability). Now we need a loss. Maximum likelihood says: choose parameters that make the observed labels most probable. For one example with true class y, the likelihood is just p_y — the predicted probability of the true class. Why the negative log? The log is monotone, so taking it leaves the argmax unchanged while turning the product of per-example likelihoods (over the dataset) into a sum we can differentiate term by term; and negating turns "maximize likelihood" into "minimize a loss". So we minimize the negative log-likelihood:

指数运算强制了正性；除以总和强制了归一化；这个比值对给所有 logit 加上同一个常数保持不变（这正是各种实现为了数值稳定性会先减去 max(z) 的原因）。现在我们需要一个损失。极大似然的说法是：选择使观测到的标签最可能出现的参数。对于一个真实类别为 y 的样本，其似然就是 p_y——真实类别的预测概率。为什么要取负对数？log 是单调的，所以取对数不会改变 argmax，同时把（在整个数据集上的）逐样本似然之积变成一个可以逐项求导的和；而取负号则把"最大化似然"变成了"最小化一个损失"。于是我们最小化负对数似然：

That is cross-entropy. The name comes from writing the label as a one-hot distribution q (all mass on y): cross-entropy H(q,p) = −∑_c q_c log p_c collapses to −log p_y because every other q_c is zero. The loss is small when the model puts mass on the right class and grows without bound as p_y → 0 — it punishes confident mistakes savagely.

这就是交叉熵。这个名字来自把标签写成一个 one-hot 分布 q（全部概率质量都在 y 上）：交叉熵 H(q,p) = −∑_c q_c log p_c 会坍缩成 −log p_y，因为其余每一个 q_c 都为零。当模型把概率质量放到正确类别上时损失很小，而当 p_y → 0 时损失会无上界地增长——它会残酷地惩罚自信的错误。

The gradient is the reason this loss is everywhere. Differentiate L with respect to the logits:

梯度正是这个损失无处不在的原因。对 logit 求 L 的导数：

The gradient is exactly prediction minus target — the same clean form as linear/logistic regression. No vanishing softmax derivative, no awkward constants; the network is pushed in proportion to how wrong its probability is.

这个梯度恰好就是预测值减目标值——和线性/逻辑回归一样干净的形式。没有会消失的 softmax 导数，也没有别扭的常数；网络被推动的幅度正比于它的概率错得有多离谱。

2 · One answer vs many: softmax vs sigmoid

2 · 一个答案还是多个：softmax 与 sigmoid

Softmax encodes a hard modeling assumption: the classes are mutually exclusive — exactly one is correct, so the probabilities compete for a fixed budget of 1. That is right for "cat vs dog vs car". It is wrong for an image that is both "beach" and "sunset" and "people". When labels are not mutually exclusive you have a multi-label problem, and you give each class its own independent sigmoid with its own binary cross-entropy:

Softmax 编码了一个很强的建模假设：各类别相互排斥——恰好有一个是正确的，所以各概率在总额为 1 的固定预算上相互竞争。对于"猫 vs 狗 vs 车"这是对的。但对于一张同时是"海滩""日落"和"人物"的图像，它就错了。当标签并非相互排斥时，你面对的是一个多标签问题，此时你给每个类别各配一个独立的 sigmoid，各自使用二元交叉熵：

Picking softmax for a genuinely multi-label problem is a quiet, common bug: the model is forced to suppress one true tag to raise another.

给一个本质上是多标签的问题选用 softmax，是一个隐蔽而常见的 bug：模型被迫压低一个真实标签，才能抬高另一个。

3 · Metrics for the world that is not balanced

3 · 面向不平衡世界的评价指标

Top-1 accuracy — fraction where argmax is correct — is the right metric only when classes are balanced and one answer is expected. Three realities break it:

Top-1 准确率——argmax 正确的样本比例——只有在类别平衡且只期望一个答案时才是合适的指标。三个现实会把它打破：

Top-k accuracy credits a prediction if the true class is among the k highest logits. It is meaningful when classes are genuinely confusable or fine-grained (a hundred dog breeds), and misleading when used to paper over a model that simply cannot decide — a system that acts on the top-1 gets no benefit from a good top-5.

Top-k 准确率：只要真实类别落在 logit 最高的 k 个之中，就算这个预测正确。当类别确实容易混淆或属于细粒度（上百种狗的品种）时，它是有意义的；而当它被用来掩盖一个根本下不了决心的模型时，它就具有误导性——一个只依据 top-1 采取行动的系统，从漂亮的 top-5 中得不到任何好处。

Long tail. Real label distributions are heavy-tailed: a few head classes hold most of the images. A model that ignores every rare class can still post high top-1 accuracy because the head dominates the average. The fix is to report macro-F1 — the F1 score (harmonic mean of precision (of what you flagged positive, the fraction truly positive) and recall (of all true positives, the fraction you found)) computed per class and then averaged with equal weight, so a rare class counts as much as a common one — and per-class recall. Training-side levers (re-weighting, resampling) come from lesson 07; the metric is how you see the tail.

长尾。真实的标签分布是重尾的：少数头部类别占据了绝大部分图像。一个忽略所有稀有类别的模型，仍然可以给出很高的 top-1 准确率，因为头部主导了平均值。解决办法是报告宏 F1——即 F1 分数（精确率（在你判为正例的样本中真正为正的比例）与召回率（在所有真正的正例中你找回的比例）的调和平均）逐类别计算后再等权平均，这样一个稀有类别的分量就和一个常见类别一样重——外加逐类别的召回率。训练侧的手段（重加权、重采样）来自第 07 课；而指标是你看见长尾的方式。

Open-set / OOD. Closed-set classification assumes every serving image belongs to one of the K known classes. Deployment violates this constantly — a novel object, a blurry frame, a different domain. Open-set recognition needs a reject option: abstain when the input is out-of-distribution (OOD). Signals include max-softmax probability (weak but cheap); the energy score — a single scalar read off the logits, −log∑_j exp(z_j) (the negative of the same logsumexp from the cross-entropy denominator), which runs low for in-distribution inputs and high for OOD — an in-distribution image fires some class strongly, making ∑exp large and the energy very negative; an OOD image fires nothing strongly, so the sum stays small and energy stays high; embedding distance to training clusters — an unfamiliar input lands far from every class cluster in feature space, so a large nearest-cluster distance flags it as OOD; or a dedicated OOD detector. The catch: a softmax can be confidently wrong on garbage, which is exactly why the next section matters.

开集 / OOD。闭集分类假设每一张服务时的图像都属于 K 个已知类别之一。部署会不断违反这一假设——一个新奇的物体、一帧模糊的画面、一个不同的域。开集识别需要一个拒识选项：当输入是分布外（OOD）时就弃权。可用的信号包括最大 softmax 概率（弱但廉价）；能量分数——一个从 logit 直接读出的标量 −log∑_j exp(z_j)（即交叉熵分母里那个 logsumexp 取负），它对分布内输入偏低、对 OOD 偏高——分布内图像会强烈激活某个类别，使 ∑exp 很大、能量非常负；OOD 图像没有任何类别被强烈激活，于是这个和保持很小、能量保持很高；到训练簇的嵌入距离——一个陌生的输入在特征空间中离每一个类别簇都很远，因此较大的最近簇距离会把它标记为 OOD；或者用一个专门的 OOD 检测器。麻烦之处在于：softmax 可能对垃圾输入自信地判错，而这正是下一节之所以重要的原因。

4 · Calibration — can you believe the number?

4 · 置信度校准——那个数字可信吗？

A model is calibrated if, among all predictions it makes with confidence 0.8, about 80% are actually correct. Accuracy and calibration are different axes: a model can rank perfectly (high accuracy) yet be systematically overconfident (bad calibration). This matters whenever the confidence drives a downstream decision — routing to a human reviewer, abstaining, thresholding, or fusing with another sensor.

一个模型是校准良好的，是指在它所有以置信度 0.8 给出的预测中，大约 80% 确实是正确的。准确率和校准是两个不同的维度：一个模型可以排序完美（高准确率），却系统性地过度自信（校准很差）。只要置信度会驱动下游的某个决策——转交人工复核、弃权、阈值判定，或与另一个传感器融合——这一点就很重要。

Expected Calibration Error (ECE) measures the gap. Bin predictions by confidence into M bins; in each bin compare the average confidence to the empirical accuracy; average the absolute gaps weighted by bin size:

期望校准误差（ECE）度量的就是这个差距。按置信度把预测分到 M 个箱（bin）里；在每个箱内比较平均置信度与经验准确率；再按箱的大小加权，对这些绝对差距求平均：

Worked: a bin holds 100 predictions with average confidence 0.90 but only 75 are correct (accuracy 0.75). That bin contributes (100/n)·|0.75 − 0.90| = (100/n)·0.15 — the model is 15 points overconfident there. A reliability diagram plots accuracy vs confidence per bin; perfect calibration is the diagonal, and bars below it are overconfidence (the usual case for modern deep nets).

算一算：某个箱里有 100 个预测，平均置信度为 0.90，但只有 75 个正确（准确率 0.75）。这个箱贡献 (100/n)·|0.75 − 0.90| = (100/n)·0.15——模型在这里过度自信了 15 个百分点。可靠性图逐箱绘制准确率对置信度；完美校准就是那条对角线，落在它下方的柱子就是过度自信（现代深度网络的常见情形）。

Brier score is a complementary, threshold-free measure: the mean squared error between the predicted probability vector and the one-hot truth, ∑_c(p_c − q_c)² averaged over examples. Reusing the worked probabilities p = (0.659, 0.242, 0.099) with one-hot truth q = (1, 0, 0): Brier = (0.659 − 1)² + 0.242² + 0.099² = 0.1163 + 0.0586 + 0.0098 ≈ 0.185 for that single example. It decomposes into calibration + refinement (refinement ≈ sharpness — how spread out, i.e. decisive, the predictions are), so it rewards both honesty and sharpness at once.

Brier 分数是一个互补的、无需阈值的度量：预测概率向量与 one-hot 真值之间的均方误差，即 ∑_c(p_c − q_c)² 在样本上取平均。沿用前面算例的概率 p = (0.659, 0.242, 0.099) 和 one-hot 真值 q = (1, 0, 0)：对那一个样本，Brier = (0.659 − 1)² + 0.242² + 0.099² = 0.1163 + 0.0586 + 0.0098 ≈ 0.185。它可以分解为校准项 + 精细度项（精细度 ≈ 锐度——预测有多分散，也就是多果断），因此它同时奖励诚实与锐度。

Temperature scaling is the simplest fix. After training, divide the logits by a single learned scalar T > 0 before softmax, fit on the validation set to minimize NLL:

温度缩放是最简单的修复方法。训练之后，在做 softmax 之前把 logit 除以一个学到的标量 T > 0，并在验证集上拟合它以最小化 NLL：

Because it divides all logits by the same number, it cannot change the argmax — accuracy and top-k are untouched — it only rescales confidence. That is the whole appeal: free recalibration with one parameter. The widget below lets you drag T and watch ECE fall.

因为它把所有 logit 都除以同一个数，所以它无法改变 argmax——准确率和 top-k 都原封不动——它只是重新缩放置信度。这正是它的全部魅力：用一个参数免费完成重新校准。下面的小部件让你拖动 T，看着 ECE 下降。

Label smoothing attacks the same overconfidence at training time. Instead of a hard one-hot target, smear a little mass ε uniformly over all classes: the true class gets 1−ε+ε/K, every other gets ε/K (typical ε = 0.1). Cross-entropy against this soft target stops the network from driving the correct logit to +∞ and the rest to −∞, which is what produces the overconfidence in the first place. It usually improves both accuracy and calibration, and it is the same idea as Mixup/CutMix's label blending from lesson 07.

标签平滑在训练阶段攻击同样的过度自信。不再使用一个硬的 one-hot 目标，而是把一小份概率质量 ε 均匀地抹到所有类别上：真实类别得到 1−ε+ε/K，其余每个得到 ε/K（典型取 ε = 0.1）。针对这个软目标做交叉熵，能阻止网络把正确的 logit 推向 +∞、把其余的推向 −∞，而后者正是过度自信的根源。它通常能同时改善准确率和校准，与第 07 课 Mixup/CutMix 的标签混合是同一个思路。

// Each prediction is a 3-logit vector + true class. Temperature divides logits.
function softmax(z, T){
  const m = Math.max(...z);
  const e = z.map(v => Math.exp((v - m)/T));
  const s = e.reduce((a,b)=>a+b,0);
  return e.map(v => v/s);
}
function rescore(preds, T){                 // confidence = max prob; correct = argmax==y
  return preds.map(p => {
    const q = softmax(p.z, T);
    let am = 0; for (let i=1;i<q.length;i++) if (q[i]>q[am]) am=i;
    return { conf:q[am], correct: am===p.y };
  });
}
function ece(scored, M){                     // M equal-width confidence bins
  const bins = Array.from({length:M}, ()=>({n:0,acc:0,conf:0}));
  for (const s of scored){
    const b = Math.min(M-1, Math.floor(s.conf*M));
    bins[b].n++; bins[b].acc += s.correct?1:0; bins[b].conf += s.conf;
  }
  let e = 0;
  for (const b of bins) if (b.n) e += (b.n/scored.length)*Math.abs(b.acc/b.n - b.conf/b.n);
  return {ece:e, bins};
}

Where this points next

这将指向何处

Classification answers what; many tasks need what AND where, and that single extra demand is what forces the box machinery — IoU, NMS — of lesson 09. We can now name an image, train the naming with a loss we derived, and decide whether to trust the answer. But a single label is structurally blind: it cannot say where the cat is, cannot count three cats, and is forced to pick one when the scene holds many objects. Lesson 09 breaks that ceiling — it predicts a box and a class for every object, which immediately raises two new first-principles problems this lesson did not have: how to measure overlap between a predicted and true box (IoU, worked by hand), and how to collapse a flood of overlapping candidate boxes into clean detections (non-maximum suppression, as a mechanical algorithm). The classifier head you just built becomes one branch of the detector.

分类回答是什么；而许多任务需要是什么"以及"在哪里，正是这一个额外的要求逼出了第 09 课的边界框机制——交并比（IoU）、非极大值抑制（NMS）。现在我们能给一张图像命名、用我们推导出的损失训练这个命名，并判断是否该相信这个答案。但单个标签在结构上是"盲"的：它说不出猫在哪里，数不出三只猫，而当场景中有多个物体时它被迫只挑一个。第 09 课打破了这个天花板——它为每一个物体预测一个边界框和一个类别，这立刻带来了本课没有的两个新的第一性原理问题：如何度量预测框与真值框的重叠（IoU，亲手算），以及如何把一堆相互重叠的候选框收拢成干净的检测结果（非极大值抑制，作为一个机械的算法）。你刚刚搭好的分类头会变成检测器的一条分支。

Interview prompts

面试题

• Derive cross-entropy from softmax and give its gradient. (L = −log p_y = −z_y + log∑e^z_j; ∂L/∂z_c = p_c − q_c.)从 softmax 推导交叉熵并给出它的梯度。(L = −log p_y = −z_y + log∑e^z_j；∂L/∂z_c = p_c − q_c。)
• Softmax vs sigmoid head — when each? (mutually-exclusive classes → softmax; independent multi-label tags → per-class sigmoid + BCE.)softmax 头还是 sigmoid 头——分别在什么时候用？(相互排斥的类别 → softmax；相互独立的多标签标签 → 逐类别 sigmoid + BCE。)
• When is top-5 accuracy meaningful vs misleading? (meaningful for confusable/fine-grained classes; misleading when a system must act on top-1.)top-5 准确率什么时候有意义、什么时候有误导性？(对易混淆/细粒度的类别有意义；当系统必须依据 top-1 行动时具有误导性。)
• How do you handle long-tail classification? (macro-F1 / per-class recall to see it; re-weight or resample to fix it, watching head precision and calibration.)你如何处理长尾分类？(用宏 F1 / 逐类别召回率去看见它；用重加权或重采样去修复它，同时盯住头部类别的精确率和校准。)
• What is open-set recognition? (a reject option for OOD inputs since closed-set assumes all K classes are present.)什么是开集识别？(为 OOD 输入设置的拒识选项，因为闭集假设 K 个类别全都在场。)
• Why can a classifier be accurate but poorly calibrated, and how do you fix it? (it ranks correctly but is overconfident; measure with ECE/Brier, fix with temperature scaling and label smoothing.)为什么一个分类器可以准确却校准很差，你如何修复？(它排序正确却过度自信；用 ECE/Brier 度量，用温度缩放和标签平滑修复。)
• Why does temperature scaling not change accuracy? (dividing all logits by one scalar preserves their order, so the argmax is unchanged.)为什么温度缩放不会改变准确率？(把所有 logit 除以同一个标量会保持它们的顺序，所以 argmax 不变。)