Keypoints, pose, and tracking primitives

关键点、姿态与跟踪基础

Lesson 10 gave us a label at every pixel — but a mask is an amorphous region. It cannot say "this pixel is the left elbow" or "this is the same person who was here last frame." Two new output shapes are needed: sparse structured points (joints, landmarks) and persistent identity over time. This lesson builds both, and ends by noticing that identity-by-appearance is secretly a distance problem — the subject of lesson 12.

第 10 课让我们在每个像素上都得到了一个标签——但掩码是一片无定形的区域。它说不出"这个像素是左肘"或"这是上一帧还在这里的同一个人"。我们需要两种新的输出形态：稀疏的结构化点（关节、关键点）和随时间保持的身份。本课把两者都构建出来，并在结尾指出：以外观来判定身份，其实暗地里是一个距离问题——这正是第 12 课的主题。

1 · Keypoints: heatmaps vs coordinate regression

1 · 关键点：热力图 vs 坐标回归

A keypoint model predicts specific landmark locations — the 17 body joints of a person, 68 facial landmarks, the corners of a license plate. The naive approach is coordinate regression: have the network output 2K numbers (x,y for each of K keypoints) and train with L2 loss. It works, but it has three problems. It is spatially destructive — squashing a feature map to a single (x,y) via fully-connected layers discards the spatial structure the convolutions worked to build. It cannot represent uncertainty — one point, no notion of "I'm confident it's here, unsure between there and there." And it cannot represent multiple modes — if two joints are plausible, the L2-optimal answer is the average, which may be a location that is wrong for both.

关键点模型预测特定的地标位置——人体的 17 个关节、68 个人脸关键点、车牌的四个角点。最朴素的做法是坐标回归：让网络输出 2K 个数（K 个关键点各自的 x、y），并用 L2 损失训练。它能用，但有三个问题。它破坏空间结构——通过全连接层把特征图压成单个 (x,y)，丢弃了卷积辛苦构建起来的空间结构。它无法表示不确定性——只有一个点，没有"我确信在这里，但在那里和那里之间拿不准"的概念。它也无法表示多个模态——如果两个关节都合理，L2 最优解是取平均，而这个位置可能对两者都是错的。

The dominant alternative is heatmap prediction. For each keypoint the network outputs a full-resolution map (one channel per keypoint) whose value at each pixel is "how likely the keypoint is here." This keeps the prediction spatial, naturally represents confidence (peak height) and uncertainty (peak width), and can even show multiple candidate peaks. It is the reason heatmap methods (from the Stacked Hourglass network onward) beat direct regression on pose benchmarks.

占主导地位的替代方案是热力图预测。对每个关键点，网络输出一张全分辨率的图（每个关键点一个通道），其在每个像素上的值表示"关键点在这里的可能性有多大"。这让预测保持空间性，天然地表示置信度（峰值高度）和不确定性（峰的宽度），甚至能呈现多个候选峰。这正是热力图方法（从 Stacked Hourglass 网络起）在姿态基准上胜过直接回归的原因。

1a · Building the Gaussian target (the actual mechanic)

1a · 构造高斯目标（真正的机理）

To train with a heatmap you need a target heatmap, and a single bright pixel at the true location is a bad target — almost all pixels are zero, so the loss is dominated by "predict zero everywhere" and gradients near the keypoint are tiny. Instead we paint a 2D Gaussian bump centered on the ground-truth keypoint (x₀, y₀):

要用热力图训练，你需要一张目标热力图，而在真实位置放一个亮像素是个糟糕的目标——几乎所有像素都是零，于是损失被"处处预测零"主导，关键点附近的梯度极小。我们改为在真值关键点 (x₀, y₀) 处画一个二维高斯凸包：

The Gaussian gives a smooth gradient that grows as you approach the true location, so the network gets useful signal in a neighborhood, not just at one pixel. The spread σ (often 1–3 px at the heatmap resolution) is a deliberate knob: small σ demands precise localization (sharp, hard to train); large σ is forgiving but blurs nearby joints together. Training loss is then a per-pixel L2 (MSE) between predicted and target heatmaps.

高斯给出一个平滑的梯度，越靠近真实位置梯度越大，于是网络在一个邻域内都能得到有用的信号，而不只是在单个像素处。展宽 σ（在热力图分辨率下通常为 1–3 px）是一个刻意设置的旋钮：小 σ 要求精确定位（尖锐、难训练）；大 σ 更宽容，但会把邻近的关节模糊到一起。训练损失就是预测热力图与目标热力图之间的逐像素 L2（MSE）。

1b · Soft-argmax: decoding a coordinate, differentiably

1b · Soft-argmax：以可微方式解码坐标

At inference you must turn the heatmap back into a coordinate. The obvious choice is argmax — take the brightest pixel — but it has two flaws: it is limited to integer pixel resolution (the heatmap is often downsampled, so one pixel = several image pixels), and it is not differentiable, so you cannot back-propagate a coordinate loss through it. Soft-argmax fixes both. Normalize the heatmap with a softmax so it becomes a probability distribution over locations, then take the expected coordinate:

在推理时，你必须把热力图变回一个坐标。显而易见的选择是 argmax——取最亮的像素——但它有两个缺陷：它被限制在整数像素分辨率上（热力图常被下采样，所以一个像素 = 若干图像像素），而且它不可微，因此你无法通过它反向传播坐标损失。Soft-argmax 同时修复了这两点。用 softmax 对热力图归一化，使其成为位置上的概率分布，然后取期望坐标：

Worked example. A tiny 1-D heatmap over pixels {0,1,2,3} with softmax probabilities [0.05, 0.15, 0.60, 0.20]. Argmax says pixel 2. Soft-argmax says 0·0.05 + 1·0.15 + 2·0.60 + 3·0.20 = 0 + 0.15 + 1.20 + 0.60 = 1.95 — a sub-pixel estimate pulled slightly below 2 by the mass at pixel 1, and a smooth function of the heatmap values, so gradients flow. That sub-pixel precision and differentiability is why soft-argmax is the standard decoder.

算个具体例子。一张定义在像素 {0,1,2,3} 上的一维小热力图，softmax 概率为 [0.05, 0.15, 0.60, 0.20]。Argmax 给出像素 2。Soft-argmax 给出 0·0.05 + 1·0.15 + 2·0.60 + 3·0.20 = 0 + 0.15 + 1.20 + 0.60 = 1.95——一个亚像素估计，被像素 1 处的概率质量稍微拉到 2 以下，且它是热力图值的平滑函数，所以梯度能流动。正是这种亚像素精度与可微性，使 soft-argmax 成为标准的解码器。

For multi-person pose there are two regimes. Top-down: detect each person (lesson 09), crop, run single-person pose on each crop — accurate, but cost scales with the number of people. Bottom-up (e.g. OpenPose): detect all keypoints of all people at once, then group them into skeletons — constant cost regardless of crowd size, but grouping is the hard part. The grouping signal comes from part-affinity fields: per-limb 2D vector fields the network predicts, where each field encodes the direction of the bone connecting two keypoints (e.g. elbow→wrist). Integrating a candidate field along the line between a detected elbow and a detected wrist scores how well they line up, giving the grouping step a learned signal for which wrist belongs to which elbow rather than a brittle nearest-distance guess.

对于多人姿态，有两种范式。自顶向下：先检测每个人（第 09 课），裁剪，再对每个裁剪块运行单人姿态估计——精确，但代价随人数线性增长。自底向上（如 OpenPose）：一次性检测所有人的所有关键点，再把它们分组成骨架——无论人群多大代价恒定，但分组才是难点。分组信号来自部件亲和场（part-affinity fields, PAF）：网络预测的逐肢体二维向量场，每个场编码连接两个关键点的骨骼的方向（如 肘→腕）。沿着检测到的肘与检测到的腕之间的连线对候选场做积分，就能给它们的对齐程度打分，从而为分组步骤提供一个学习到的信号——判断哪只腕属于哪只肘，而不是靠脆弱的最近距离猜测。

2 · Pose metrics: PCK and OKS

2 · 姿态度量：PCK 与 OKS

PCK = Percentage of Correct Keypoints. A predicted keypoint counts as correct if it falls within a threshold distance of the ground truth, and PCK is the fraction that do. The threshold is made scale-relative — typically a fraction of a reference length like head size or torso diameter (PCKh@0.5 = within 50% of head segment length) — so a 10-pixel error is "correct" on a large nearby person and "wrong" on a small distant one. PCK is simple and interpretable but has a hard pass/fail boundary.

PCK = 正确关键点百分比（Percentage of Correct Keypoints）。如果一个预测关键点落在真值的某个阈值距离以内，就算作正确，PCK 就是正确的那部分占比。阈值被设成尺度相对的——通常是某个参考长度（如头部尺寸或躯干直径）的一个比例（PCKh@0.5 = 在头部线段长度的 50% 以内）——所以 10 像素的误差在近处的大人身上算"正确"，在远处的小人身上算"错误"。PCK 简单、可解释，但有一个硬性的通过/不通过边界。

OKS = Object Keypoint Similarity, the COCO standard, is a smooth analogue that plays the role IoU plays for boxes (so you can compute a keypoint "AP" by thresholding OKS). For one person it is a Gaussian on the per-keypoint error, averaged over visible keypoints.

OKS = 目标关键点相似度（Object Keypoint Similarity），COCO 的标准，是一个平滑的类比，扮演着 IoU 对边界框所扮演的角色（这样你就能通过对 OKS 设阈值来计算关键点"AP"）。对单个人，它是关于逐关键点误差的一个高斯，在可见关键点上取平均。

Read the symbols first, then the formula. Each keypoint contributes a number between 0 and 1: 1 for a perfect hit, decaying toward 0 as the prediction drifts away. The decay is a Gaussian whose width is set by the person's size s (the square root of the segment area) times a per-joint tolerance k_i (eyes are pinpointed precisely → small k; hips are fuzzy → large k). So the quantity that matters is the error measured in units of s·k_i, not raw pixels.

先读符号，再读公式。每个关键点贡献一个介于 0 与 1 之间的数：完美命中为 1，随着预测偏离而衰减趋于 0。衰减是一个高斯，其宽度由人的尺寸 s（线段面积的平方根）乘以一个逐关节容差 k_i 决定（眼睛能被精确定位 → 小 k；髋部模糊 → 大 k）。所以真正重要的量，是以 s·k_i 为单位度量的误差，而不是原始像素。

One worked term. Take a keypoint with tolerance k_i = 0.05 on a person of scale s = 100 px, predicted d_i = 5 px from truth. The natural error unit is s·k_i = 100·0.05 = 5 px — so this prediction is exactly "one tolerance" off. Its term is exp(−5² / (2·100²·0.05²)) = exp(−25/50) = exp(−0.5) ≈ 0.61. Land the same joint at d_i = 0 and the term is exp(0) = 1; push it to d_i = 15 px (three tolerances) and it falls to exp(−4.5) ≈ 0.011. The per-keypoint scores are then averaged over visible joints. With that picture, the formula reads directly:

算一项具体的。取一个容差 k_i = 0.05 的关键点，在尺度 s = 100 px 的人身上，预测位置离真值 d_i = 5 px。自然的误差单位是 s·k_i = 100·0.05 = 5 px——所以这个预测恰好偏了"一个容差"。它的这一项是 exp(−5² / (2·100²·0.05²)) = exp(−25/50) = exp(−0.5) ≈ 0.61。把同一个关节落在 d_i = 0，这一项就是 exp(0) = 1；把它推到 d_i = 15 px（三个容差），它就降到 exp(−4.5) ≈ 0.011。这些逐关键点的分数随后在可见关节上取平均。有了这幅图景，公式就可以直接读懂：

Re-reading the symbols against the worked number: d_i is the pixel distance between predicted and true keypoint i; v_i is its visibility flag (unlabeled/occluded keypoints are skipped); k_i is the per-joint constant; and s is the object scale — the square root of the person's segment area.

对照那个具体数字重读符号：d_i 是预测关键点与真实关键点 i 之间的像素距离；v_i 是它的可见性标记（未标注/被遮挡的关键点会被跳过）；k_i 是逐关节常数；s 是目标尺度——人体线段面积的平方根。

3 · Tracking-by-detection

3 · 检测跟踪（tracking-by-detection）

Now the second new output: identity over time. The dominant paradigm is tracking-by-detection. Run a detector (lesson 09) independently on every frame, then link detections across frames into persistent tracks. The detector is a black box; tracking is the linking problem. Three components do the work.

现在来看第二种新输出：随时间保持的身份。占主导地位的范式是检测跟踪（tracking-by-detection）。在每一帧上独立运行一个检测器（第 09 课），再把跨帧的检测结果连接成持久的轨迹（tracks）。检测器是个黑盒；跟踪就是这个连接问题。三个组件承担了这项工作。

3a · Kalman filter — a motion model that predicts then corrects

3a · 卡尔曼滤波——先预测后校正的运动模型

Between frames, an object moves. A Kalman filter maintains a belief about each track's state — typically position and velocity, e.g. (x, y, scale, aspect, plus their rates) — as a Gaussian (a mean estimate plus a covariance encoding uncertainty), and runs a two-step cycle every frame:

在帧与帧之间，物体会移动。卡尔曼滤波把对每条轨迹状态的信念——通常是位置与速度，例如 (x, y, 尺度, 长宽比，以及它们的变化率)——维护为一个高斯（一个均值估计加上一个编码不确定性的协方差），并在每一帧运行一个两步循环：

• Predict. Advance the state with a constant-velocity assumption: new position ≈ old position + velocity·Δt. Because prediction extrapolates, uncertainty grows — the covariance inflates. This gives a prior on where the object should appear next, before looking at the new frame.预测。用匀速假设推进状态：新位置 ≈ 旧位置 + 速度·Δt。因为预测是外推，不确定性会增长——协方差膨胀。这在看新一帧之前，给出了物体应当出现在何处的一个先验。
• Update. When a detection is matched to this track, blend the prediction with the measurement, weighted by their relative uncertainties (the Kalman gain). A confident measurement pulls the estimate toward itself and shrinks the uncertainty; a noisy one barely moves it. Concretely: in 1-D the gain is K = σ²_pred / (σ²_pred + σ²_meas), and the new estimate is old + K·(measurement − old). If measurement variance ≪ prediction variance, K ≈ 1 → trust the detector (snap onto the measurement); if measurement variance ≫ prediction variance, K ≈ 0 → trust physics (keep coasting on the prediction). With σ²_pred = 9, σ²_meas = 1, K = 9/10 = 0.9 — the estimate jumps 90% of the way to the detection.更新。当一个检测被匹配到这条轨迹时，把预测与测量融合，按它们各自的不确定性加权（卡尔曼增益）。一个可信的测量会把估计拉向自身，并收缩不确定性；一个含噪的测量几乎不移动它。具体地：在一维中增益为 K = σ²_pred / (σ²_pred + σ²_meas)，新估计为 old + K·(measurement − old)。若测量方差 ≪ 预测方差，K ≈ 1 → 相信检测器（贴到测量上）；若测量方差 ≫ 预测方差，K ≈ 0 → 相信物理（继续沿预测滑行）。当 σ²_pred = 9, σ²_meas = 1 时，K = 9/10 = 0.9——估计朝检测跳过了 90% 的路程。

The intuition that matters for interviews: the filter is a running compromise between "where physics says it should be" and "where the detector says it is," and it weights them by how much it trusts each. This is what lets a track coast through a missed detection (predict-only for a few frames) and snap back when the object reappears. It fails under abrupt, non-linear motion (the constant-velocity prior is wrong) or a bad Δt.

面试中真正重要的直觉：滤波器是"物理说它应该在哪"与"检测器说它在哪"之间的一个动态折中，并按对各自的信任程度加权。正是这一点让一条轨迹能滑行穿过一次漏检（连续几帧只预测），并在物体重新出现时贴回去。它在突兀的非线性运动下（匀速先验错误）或 Δt 取错时会失效。

3b · Data association via Hungarian matching

3b · 通过匈牙利匹配做数据关联

Each frame you have M existing tracks (each with a Kalman prediction) and N new detections. Which detection belongs to which track? Build an M×N cost matrix where entry (i,j) is the cost of assigning detection j to track i — low cost = good match. Then solve for the assignment that minimizes total cost with each track getting at most one detection and vice versa. That is the classic assignment problem, solved optimally in polynomial time by the Hungarian algorithm (Kuhn–Munkres). Greedily picking the best pair one at a time can be globally suboptimal; the Hungarian method finds the best overall matching. Worked (the cost matrix below): a greedy matcher grabs the single cheapest edge first — A→1 at cost 0.1 — which then strands track B with its only remaining option at cost 0.8, for a total of 0.1 + 0.8 = 0.9. The Hungarian algorithm instead optimizes the pair jointly and takes A→1 (0.1) together with B→3 (0.2), total 0.1 + 0.2 = 0.3 — a third of the greedy cost, because it is willing to pass on a locally tempting edge to keep the global assignment cheap.

每一帧你有 M 条已有轨迹（各自带一个卡尔曼预测）和 N 个新检测。哪个检测属于哪条轨迹？构建一个 M×N 的代价矩阵，其中 (i,j) 项是把检测 j 指派给轨迹 i 的代价——代价越低 = 匹配越好。然后求解一种指派，使总代价最小，且每条轨迹至多得到一个检测、反之亦然。这就是经典的指派问题，可由匈牙利算法（Kuhn–Munkres）在多项式时间内最优求解。一次贪心地挑一对最好的，可能在全局上并非最优；匈牙利方法找到的是最好的整体匹配。算一遍（下方代价矩阵）：贪心匹配器先抓住那条唯一最便宜的边——A→1，代价 0.1——这就把轨迹 B 困在它仅剩的、代价 0.8 的选项上，合计 0.1 + 0.8 = 0.9。匈牙利算法则联合优化这一对，取 A→1（0.1）连同 B→3（0.2），合计 0.1 + 0.2 = 0.3——只有贪心代价的三分之一，因为它愿意放过一条局部诱人的边，以保持全局指派的低代价。

3c · Re-ID embeddings — appearance when motion isn't enough

3c · Re-ID 嵌入——当运动信息不够时用外观

What goes into the cost? Simple trackers (SORT) use only motion: cost = 1 − IoU between the Kalman prediction and the detection box. That breaks under occlusion — when two people cross, their boxes overlap and the predictions get swapped, causing an identity switch. The fix (DeepSORT and successors) adds an appearance term: a small network embeds each detection's crop into a Re-Identification (Re-ID) feature vector, and the association cost includes the distance between a detection's embedding and the track's stored appearance. Now even if motion is ambiguous, "this crop looks like the person we lost behind the pillar" recovers the identity. Track management wraps it all: birth (a detection unmatched for a few frames becomes a new track), death (a track unmatched too long is retired), and a grace period so a track survives brief occlusion.

代价里装的是什么？简单的跟踪器（SORT）只用运动：代价 = 1 − 卡尔曼预测框与检测框之间的 IoU。这在遮挡下会失效——当两个人交叉时，他们的框重叠、预测被互换，造成身份切换（identity switch）。修复办法（DeepSORT 及其后续）加入一个外观项：一个小网络把每个检测的裁剪块嵌入成一个重识别（Re-Identification, Re-ID）特征向量，关联代价则包含检测嵌入与轨迹所存外观之间的距离。这样，即便运动是模糊的，"这个裁剪块看起来像我们在柱子后面跟丢的那个人"也能找回身份。轨迹管理把这一切包起来：诞生（一个连续几帧未匹配的检测成为新轨迹）、消亡（一条太久未匹配的轨迹被退役），以及一个宽限期，让轨迹能挺过短暂的遮挡。

3.5 · Interactive · tracking-by-detection loop

3.5 · 交互 · 检测跟踪循环

The predict → associate → update cycle is abstract on paper, so drive it by hand here. Each frame runs the three steps in order; watch the uncertainty ellipses grow on Predict and shrink on Update, and watch the Hungarian matching redraw on Associate. Then flip on the occlusion to make a detection drop out — without appearance the tracks swap identities; turn on the Re-ID term and the swap is repaired.

预测 → 关联 → 更新的循环在纸面上很抽象，所以在这里亲手驱动它。每一帧按顺序运行这三步；看不确定性椭圆在预测时变大、在更新时缩小，看匈牙利匹配在关联时重绘。然后打开遮挡，让一个检测掉出——没有外观时，两条轨迹会互换身份；打开 Re-ID 项，互换就被修复。

// one Kalman step per axis: predict inflates variance, update shrinks it via the gain
function predict(t){ t.x += t.vx; t.P += Q; }              // Q = process noise → ellipse grows
function update(t, z){                                      // z = matched detection position
  const K = t.P / (t.P + R);                                // Kalman gain, R = meas. variance
  t.x += K * (z - t.x);                                     // snap toward detection
  t.P  = (1 - K) * t.P;                                     // ellipse shrinks
}
// association cost: motion (gated distance) + optional appearance (Re-ID) term
function cost(track, det){
  let c = dist(track.pred, det);                            // SORT: motion only
  if (reidOn) c += APP * appearanceDist(track.color, det.color); // DeepSORT: + appearance
  return c;
}
// Hungarian picks the global min-cost 1-to-1 matching (here: brute force for 2x2)

4 · Tracking metrics: MOTA and IDF1

4 · 跟踪度量：MOTA 与 IDF1

Per-frame detection accuracy is not enough — a tracker must keep identities consistent over time, and two metrics capture different aspects of that.

逐帧的检测精度还不够——跟踪器必须让身份随时间保持一致，而有两个度量分别刻画了这件事的不同侧面。

MOTA = Multiple Object Tracking Accuracy. It sums the three error types over all frames and normalizes by the number of ground-truth objects:

MOTA = 多目标跟踪精度（Multiple Object Tracking Accuracy）。它把三类错误在所有帧上求和，再用真值目标的数量归一化：

where FN = misses, FP = false positives, and IDSW = identity switches. MOTA is detection-dominated — because misses and false positives usually vastly outnumber switches, MOTA mostly grades did you find the objects? and barely penalizes losing track of who's who. Worked example: over a sequence with ΣGT = 100 ground-truth object instances, suppose the tracker accumulates ΣFN = 20 misses, ΣFP = 15 false positives, and ΣIDSW = 5 switches. Then MOTA = 1 − (20 + 15 + 5)/100 = 1 − 40/100 = 0.60. Note the errors are summed without weighting, so the metric can punch through zero: if a worse tracker racks up errors summing to 130 (say 70 misses + 50 false positives + 10 switches) against the same 100 ground-truth instances, MOTA = 1 − 130/100 = −0.30. A negative MOTA is not a bug — it simply means the tracker produced more errors than there were objects to find.

其中 FN = 漏检，FP = 误检，IDSW = 身份切换。MOTA 由检测主导——因为漏检和误检的数量通常远远超过切换，MOTA 主要评的是你找到那些目标了吗？，几乎不惩罚跟丢了谁是谁。算个具体例子：在一个 ΣGT = 100 个真值目标实例的序列上，假设跟踪器累积了 ΣFN = 20 次漏检、ΣFP = 15 次误检、ΣIDSW = 5 次切换。那么 MOTA = 1 − (20 + 15 + 5)/100 = 1 − 40/100 = 0.60。注意这些错误是不加权求和的，所以这个度量可以击穿零：如果一个更差的跟踪器针对同样的 100 个真值实例累积出合计 130 的错误（比如 70 次漏检 + 50 次误检 + 10 次切换），MOTA = 1 − 130/100 = −0.30。负的 MOTA 不是 bug——它只是意味着跟踪器产生的错误比要找的目标还多。

IDF1 = the ID F1 score: the F1 (harmonic mean of precision and recall) computed over identity-consistent matches across the whole sequence, after a global optimal matching of predicted trajectories to ground-truth trajectories. IDF1 directly grades how long you kept the right identity, so it is the metric to watch when identity preservation is the product requirement — e.g. counting unique shoppers in a crowded retail camera, where one ID switch corrupts the count more than a one-frame localization wobble.

IDF1 = ID F1 分数：在把预测轨迹与真值轨迹做一次全局最优匹配之后，在整个序列上对身份一致的匹配计算出的 F1（精确率与召回率的调和平均）。IDF1 直接评的是你把正确的身份保持了多久，因此当身份保持是产品需求时，它就是要盯的度量——例如在拥挤的零售摄像头里统计独立顾客数，此时一次身份切换对计数的破坏，比一帧的定位抖动要严重得多。

Where this points next

这将指向何处

Look again at the Re-ID term in section 3c. Appearance association works by embedding each crop into a vector and measuring the distance between embeddings — "same person ⟺ small distance, different person ⟺ large distance." We bolted that on as a heuristic, but it is in fact a deep, general capability: learning a representation where distance encodes similarity. The same machinery powers face verification, product search, duplicate detection, and the contrastive pre-training that underpins modern vision. Lesson 12 builds it properly — the contrastive, triplet, and InfoNCE losses that shape an embedding space, why L2-normalizing turns dot products into cosine similarity, hard-negative mining, and the approximate-nearest-neighbor indexes that make distance search scale to a billion items.

再看一眼第 3c 节里的 Re-ID 项。外观关联的做法是把每个裁剪块嵌入成一个向量，并度量嵌入之间的距离——"同一个人 ⟺ 距离小，不同的人 ⟺ 距离大"。我们把它当作一个启发式手段拼上去，但它其实是一种深刻而通用的能力：学习一种用距离编码相似性的表示。同一套机制支撑着人脸验证、商品检索、重复检测，以及支撑现代视觉的对比预训练。第 12 课会把它正经地构建出来——塑造嵌入空间的对比、三元组与 InfoNCE 损失，为什么 L2 归一化能把点积变成余弦相似度，难负样本挖掘，以及让距离检索扩展到十亿量级的近似最近邻（ANN）索引。

Interview prompts

面试题

• Why predict keypoint heatmaps instead of regressing coordinates? Give three concrete advantages.为什么要预测关键点热力图，而不是回归坐标？给出三个具体优势。
• How is a Gaussian heatmap target constructed, and what does σ trade off?高斯热力图目标是如何构造的，σ 在权衡什么？
• What is soft-argmax, why is it preferred over argmax, and work a small numeric example.什么是 soft-argmax，为什么它比 argmax 更受青睐，并算一个小的数值例子。
• Define PCK and OKS. Why must OKS scale by object size — what is the role of s² in the formula?定义 PCK 与 OKS。OKS 为什么必须按目标尺寸缩放——公式里 s² 起什么作用？
• Walk through one frame of tracking-by-detection: predict, associate, update, manage.走一遍检测跟踪的一帧：预测、关联、更新、管理。
• Explain the Kalman predict/update cycle. How does it let a track survive a missed detection?解释卡尔曼的预测/更新循环。它是如何让一条轨迹挺过一次漏检的？
• Why use the Hungarian algorithm rather than greedy matching for data association?数据关联为什么用匈牙利算法而不是贪心匹配？
• What causes identity switches, and how do Re-ID embeddings reduce them?是什么造成了身份切换，Re-ID 嵌入又如何减少它们？
• MOTA vs IDF1 — which would you optimize for a retail people-counter and why?MOTA 与 IDF1——对一个零售人数统计器你会优化哪个，为什么？