Self-supervised visual representation learning

自监督视觉表征学习

Lesson 13 built a data-hungry architecture — the ViT — and ended on its catch: it shines only with large-scale pretraining, but human-labeled data does not exist at internet scale. This lesson removes the bottleneck. We learn representations from unlabeled images by inventing tasks whose answers come from the data itself. The danger throughout is one specific failure — representation collapse — and the lesson is really the story of the tricks that prevent it.

第 13 课构建了一个数据饥渴的架构——ViT——并落在它的一个软肋上：它只有在大规模预训练下才出彩，但人工标注的数据在互联网规模上并不存在。本课去掉这个瓶颈。我们通过发明一些答案就来自数据本身的任务，从无标签图像中学习表征。贯穿全程的危险是一个特定的失败模式——表征坍缩——而这一课其实就是一整套防止它的技巧的故事。

What we are standing on

我们脚下的基础

We already have most of the machinery. Lesson 12 built embedding spaces and wrote down the contrastive objective — including InfoNCE, which pulls a positive pair together and pushes negatives apart through a softmax over similarities (the temperature τ in that softmax sharpens the distribution over cosine similarities; lesson 12 works it numerically, so we lean on it here without re-deriving the number). We will reuse that loss here, not re-derive it. Lesson 13 gave us the ViT backbone — and that is the concrete thing this lesson pretrains: both families below run on the ViT from lesson 13 with no labels (MAE masks patches and reconstructs them; DINO distills across two views), turning the data-hungry architecture lesson 13 ended on into one you can train on a billion unlabeled images. What is missing is only the supervision: lesson 12 still assumed someone told us which pairs were "same." Self-supervised learning (SSL) manufactures that signal for free, so the same embedding machinery can run on a billion unlabeled images.

我们已经拥有了大部分机器。第 12 课构建了嵌入空间并写下了对比目标——包括 InfoNCE，它通过对相似度做 softmax，把一对正样本拉近、把负样本推远（softmax 中的温度 τ 会锐化余弦相似度上的分布；第 12 课已把它数值化推演过，所以这里我们直接借用而不再重新推导这个数）。我们在这里会复用那个损失，而不重新推导它。第 13 课给了我们 ViT 骨干网络——而这正是本课要预训练的那个具体对象：下面两大家族都在第 13 课的 ViT 上、以无标签方式运行（MAE 遮蔽图块并重建它们；DINO 在两个视图之间做蒸馏），把第 13 课结尾那个数据饥渴的架构，变成一个你可以在十亿张无标签图像上训练的架构。缺的只是监督信号：第 12 课仍然假设有人告诉我们哪些对是"相同的"。自监督学习（SSL）免费地制造出这个信号，于是同一套嵌入机器就能在十亿张无标签图像上运行。

The plan计划

Four moves. (1) Pretext tasks: the general trick of hiding part of the data and predicting it. (2) The contrastive family (recalling 12's InfoNCE) and the trap it walks into — collapse — plus the exact tricks BYOL/SimSiam/DINO use to dodge it: stop-gradient, a predictor head, and an EMA target. (3) The masked family: MAE, which reconstructs hidden patches and needs no negatives at all. (4) How you measure a representation without fine-tuning it — linear probing and kNN — and where the field has landed: DINOv2 as the frozen-backbone default. 四步走。(1) 前置任务（pretext task）：隐藏一部分数据并预测它的这个通用技巧。(2) 对比家族（回顾第 12 课的 InfoNCE）以及它踩进的陷阱——坍缩——外加 BYOL/SimSiam/DINO 用来躲开它的那几个确切技巧：停止梯度、一个预测器头，以及一个 EMA 目标。(3) 掩码家族：MAE，它重建被隐藏的图块，完全不需要负样本。(4) 如何在不微调一个表征的情况下衡量它——线性探针与 kNN——以及这一领域最终落到了哪里：DINOv2 作为冻结骨干的默认选择。

1 · Pretext tasks: supervision from the data itself

1 · 前置任务：来自数据本身的监督

A pretext task is an artificial problem you can construct from an unlabeled image, where the "label" is derived automatically. Early examples: predict the rotation applied to an image (0/90/180/270°), solve a jigsaw of shuffled patches, or colorize a grayscale photo. To do any of these well the network must learn what objects look like, how parts fit together, what colors things tend to be — i.e. it must build a useful representation as a side effect. The downstream task (classification, detection) is then trained on top of those features. Modern SSL keeps the principle — label = a transformation of the input — but uses two much stronger families: contrastive and masked.

一个前置任务是你能从一张无标签图像构造出来的人造问题，其中的"标签"是自动导出的。早期的例子有：预测施加在图像上的旋转（0/90/180/270°）、拼好被打乱的图块拼图，或给一张灰度照片上色。要把这些中的任何一个做好，网络就必须学会物体长什么样、各部分如何拼合、东西通常是什么颜色——也就是说，它必须作为副产品构建出一个有用的表征。随后，下游任务（分类、检测）就在这些特征之上训练。现代 SSL 保留了这条原则——标签 = 输入的某种变换——但使用了两个强得多的家族：对比式与掩码式。

2 · The contrastive family — and the collapse it courts

2 · 对比家族——以及它招惹的坍缩

The contrastive recipe: take one image, apply two random augmentations (crop, color jitter, blur) to get two views, and declare them a positive pair — they should map to nearby embeddings because they are the same image. Every other image in the batch is a negative. This is exactly lesson 12's setup, and the loss is exactly its InfoNCE over cosine similarities with temperature τ (small τ sharpens the softmax so near-miss negatives are penalized harder — lesson 12 puts a number on this, and lesson 15 works it again for CLIP; SimCLR is InfoNCE with heavy augmentation and large batches for many in-batch negatives). The augmentations define the invariances: by forcing two crops of the same image together, you are telling the model "ignore crop, color, and blur — keep what is invariant to them." Choose augmentations that destroy what the downstream task cares about and you destroy the representation, which is the trap below.

对比式的配方：取一张图像，施加两次随机数据增强（裁剪、颜色抖动、模糊）得到两个视图，并把它们声明为一对正样本——因为它们是同一张图像，所以应当映射到相近的嵌入。批次中的其他每一张图像都是一个负样本。这正是第 12 课的设定，损失也正是它在余弦相似度上、带温度 τ 的 InfoNCE（小的 τ 会锐化 softmax，从而对差点混淆的负样本惩罚得更重——第 12 课给它标了一个数，第 15 课又为 CLIP 再算一遍；SimCLR 就是配上强数据增强和大批量、以获得大量批内负样本的 InfoNCE）。这些数据增强定义了不变性：通过强行把同一张图像的两个裁剪拉到一起，你是在告诉模型"忽略裁剪、颜色和模糊——保留对它们不变的东西"。若你选择的数据增强恰好破坏了下游任务所关心的东西，你就毁掉了这个表征，这正是下面这个陷阱。

The villain: representation collapse反派：表征坍缩

The objective "make two views of the same image close" has a trivial, catastrophic solution: map every image to the same constant vector. Then all positives are perfectly close — loss zero — and the representation carries no information at all. This is collapse. Contrastive methods avoid it because the negatives fight back: pushing other images away makes a constant solution impossible (you cannot be far from yourself). But negatives are expensive — they want large batches or memory banks. The interesting question, and the one BYOL/SimSiam/DINO answer, is: can you avoid collapse without negatives? "让同一张图像的两个视图靠近"这个目标有一个平凡而灾难性的解：把每一张图像都映射到同一个常数向量。这样所有正样本都完美地靠近——损失为零——而表征根本不携带任何信息。这就是坍缩。对比方法能避开它，是因为负样本在反抗：把其他图像推开，使得常数解变得不可能（你没法离你自己很远）。但负样本代价高昂——它们需要大批量或记忆库（memory bank）。有意思的问题，也是 BYOL/SimSiam/DINO 回答的问题是：你能不用负样本就避免坍缩吗？

The answer is yes, and the tricks are specific and nameable. The shared template is a student–teacher (or online–target) pair of networks fed the two views; the loss asks the student's output to match the teacher's. Left alone, that loss collapses — both networks could output a constant. Three ingredients break the symmetry:

答案是能，而且这些技巧是具体、可命名的。共同的模板是一对学生–教师（或在线–目标）网络，分别喂入两个视图；损失要求学生的输出去匹配教师的输出。放任不管，这个损失会坍缩——两个网络都可以输出一个常数。三种要素打破了这种对称：

Stop-gradient. The loss flows back only into the student; the teacher branch is treated as a fixed target (no gradient) — concretely, we detach the teacher's output from backprop, so the optimizer cannot nudge the teacher to meet the student halfway. This matters because if gradients flowed both ways, the cheapest joint move is for both branches to walk to the same constant together; freezing one side breaks that degenerate dynamic — the student is always chasing a target it cannot directly collapse toward. SimSiam showed this single trick, plus the next, is enough — no negatives, no momentum.停止梯度（stop-gradient）。损失只回流进学生；教师分支被当作一个固定的目标（无梯度）——具体来说，我们把教师的输出从反向传播中分离（detach）出去，于是优化器无法把教师推向与学生各让一步的中间点。这一点之所以重要，是因为如果梯度双向流动，最省事的联合动作就是两个分支一起走到同一个常数；冻结其中一侧就打破了这种退化的动力学——学生始终在追一个它无法直接朝其坍缩过去的目标。SimSiam 表明，仅这一个技巧，再加上下一个，就够了——不用负样本，也不用动量。
Predictor head. A small extra MLP sits on the student only, mapping its output toward the teacher's. Because only the student passes through this predictor, the two branches compute different functions of the same input, so the loss can be driven down by the predictor learning the relationship between views rather than by both encoders trivially emitting the same constant. That asymmetry is what makes stop-gradient work rather than trivially collapse; remove the predictor and SimSiam collapses.预测器头（predictor head）。一个额外的小 MLP 只装在学生这一侧，把它的输出映射向教师的输出。因为只有学生会经过这个预测器，两个分支对同一个输入计算的是不同的函数，于是损失可以靠预测器学习视图之间的关系来降低，而不是靠两个编码器平凡地吐出同一个常数来降低。正是这种不对称让停止梯度能起作用、而不是平凡地坍缩；去掉预测器，SimSiam 就会坍缩。
EMA target. In BYOL and DINO the teacher is not a separate trained network but an exponential moving average of the student's weights: θ_teacher ← m·θ_teacher + (1−m)·θ_student, with momentum m near 0.99–0.999. The teacher is a slow, stable version of the student — a moving target that lags behind, which both stabilizes training and prevents collapse. Intuition for the number: each step the teacher keeps m of itself and absorbs only (1−m) of the student, so its effective memory is about 1/(1−m) steps — at m = 0.999 that is ≈ 1000 steps, i.e. the teacher trails the student by roughly a thousand updates: slow enough to be a stable target, current enough to keep improving.EMA 目标。在 BYOL 和 DINO 中，教师不是一个单独训练的网络，而是学生权重的一个指数移动平均：θ_teacher ← m·θ_teacher + (1−m)·θ_student，动量 m 在 0.99–0.999 附近。教师是学生的一个缓慢、稳定的版本——一个滞后的移动目标，它既稳定了训练又防止了坍缩。对这个数的直觉：每一步教师保留自己的 m、只吸收学生的 (1−m)，所以它的有效记忆大约是 1/(1−m) 步——在 m = 0.999 时约为 1000 步，也就是教师大致落后学生约一千次更新：慢到足以成为一个稳定目标，又新到足以持续改进。

DINO adds two more anti-collapse devices on top of the EMA teacher: centering (subtract a running mean from the teacher's outputs so no single dimension dominates — guards against one component swallowing all the mass) and sharpening (a low teacher temperature → peakier targets, so the student is pulled toward a confident distribution rather than a flat one). The two pull in opposite directions and balance each other: centering spreads the output out, sharpening concentrates it, and together they keep the targets both diverse and decisive. Its self-distillation with no labels produces features so clean that attention maps segment objects for free.

DINO 在 EMA 教师之上又加了两个防坍缩装置：居中（centering）（从教师的输出中减去一个滑动均值，使得没有单一维度占主导——防止某个分量吞掉所有质量）和锐化（sharpening）（一个较低的教师温度 → 更尖峰的目标，于是学生被拉向一个自信的分布而非一个平坦的分布）。这两者朝相反方向拉扯、彼此平衡：居中把输出摊开，锐化把它收紧，二者合力让目标既多样又果断。它无标签的自蒸馏产出的特征如此干净，以至于注意力图能免费地把物体分割出来。

The two embedding spaces below make the difference concrete: collapse maps everything to one point; a healthy space spreads classes apart while keeping augmented views of one image together.

下面这两个嵌入空间把差别讲具体了：坍缩把一切都映射到一个点上；一个健康的空间则把各类别拉开，同时让同一张图像的多个增强视图待在一起。

2.5 · Interactive · representation collapse

2.5 · 交互 · 表征坍缩

The static figure above is the destination; this widget is the journey. Each colored pair is two augmented views of one image — the loss pulls each pair together. With every guard off, nothing stops the trivial solution, so every point slides to a single dot: collapse. Switch on stop-gradient + a predictor head (or just add negatives) and the pairs stay close while the classes stay apart — a healthy space. Step it and watch which way the geometry moves.

上面那张静态图是终点；这个小部件是旅程。每一对同色点是一张图像的两个增强视图——损失会把每一对拉到一起。当所有防护都关闭时，没有任何东西阻止那个平凡解，于是每个点都滑向同一个点：坍缩。打开停止梯度 + 一个预测器头（或者干脆加上负样本），各对就会保持靠近、而各类别保持分开——一个健康的空间。逐步运行它，看看几何结构朝哪个方向移动。

Representation collapse — feel why the anti-collapse tricks work

表征坍缩——体会为什么这些防坍缩技巧有效

Six points = three positive pairs (matching colors), each pair = two augmented views of one image. Step runs the student→teacher pull. All toggles off: the pull has nothing opposing it → every point converges to one dot (collapse, loss→0, but spread→0 so the features are useless). stop-grad + predictor (asymmetry breaks the degenerate dynamic) or negatives (push other images apart) keeps pairs together while spreading the three classes out. Caveat: here the predictor is illustrated as a fixed outward-nudge map, not a learned one — it stands in for the trained asymmetry a real predictor head provides.

六个点 = 三对正样本（颜色相同），每一对 = 一张图像的两个增强视图。Step 运行学生→教师的拉拽。所有开关都关时：这个拉拽没有任何对抗力量 → 每个点都收敛到同一个点（坍缩，损失→0，但分散度→0，所以特征毫无用处）。停止梯度 + 预测器（不对称打破了退化的动力学）或负样本（把其他图像推开）能让各对待在一起、同时把三个类别摊开。提醒：这里预测器被图示为一个固定的向外微推映射，而不是学出来的映射——它只是代替真实预测器头所提供的那种"训练出来的不对称"。

stop-gradient停止梯度 predictor head预测器头 negatives负样本

Mean pair dist

平均对内距离

—

Embedding spread

嵌入分散度

—

Steps

步数

Collapsed?

是否坍缩？

—

Show the core JS查看核心代码

// each point p has a position; pairs share an id. The "teacher" target for a point
// is its partner's position. The student is pulled toward it.
for (const p of points){
  const partner = points[p.partnerIdx];
  // stop-grad: teacher target is FIXED (don't also drag the partner toward us)
  const target = stopGrad ? frozenCopy(partner) : partner;
  // predictor head: a small asymmetric map applied to the student before matching
  const q = predictor ? predict(p) : p;
  pull(p, target, q);                       // attract: positives come together
  if (negatives){                           // push every OTHER image away
    for (const o of points) if (o.id !== p.id) repel(p, o);
  }
}
// WITHOUT (stop-grad+predictor) and WITHOUT negatives there is no opposing force
// -> the global minimum is "all points equal" -> collapse.

3 · The masked family: reconstruct what you hid (MAE)

3 · 掩码家族：重建你藏起来的东西（MAE）

A completely different pretext, borrowed from language's masked-language modeling. The Masked Autoencoder (MAE) takes a ViT, randomly masks a large fraction of patches (typically 75%), feeds only the visible 25% to the encoder, and asks a lightweight decoder to reconstruct the missing patches' pixels. The loss is mean-squared error on the masked patches only. To fill a hole that large the model must learn object shapes and scene context — you cannot inpaint 75% of an image by copying neighbors. Two practical wins: there are no negatives and no collapse risk (the target is the real pixels, an information-rich signal that a constant cannot match), and because the encoder only processes 25% of patches, pretraining is fast and memory-light. MAE excels at fine-tuning transfer; contrastive/DINO features tend to be stronger off-the-shelf (frozen). The iBOT line fuses both ideas — masked-image modeling inside a DINO-style self-distillation (distillation = training a student to reproduce a teacher's output distribution; self-distillation means the teacher is just a slow EMA copy of the student, so the network teaches itself) — and is the direct lineage of DINOv2.

一个完全不同的前置任务，借鉴自语言里的掩码语言建模。掩码自编码器（MAE）取一个 ViT，随机遮蔽掉很大一部分图块（通常 75%），只把可见的那 25% 喂给编码器，再让一个轻量的解码器重建缺失图块的像素。损失只在被遮蔽的图块上做均方误差。要填补这么大的一个洞，模型就必须学会物体形状和场景上下文——你没法靠复制邻居来修补一张图像的 75%。两个实际的收益：没有负样本、也没有坍缩风险（目标是真实像素，一个信息丰富、常数无法匹配的信号），并且因为编码器只处理 25% 的图块，预训练又快又省显存。MAE 在微调迁移上很出色；对比式/DINO 的特征则往往在开箱即用（冻结）时更强。iBOT 这一脉把两种思想融合起来——把掩码图像建模放进一个 DINO 式的自蒸馏里（蒸馏 = 训练一个学生去复现教师的输出分布；自蒸馏意味着教师只是学生的一个缓慢 EMA 副本，于是网络在自我教学）——并且是 DINOv2 的直接谱系。

Method方法	Family家族	Needs negatives?需要负样本？	Anti-collapse trick防坍缩技巧	Best at最擅长
SimCLR	Contrastive (InfoNCE)对比式（InfoNCE）	Yes (large batch)需要（大批量）	Negatives push apart负样本互相推开	Simple, strong baseline简单、强基线
SimSiam	Student–teacher学生–教师	No不需要	Stop-grad + predictor停止梯度 + 预测器	Minimal recipe, no momentum极简配方、无动量
BYOL	Student–teacher学生–教师	No不需要	Stop-grad + predictor + EMA target停止梯度 + 预测器 + EMA 目标	Stable, no negatives稳定、无负样本
DINO	Self-distillation自蒸馏	No不需要	EMA + centering + sharpeningEMA + 居中 + 锐化	Clean semantic / frozen features干净的语义 / 冻结特征
MAE	Masked recon掩码重建	No不需要	Real-pixel target (no collapse)真实像素目标（不坍缩）	Fine-tuning transfer; cheap pretrain微调迁移；廉价预训练

4 · Evaluating a representation without fine-tuning it

4 · 在不微调的情况下评估一个表征

SSL gives you a frozen backbone; you need to judge it before committing to a downstream task, and ideally without changing the weights (otherwise you are measuring the fine-tuning, not the representation). Two standard protocols:

SSL 给你一个冻结的骨干网络；你需要在投入一个下游任务之前就判断它，而且最好不改动权重（否则你衡量的是微调，而不是表征）。两个标准协议：

Linear probing. Freeze the backbone, train a single linear layer (a logistic-regression classifier) on top of its features using the downstream labels. Accuracy answers: are the classes linearly separable in this representation? A high linear-probe score means the features already organize the relevant structure — the most-cited SSL metric.线性探针（linear probing）。冻结骨干网络，用下游标签在它的特征之上训练单独一个线性层（一个逻辑回归分类器）。准确率回答的是：在这个表征里，各类别是否线性可分？高的线性探针分数意味着特征已经组织好了相关的结构——这是被引用最多的 SSL 指标。
kNN. Freeze the backbone, embed a labeled reference set, and classify each query by the majority label of its k nearest neighbors (cosine distance — lesson 12). It trains nothing, so it is the purest test of "is similarity in this space semantically meaningful?" — and it is what DINO famously reports.kNN。冻结骨干网络，把一个有标签的参考集嵌入，然后按每个查询的 k 个最近邻的多数标签给它分类（余弦距离——第 12 课）。它什么都不训练，所以它是对"这个空间里的相似度在语义上有意义吗？"最纯粹的检验——也正是 DINO 著名地报告的那个指标。
Fine-tuning (for contrast) unfreezes the whole backbone and trains end to end. It measures adaptability, not the frozen representation, and a strong fine-tuning number can hide weak frozen features — which is why linear-probe and kNN are reported alongside it.微调（作为对照）会解冻整个骨干网络并做端到端训练。它衡量的是适应性，而不是冻结的表征，而一个亮眼的微调数字可能掩盖了孱弱的冻结特征——这正是为什么线性探针和 kNN 会与它一并报告。

Worked example. Suppose a frozen backbone gives a linear-probe accuracy of 79% and a kNN (k=20) accuracy of 76% on ImageNet, while full fine-tuning reaches 84%. Read it as: the features are highly linearly separable (79% from one linear layer is excellent), similarity is meaningful (kNN within ~3 points of the probe), and fine-tuning adds only modest headroom — a hallmark of a strong general representation. If instead the linear probe were 60% but fine-tuning hit 84%, the representation would be adaptable but not directly usable frozen — a different, weaker kind of pretraining for the frozen-backbone use case.

算个具体例子。假设一个冻结骨干在 ImageNet 上给出 79% 的线性探针准确率、76% 的 kNN（k=20）准确率，而完整微调达到 84%。这样解读：特征高度线性可分（单个线性层就有 79% 已经很出色），相似度有意义（kNN 与探针相差约 3 个点以内），而微调只带来不多的提升空间——这是一个强大通用表征的标志。反过来，如果线性探针只有 60%、微调却达到 84%，那这个表征就是可适应、但冻结着无法直接使用——对冻结骨干这个使用场景而言，是一种不同的、更弱的预训练。

Trap: SSL quality is downstream-dependent陷阱：SSL 的好坏取决于下游

Augmentation choices bake in invariances. A representation tuned to ignore color and crop is great for object classification and terrible for color-critical or geometry-sensitive tasks (medical anomalies, tiny defects, exact localization). "State of the art on ImageNet linear probe" does not transfer automatically — always probe on a task close to yours. 数据增强的选择把不变性烙进了表征。一个被调成忽略颜色和裁剪的表征，对物体分类很棒，对颜色关键或几何敏感的任务（医学异常、微小缺陷、精确定位）却很糟。"在 ImageNet 线性探针上达到最先进水平"并不会自动迁移——永远要在一个贴近你自己任务的任务上做探针。

Where the field landed: DINOv2这一领域最终落到了哪里：DINOv2

DINOv2 is, as of this writing, the default frozen vision backbone: a large ViT trained at scale on a heavily curated image set with the iBOT-style fusion of self-distillation (DINO lineage) and masked-image modeling, producing features so general that a linear probe or kNN on top is competitive across classification, segmentation, and depth — without fine-tuning. It also introduced register tokens: extra learnable tokens added to the sequence that absorb the high-norm "artifact" activations ViTs otherwise dump into random background patches, cleaning up the attention/feature maps. The practical takeaway for an interview: for most frozen-feature CV tasks today, "start from DINOv2" is the strong default answer. 截至撰写本文时，DINOv2 是默认的冻结视觉骨干：一个大型 ViT，在一个经过大量清洗筛选的图像集上大规模训练，采用 iBOT 式地融合自蒸馏（DINO 谱系）与掩码图像建模，产出的特征如此通用，以至于在其上做线性探针或 kNN 就能在分类、分割和深度估计上都有竞争力——无需微调。它还引入了寄存器 token（register tokens）：加进序列的额外可学习 token，用来吸收 ViT 原本会往随机背景图块里倾倒的那些高范数"伪影"激活，从而清理注意力/特征图。给面试的实用要点是：对如今大多数冻结特征的 CV 任务，"从 DINOv2 起步"是那个强有力的默认答案。

Where this points next

这将指向何处

Self-supervision let images supervise themselves — pretext tasks and augmentations manufacture the signal. But that signal is bounded by what augmentations and masks can express: it teaches invariance and structure, not meaning in human terms. The richest, and essentially free, label for an image is the text that already accompanies it across the web — alt-text, captions, surrounding words. Lesson 15 makes text the supervisor: CLIP trains an image and a text encoder with a symmetric version of the very InfoNCE loss we leaned on here, over an image–text similarity matrix, unlocking zero-shot classification and, ultimately, the vision-encoder→LLM models that define multimodal AI today.

自监督让图像监督自己——前置任务和数据增强制造出信号。但那个信号被数据增强和掩码所能表达的东西所限：它教会的是不变性和结构，而不是人类意义上的含义。对一张图像来说，最丰富、且基本免费的标签，是遍布网络、早已伴随它的文本——alt 文本、图注、周围的文字。第 15 课让文本成为监督者：CLIP 用我们这里所倚重的那个 InfoNCE 损失的对称版本，在一个图像–文本相似度矩阵上训练一个图像编码器和一个文本编码器，解锁了零样本分类，并最终解锁了定义了今日多模态 AI 的那些视觉编码器→LLM 模型。

Takeaway要点

Self-supervised learning trains on unlabeled images by inventing a pretext task whose label is a transformation of the input. The contrastive family (SimCLR) reuses lesson 12's InfoNCE — positives are two augmented views, negatives are other images, and the negatives are what prevent collapse (everything mapping to one vector). To drop the expensive negatives, BYOL/SimSiam/DINO use a student–teacher setup with three named tricks — stop-gradient, a predictor head on the student, and an EMA target teacher — plus DINO's centering/sharpening. The masked family (MAE) hides 75% of patches and reconstructs them, needing no negatives. Judge frozen features with linear probing and kNN (not just fine-tuning). The current default frozen backbone is DINOv2 (iBOT lineage, plus register tokens). 自监督学习通过发明一个前置任务——其标签是输入的某种变换——在无标签图像上训练。对比家族（SimCLR）复用了第 12 课的 InfoNCE——正样本是两个增强视图，负样本是其他图像，而正是这些负样本防止了坍缩（一切都映射到同一个向量）。为了甩掉代价高昂的负样本，BYOL/SimSiam/DINO 使用一个学生–教师设定，配上三个有名字的技巧——停止梯度、学生上的一个预测器头，以及一个 EMA 目标教师——再加上 DINO 的居中/锐化。掩码家族（MAE）遮蔽 75% 的图块并重建它们，无需负样本。用线性探针和 kNN（而不只是微调）来判断冻结特征。当前默认的冻结骨干是 DINOv2（iBOT 谱系，外加寄存器 token）。

Interview prompts

面试题

Why can contrastive learning work with no manual labels — and what is the positive pair?为什么对比学习能在没有人工标签的情况下奏效——正样本对又是什么？
What is representation collapse, and why don't contrastive methods with negatives suffer it?什么是表征坍缩，为什么带负样本的对比方法不会遭受它？
How do BYOL/SimSiam/DINO avoid collapse without negatives? Name stop-gradient, the predictor head, and the EMA target.BYOL/SimSiam/DINO 如何在不用负样本的情况下避免坍缩？说出停止梯度、预测器头和 EMA 目标。
How do augmentations define invariance, and when does that backfire?数据增强如何定义不变性，它又在什么时候适得其反？
Explain MAE: what is masked, what is the target, and why is there no collapse risk?解释 MAE：什么被遮蔽、目标是什么，以及为什么没有坍缩风险？
Linear probe vs kNN vs fine-tuning — what does each measure, and why report all three?线性探针 vs kNN vs 微调——各自衡量什么，为什么要把三个都报告出来？
Why is DINOv2 the default frozen backbone, and what do register tokens fix?为什么 DINOv2 是默认的冻结骨干，寄存器 token 又修复了什么？