Effective hours: duplicates, quality and collapse

Lesson 21 spent a budget by reading curves of success against hours bought, every hour a layout the policy had not seen, and a real corpus counts recorded episodes. On the Bench 80 layouts recorded 32 times each are 2,560 episodes and 6.61 hours, and the policy trained on them reaches what 80 layouts reach once, 84.4 %. This lesson defines the size of a dataset by what the policy reaches, in the unit of lesson 16, and tests three discounts against it: counting layouts and not episodes; a batch recorded with an offset that passes every check made inside it and is worth less than nothing; and a generator trained on its own samples, which forgets its rarest layouts first. It does not say what it costs to move the hours that survive.

1 · The same layouts, recorded again

The layouts curves of lessons 16 to 21 count different layouts: the ledger's own curve counts own layouts, each demonstrated once, and a source adds attempted layouts, each a new one. A corpus need not be built that way. Take the first 80 layouts of your own file and record each m times; the expert runs the layout again from a start about a centimetre off, so every recording is a new file of the same situation. The policy is the lookup of lesson 7: it copies the stored layout nearest to a new one, here with every repeat of that layout in its table, and success is the share of 1000 new layouts it completes. A recorded hour is 387.1 demonstrations of 9.3 s.

Thirty-two times the hours buy the same policy: 84.3 to 84.7 % for every m, which is well inside the sampling error of the 1000 test layouts. A filter that drops identical files has nothing to drop, because the start offset makes all 2,560 files different. At real scale duplicates are measured as well: near-duplicate filtering removes 3.04 % of the examples of C4 and 13.63 % of those of RealNews, and one 61-word sequence occurs 61,036 times in the C4 training set (Lee et al., 2022). The baton's corpus has the same ratio on a larger scale: 100,000 episodes of 9.3 s are 258 hours and 800 situations are 2.07 hours, a factor of 125.

At lesson 17's assumed price of $123.6 for a motion hour of your own robot, the 6.61 hours cost $817 and bought what 0.207 hours, $25.5, would have bought: the hours that did any work cost $3,956 each. The plan of lesson 21 reads its curves at the hours it has bought, and for this corpus the number to read at is 0.207. What is the size of a dataset?

2 · A size defined by what the policy reaches

Four constraints. The size has the unit of lesson 16, an own layout demonstrated once; it is measured on the policy and not asserted; it is tied to the task, the policy and the test layouts; and a planner needs it before the money is spent, so it must be predictable from the corpus. The first three give a definition. Let S be the success of the policy trained on the dataset on the 1000 test layouts, and N_eq(S) the number of own layouts that reach S on the ledger's own curve, read off by the inversion of lesson 16 (its interval is the Wilson interval of S pushed through it):

n_eff = N_eq(S)

For the corpus of §1 it is 80 layouts (73 to 86). The fourth constraint asks for a count that predicts it, and five corpora give the candidates something to disagree about. In a field corpus the episodes are recorded where the world presents them: layout c has probability p_c proportional to rank^−a, the ranks assigned at random over the 256 layouts of your file (a = 0: every layout alike; a = 1: a long tail; a = 2: a few layouts take most episodes), and R episodes are drawn. Write R for the episodes recorded, k_c for those of layout c, D for the number of layouts with k_c > 0, and n_K = (Σ k_c)² / Σ k_c² for Kish's formula applied to the counts. One draw of each corpus:

Recorded episodes overstate the size by a factor from 1.2, where every layout is alike, to 107.7, where a few dominate; files are no better, since every recording is a different file. Kish's formula on the counts falls below the measured size in every row but the first, by a factor of 5 at a = 1 and 24 to 39 at a = 2. The number of different layouts comes within a fifth: over eight draws of each corpus the measured size is 0.80 to 0.96 of D at a = 1 and R = 320 and 0.81 to 1.14 at a = 2 and R = 2,560 (0.81 to 1.38 at R = 10,240, where success passes 95 % and the own curve flattens). The intervals of the table carry the sampling error of the 1000 test layouts, and the spread over draws the luck of which layouts were drawn.

The reason is in the policy. For each test layout it copies one demonstration of the nearest stored layout, so the number of repeats behind a layout never enters (§1): success depends on which layouts are present and where they lie, and a layout recorded once and one recorded a hundred times are the same entry in the table. The count that matches is the number of entries. Exact repeats are found by a hash of the layout; layouts rebuilt by hand to within a few millimetres are clusters of layouts closer than the distance within which a stored layout serves a new one, which this lesson does not run.

Kish (1965) defined (Σ w)² / Σ w² for the precision of a weighted mean: n for equal weights, never more, and a measure of the loss from unequal weights, not of the number of different units. Applied to episodes per layout it is this lesson's construct, with one exact reading: Σ (k_c / R)² is the probability that two episodes drawn at random are of the same layout, so n_K is its inverse (the checkpoint works it by hand). It is 80 for the first corpus, which cannot tell it from D.

3 · A duplicate is also a weight

The lookup cannot be moved by counts, by construction; a learner trained on a mean loss can. Over the recorded episodes i, the loss Σ_i K_i (y − y_i)², with K_i a kernel of the distance between the new layout and the layout of episode i, is minimised by the weighted average y = Σ K_i y_i / Σ K_i, and a layout recorded k_c times enters it k_c times: a duplicate is up-weighted in the loss and not merely wasted. Run that learner on the Bench: for each test layout take the 8 nearest stored layouts, weight each by k_c exp(−d² / 2h²) with h = 1.5 cm (d is how far the farthest post moves between the two layouts, as in lesson 7), average their hand paths and follow the average. The same 80 layouts and 2,560 episodes, with the counts drawn at exponent a (ranges over eight draws):

With equal counts the average interpolates and beats the lookup, 93.4 % to 84.4 %. At a = 1 every layout is still present and the lookup is where it was, while the average loses 6.8 points: the popular layouts pull the average at every neighbouring test layout toward their paths. At a = 2, 30 layouts are never recorded, the lookup loses what they covered, and the average loses 34.1 points. Kish's formula orders the three corpora as the averaging learner does, 78.3 > 14.8 > 2.5 against 93.4 > 86.6 > 59.3 %, and sizes neither learner: it names no number of layouts that either is worth. For a mean-loss learner a duplicate costs twice, as hours that taught nothing and as weight taken from the rare layouts; for the lookup only the first.

4 · A batch that passes its own checks

Quality enters as a weight, and the weight can be negative. Let a rig have recorded a share of the layouts with a hand tracker offset by 1.5 cm sideways: every recorded hand target of its layouts is shifted, the layouts it states are right, and every recording succeeded in the operator's frame. In the corpus of §1 take 25 % of the layouts, 20 of 80. Nothing inside the batch shows the offset. None of its recordings touches a post of its layout (the closest comes within 5.1 cm of one, against a contact distance of 5 cm, and 6.0 cm unshifted; at 2 cm 34 of the 80 would touch), its hours look like any batch's, and the spread of its clearance to the posts (the sideways distance from the recorded path to a post where the hand passes it), 0.8 mm, is the rest's 0.7 mm: an offset that every recording shares cannot show in how much recordings differ.

The policy shows it. Keeping the batch gives 76.4 % and a size of 59 layouts (55 to 64), where the clean corpus gave 84.4 % and 80. Dropping the 20 layouts gives 80.8 % and 69, so each layout of the batch is worth (59.5 − 69.3) / 20 = −0.49 own layouts (−0.53 to −0.01 over the eight choices below). A lookup trusts its labels: 15.7 % of the test layouts have one of the rig's layouts as their nearest stored layout, and the arm then follows a path displaced by 1.5 cm; it completes 40.8 % of them, against 68.8 % when the rig's layouts are dropped and the next nearest layout serves (the dilution of lesson 18, by quality instead of by source). Over eight choices of which layouts the rig recorded, keeping gives 73.1 to 77.8 % and dropping 76.5 to 79.1 %, so dropping wins every time; with 10 % of the layouts on the rig the figures are 79.8 and 82.7 %.

What does see it is held-out consistency: a comparison with something the tracker did not record. The posts a layout states are known without the hand, and the expert passes them at a clearance that does not depend on the layout (at least 7.0 cm sideways). Compare the clearance of the rig's layouts with the rest's: the difference of means is 1.49 cm, z = 76 standard errors (Welch), so the batch is flagged and its offset estimated, 1.49 cm against the 1.5 cm applied. Dropping the flagged batch gives 80.8 %; subtracting the estimated offset gives 84.4 %, the clean figure. At these batch sizes the check flags offsets down to 0.59 mm (z = 3); the Bench's expert differs by under a millimetre between recordings, and a noisier demonstrator would raise the threshold in proportion to that spread, over the square root of the batch size.

5 · A generator trained on its own samples

A model that generates episodes can feed the next corpus. Take the simplest generator: it memorises what it was trained on and emits R episodes with the shares of the previous corpus, p_c = k_c / R; the policy trains on the new corpus, and the next generation starts from it. A layout with k of the R episodes is missing from the next corpus with probability (1 − k / R)^R, about e^−k: a layout recorded once survives a generation with probability 63.3 % (a simulation gives 63.6), and the layouts left after one generation number Σ_c [1 − (1 − k_c / R)^R] in expectation, 79.2 for the corpus at a = 1 and R = 320 that has 105 layouts (79.6 over 300 generations drawn). The diversity 1 − Σ p_c² is multiplied by 1 − 1 / R per generation in expectation, 0.9876 over eight generations at R = 640: the measure that the head dominates hardly moves while the layouts with small counts, the tail, go extinct. Median of eight draws, a = 1 and R = 640:

The tail goes first. After eight generations 60 % of the layouts are gone and success on the uniform test has fallen 21.1 points (71.7 to 78.1 % over the draws), while 74 % of the original episodes still have their layout present and n_K has moved from 22 to 18: a check that averages over recorded episodes sees less than half of the loss, because the layouts that go are the rare ones. Keeping a random tenth of the original corpus in every generation leaves 86 layouts, 57 % of the original, and 86.1 %: the loss is slowed and not stopped, since the original holds only the layouts it recorded. Shumailov et al. (2024) show for language models, variational autoencoders and Gaussian mixtures that training on a model's own output makes the tails of the original distribution disappear, and in their OPT-125m runs a random tenth of the original data kept in each generation gave only minor degradation; they tested no robot data.

The widget

ES.score = function (off) {
...
  for (t = 0; t < NT; t++) {
    bd = Infinity; bc = -1;
    for (c = 0; c < NS; c++) { if (off[c] !== off[c]) continue; d = DM[t * NS + c]; if (d < bd) { bd = d; bc = c; } }
    pick[t] = bc; if (bc >= 0) { okv[t] = outcome(t, bc, off[bc]); ok += okv[t]; }
  }
...
ES.generation = function (cnt, cnt0, R, fresh, seed) {
  var nf = Math.round(fresh * R), a = ES.draw(ES.share(cnt, R), R - nf, seed), b, i;
  if (nf > 0) { b = ES.draw(ES.share(cnt0, R), nf, seed + 500); for (i = 0; i < NS; i++) a[i] += b[i]; }
  return a;
};
ES.neq = function (s) { return LG.ownEquivalent(LG.ownCurve('hand'), s); };

What to try. Leave the defaults, 80 scripted layouts and 2,560 episodes: 6.61 hours recorded, 80 layouts, Kish 80.0, a measured size of 80 layouts (73 to 86), success 84.4 %, so 32 episodes per effective layout and $3,956 per effective hour. Move the slider to 80 and to 10,240 episodes: success stays 84.4 and 84.4 % while recorded hours go to 26.5 and the ratio to 128. Take the field corpus with a = 2 at 10,240 episodes: 107 layouts, Kish 2.5, a size of 95 (89 to 109), 26.5 hours recorded for 0.25 effective. Put 25 % of the scripted layouts on the rig: 76.4 % kept (size 59), 80.8 % dropped, 84.4 % corrected, at z = 76. On the field corpus with a = 1 and 640 episodes (150 layouts, 97.1 %) raise the generations to 8: 60 layouts, 75.5 %, 76 % of the original episodes still represented; keep a tenth of the original and it is 86 layouts and 86.1 %. On the scripted corpus the same eight generations keep 79 of the 80 layouts and 84.3 %: 32 episodes per layout make extinction slow, not impossible. These are single seeded draws; the table of §5 gives medians.

6 · The ledger line, and what finding it costs

The ledger has a price per motion hour (lesson 17), a rate per hour of a source (lesson 16) and a curve of value against hours (lesson 18). This lesson puts a factor in front of all three, the effective fraction e = n_eff / R. The scripted corpus has e = 1 / 32 (0.207 of 6.61 hours); the field corpus of §2 with a = 2 and 10,240 episodes has 1 / 107.7, and over eight draws 1 / 65 to 1 / 112. Every price of the ledger is paid per recorded hour and returns per effective one, so an effective hour costs the price over e: $3,956 against $123.6 for the scripted corpus. A plan that reads its curves at effective hours buys correctly; one that reads them at recorded hours buys 32 to 108 times too much.

The effective hours are fewer than the recorded ones, and finding them is a pass over the recorded ones. A duplicate is found by comparing an episode with others, a batch's offset by comparing the batch with posts recorded elsewhere, a generator's loss by comparing with the original corpus, and each of the three reads every recorded episode. The 80 layouts that survive are cheap to keep; the 2,560 episodes around them were stored, decoded and compared to find them, and a model trained on the whole corpus decodes every frame of every episode in every epoch.

Common mistakes / failure modes

Checkpoint exercise

Where this points next

Counting effective hours shrinks the corpus. The 2,560 episodes of §1 hold 80 layouts, a factor of 32 and, at the baton's scale, 125; a field corpus of 10,240 episodes whose popularity falls as the square of the rank is worth 95 layouts, 65 to 112 episodes recorded per effective layout over eight draws. The layouts that survive are cheap to keep, but finding them reads every recorded episode: a duplicate is found by comparing episodes, a bad batch by comparing it with another, a generator's lost tail by comparing with the original, and a policy that trains on video decodes the video it reads, in all 2,560 episodes, every epoch. The hours that survive still have to reach the accelerators, and if the processors wait for the data, the data budget is not the only bill. What does it cost to move the effective hours, and where does the machine stall?

Interview prompts

• A hundred thousand recorded episodes hold eight hundred situations. How do you define and measure the size of the dataset? (§2 — the own layouts that give the same success on the ledger's own curve: 80 for the 2,560 episodes of 80 layouts.)
• Why is a lookup policy immune to how often a layout was recorded and a mean-loss learner not? (§3 — the lookup copies one entry; the average gives a layout recorded k times weight k, and falls from 93.4 to 59.3 %.)
• What does Kish's formula measure, and why is it not the size here? (§2 — the loss of precision from unequal weights, the inverse collision probability on counts: 2.4 against 58 measured.)
• A batch recorded with a tracker offset passes every check inside it. What finds it, and what does keeping it cost? (§4 — comparison with posts recorded elsewhere, z = 76; keeping it costs 8.0 points.)
• Why does a generator trained on its own samples lose the tail before the head, and why does your validation not notice? (§5 — a layout with k episodes goes missing with probability about e^−k; 74 % of the original episodes are still represented when 60 % of the layouts are gone.)
• Why must you read all the recorded hours to find the effective ones? (§6 — duplicates, offsets and lost tails are found by comparison, and the comparison reads every episode.)

Companion reads: Lesson 7 · How much data: the coverage law, Lesson 8 · Other bodies and World Models · 27 The mixture is the model.