all lessons/synthetic_vision/index7 lessons · ~10h study

Synthetic Vision Data, from first principles

A linear path from a prebuilt 3D scene and engine to useful evidence for vision and world models. Specify the deployment distribution, generate structured trajectories, simulate measurements, preserve exact truth, implement the complete image pipeline in Blender, cross the reality gap, and validate decisions—not screenshots.

contractreal task + failures
simulatestate, action, sensor
learnsynthetic + real
auditreal risk → new cases
Thesis
Synthetic data is distribution and measurement engineering, not image generation. Photorealism modifies only part of the observation model. It cannot repair missing behaviors, impossible physics, mislabeled coordinate frames, data leakage, or absent counterfactual actions.

The fundamental problem: a model needs evidence, not merely generated content

Why is building synthetic vision data difficult when a 3D engine can already render an unlimited number of images? Because the scarce object is not an image. It is evidence that identifies a useful rule for deployment. A renderer answers a forward question: given a particular scene, camera, light configuration, and rendering algorithm, which pixels should appear? Training asks a different inverse question: from finite measurements, which regularities should a model retain so that it behaves correctly in worlds it has not seen?

Those questions are not equivalent. Infinitely many different simulated datasets can be rendered from the same engine. Some teach the intended relation; others make a shortcut easier. Suppose every dusk pedestrian in the synthetic set wears a bright red coat, while every empty road is rendered in blue-gray tones. A detector can minimize its training loss by treating red pixels as the danger signal. The scene may be physically valid, every mask may be exact, and every image may look convincing. Yet the learned rule is wrong because the generator made clothing color more predictive than pedestrian geometry and motion.

From the model's perspective
The model does not see “road friction,” “causal intervention,” “pedestrian identity,” or “authoritative engine truth.” It receives arrays of numbers and an objective. Gradient descent rewards whichever available regularity reduces that objective most cheaply. A design intention that is not expressed through support, variation, labels, timing, and evaluation is invisible to the learner.

Write the deployment loop in its smallest useful form. A real world has hidden state s. Sensors turn that state into an observation history h. A learned system maps the history to a prediction or action, and the environment assigns a cost:

s ∼ preal(s),   h ∼ Oreal(h|s),   a=fφ(h),   risk(φ)=E[C(s,a)]

A synthetic pipeline replaces the first two distributions with configurable approximations pθ and Oθ, then supplies labels through a query Lθ. More samples reduce sampling noise inside that chosen process. They do not prove that the process matches the real states, measurements, action effects, or costs that matter. If the generator omits partially occluded pedestrians, ten billion clean synthetic frames make the model increasingly certain about a world in which that failure mode does not exist.

This is the central distinction between variance and structural error. More samples can reduce variance: the learner estimates the synthetic relationship more precisely. They cannot automatically remove structural error: the synthetic relationship itself may be wrong. Photorealism can reduce one part of observation mismatch, but it cannot create missing action support, correct an invalid collision model, align a rolling-shutter label in time, or tell us whether a planner benefits from the result.

Three impossibilities force the architecture

  1. No support, no learning. If a deployment-relevant state–action event never occurs in the generated process, its consequence cannot be estimated from that data. Architecture scale cannot recover evidence that is absent.
  2. One observation, several worlds. The same image can come from a stationary pedestrian and an approaching pedestrian, or from a dark surface under bright light and a bright surface under dim light. When those hidden causes imply different futures, a single deterministic label or current frame is insufficient. The system needs history, multiple sensors, interventions, or explicit uncertainty.
  3. Observation is not intervention. Logged drivers brake mostly in dangerous situations. A passive dataset may therefore show more collisions after braking than after coasting. It cannot identify the effect of braking without matched context, exploration, structural assumptions, or controlled intervention.

These are not defects of a particular engine. They are information limits. The seven lessons remove them in a defensible order. Scenario programs create state and action support. Sensor models declare which hidden distinctions survive measurement. Label contracts say exactly which event is being supervised. The Blender lab then turns those contracts into an executable image-data pipeline, so implementation details remain consequences of the reasoning rather than substitutes for it. Calibration and real anchors constrain process mismatch. Checkpointed alternative futures provide interventional evidence. Closed-loop tests ask whether the remaining approximation is good enough for the decision.

Why not solve everything with one end-to-end generator?

An end-to-end learned generator can be extremely useful, but it cannot serve as its own authority. If its video says the shuttle stopped safely, we still need an independent definition of position, collision, action timing, and uncertainty to know whether that statement is true. If the same model generates the sample, invents the label, and grades the result, a shared error can pass every internal check.

The remedy is not to reject learned generation. It is to assign authority according to what each component can establish:

This separation is a debugging theorem as much as a software architecture. When the real miss rate rises, we need to ask whether the relevant world was absent, the dynamics were wrong, the sensor erased the cue, the label described a different event, the learner used a shortcut, or the planner exploited an optimistic error. A monolithic quality score entangles those hypotheses. Explicit boundaries make each one falsifiable.

The minimal promise
A synthetic pipeline does not need to reproduce reality in every detail. It must preserve the distinctions that change the downstream decision, vary nuisance factors so the model cannot rely on them, expose uncertainty where observations remain ambiguous, and demonstrate improvement on evidence the generator did not create.

One running problem, seven linear stages

The lessons reuse one case so that each abstraction has a concrete job. An autonomous shuttle approaches a parked van at dusk. A pedestrian may emerge from behind it. The shuttle observes the scene through a front rolling-shutter camera, scanning LiDAR, and radar, then must decide whether and when to brake. The expensive failure is not “an image looks unrealistic.” It is a late or unjustified braking decision caused by missing state, misleading measurements, bad supervision, or an incorrect prediction of what braking changes.

LayerQuestion the case forcesArtifact handed to the next lesson
01 · DistributionWhich real failures matter, and which state–action situations must have support?A task contract and factorized data-generating process.
02 · WorldHow do the van, pedestrian, road, intent, physics, and brake/coast alternatives become controllable state?Valid checkpointed scenario trajectories.
03 · MeasurementWhat does each sensor actually integrate or scan while the actors and shuttle move?Timed camera, LiDAR, and radar observations.
04 · SupervisionWhich state and coordinate frame define depth, flow, boxes, visibility, pose, and action labels?A replayable, synchronized, leakage-safe dataset.
05 · Build labHow do those contracts become a repeatable Blender pipeline for an image detector or segmenter?Headless RGB, instance-mask, COCO, QA, lineage, training, and feedback artifacts.
06 · TransferWhich differences from reality should be calibrated, randomized, reconstructed, or learned?A hybrid synthetic–real training and validation loop.
07 · DecisionDoes the learned model predict brake versus coast consequences well enough to reduce real closed-loop risk?An evaluation ladder and deployment acceptance decision.
How to study
Do not memorize simulator products. At every stage, identify the input contract, the transformation being applied, the information that can be lost, the invariant that can test it, and the real-world decision that ultimately judges it. Each lesson now includes derivations, a worked continuation of the shuttle case, implementation-oriented pseudocode or schemas, failure diagnosis, and exercises.

The object being engineered

An engine is a controllable data-generating process. A trajectory follows an initial-state distribution, a behavior policy, dynamics, and a sensor model; labels are deterministic queries of authoritative engine state:

s0 ∼ pθ(s0),   at ∼ πθ(at|ht),   st+1 ∼ Tθ(st+1|st,at),   ot ∼ Oθ(ot|st),   yt=L(st,ot)

The parameters θ include scene layout, assets, actor intent, physics, lighting, optics, timing, noise, and post-processing. The goal is not to maximize variety in every parameter. It is to choose a conditional joint distribution whose support contains the deployment cases that change the model’s decisions.

Content gap
The wrong worlds exist
Objects, layouts, affordances, interactions, or rare failures are absent.
Action gap
The wrong futures are observed
A passive policy never attempts the interventions a deployed agent considers.
Dynamics gap
Consequences are wrong
Contact, behavior, latency, deformation, or material response differs from reality.
Sensor gap
Measurements are wrong
Optics, exposure, scan timing, noise, ISP, dropout, or weather differs.

Why the sequence is linear

  1. Contract before content. Until sensors, outputs, decisions, failure costs, and real evaluation slices are fixed, “more realistic data” has no target.
  2. World before measurement. A sensor can only measure the simulated state. Scene structure and behavior determine whether the needed situations ever occur.
  3. Measurement before labels. Depth, flow, exposure, and timestamps must refer to a declared measurement event; otherwise exact engine truth becomes precisely wrong supervision.
  4. Truth before implementation scale. Reproducibility, invariants, and leakage-safe splits are prerequisites for a trustworthy exporter.
  5. One verified sample before a render farm. Prove that scene state, RGB, labels, metadata, and replay agree end to end; scaling an unverified exporter only multiplies hidden error.
  6. Implementation before transfer claims. Until the actual training bytes and labels pass end-to-end checks, a sim-to-real comparison cannot distinguish renderer mismatch from pipeline corruption.
  7. Reality before acceptance. Synthetic metrics select generators; frozen real-world and closed-loop metrics decide whether the system improved.

The lessons

01
The data-generating process and task contract
Factor a simulator into initial state, behavior, transition, observation, and label processes; work backward from deployment loss and support.
02
Simulation-ready scenes and structured scenarios
Turn art assets into physical, semantic state; sample conditional scene programs; branch matched interventions from identical checkpoints.
03
Sensor measurement models
Model cameras from radiance through RAW and ISP, then derive timing and failure modes for rolling shutter, LiDAR, radar, and event cameras.
04
Labels, synchronization, provenance, and QA
Define coordinate and visibility contracts; export dense synchronized truth; guarantee replay, leakage-safe splits, and geometric invariants.
05
Build a Blender pipeline for an image model
Implement a reproducible headless pipeline from task contract and scene template through conditional sampling, RGB and instance passes, COCO export, QA, lineage splits, training, and held-out real feedback.
06
Sim-to-real and hybrid training
Use structured randomization, real-to-sim calibration, learned appearance residuals with label-drift rejection, and deliberate synthetic–real mixing.
07
World-model trajectories and closed-loop evaluation
Generate action-conditioned, matched counterfactual trajectories; test state, causality, uncertainty, rollouts, and real decision utility.

A compact operating rule

Start with one expensive deployment failure. Reconstruct or author the smallest family of scenes that can produce it. Randomize the uncertain causes, preserve the known structure, replay matched action branches, export synchronized labels, train with a small trusted real anchor, and accept the new generator only if it reduces held-out real risk. Then repeat. This turns synthetic data from a one-time content dump into a measured feedback controller.

Primary-source map

The series links sources at the claim they support. Key systems include the Blender Python API and headless command-line interface, Infinigen (CVPR 2023), Kubric (CVPR 2022), BlenderProc (IJRR 2023; first released 2019), Structured Domain Randomization (ICRA 2019), Isaac Sim + Cosmos workflows (NVIDIA documentation, accessed 2026-07-18), Cosmos-Transfer1 (2025), UniSim (2023), NeuRAD (2023 / CVPR 2024), V-JEPA 2 (2025), Genie 3 (2025), and WorldSimBench (2024).