Actions and causality

Watching what agents usually do reveals correlations. Predicting what your action will cause is a different question — one that needs interventions, coverage, and a model that never confuses the data-collection policy with the world.

1 · The braking paradox

Train a model on dashcam video and it learns, correctly, that hard braking and collisions go together. So does braking cause collisions? Obviously not — drivers brake because a crash is already looming. The correlation is real and completely misleading, and a planner that trusts it will learn to avoid the brake pedal. That gap — between “what usually follows action a in the log” and “what my action a causes” — is the whole lesson.

Put it in the planner’s terms. At time t the agent has history H_t = (o_≤t, a_<t), a belief b_t, and candidate actions. Its real question isn’t “what came after a before?” It is:

We’ll write random variables in uppercase (S,O,A) and specific realized or chosen values in lowercase (s,o,a), so we can cleanly separate “the logged A_t happened to equal a” from “the agent forced A_t=a.” Those two are not the same operation, and a model can nail held-out trajectories by learning the old controller’s habits, then fail the moment a new planner picks an unusual action.

From first principles, keep three mechanisms separate:

1. The environment evolves by state and action: S_t+1 = f(S_t, A_t, ε_t), with exogenous process noise ε_t — influences from outside our model.
2. The sensor emits an observation: O_t+1 = g(S_t+1, η_t+1), with sensor noise η.
3. The behavior policy produced the log: A_t = π_b(H_t, ξ_t), with policy randomness ξ.

The behavior policy is a data-collection habit, not a law of physics. A causal world model must learn the first two mechanisms without baking in the third — otherwise changing the policy will look like changing “the world.”

2 · Where the false signal comes from

Trace the braking paradox to its root. A hidden road condition U affects both the driver’s braking A and the stopping success Y: ice makes drivers brake hard, and ice also makes stopping harder. The causal graph has U → A, U → Y, and the genuine action edge A → Y. The path A ← U → Y is a backdoor path — it manufactures association between action and outcome that the action never actually transmits.

That is confounding: a common cause of both the action and the outcome. The confounder needn’t be literally invisible; it’s enough that the state fed to the model leaves it out or blurs its relevant values. In a partially observed world, road friction, another person’s intent, battery health, or object mass can all quietly play U.

One honest caveat. If S_t were the complete causal state — Markov, fully observed, containing every common cause of A_t and S_t+1 — then the ordinary conditional transition p(s_t+1 | s_t, a_t) would equal the interventional one p(s_t+1 | s_t, do(a_t)). So it isn’t that action-conditioned models are always non-causal. The problem is that learned latents and finite histories may not be causally sufficient — and low prediction loss never proves that they are.

3 · Conditioning and intervening are exact — and different

Conditioning filters the data to cases where something was observed. p(Y | A=a) asks: among trajectories where the behavior policy happened to choose a, how often did each outcome occur? Seeing the action is evidence about its causes — if hard braking mostly happens on ice, then seeing hard braking makes ice more likely.

Intervening edits the data-generating process itself. Pearl’s do(A=a) replaces the structural equation A = π_b(H,ξ) with the constant assignment A := a. Graphically, it cuts every arrow into A while keeping the arrows out of it — the weather doesn’t change just because an experimenter forces the brake. The two operations differ in exactly one term:

Read term by term: Y is the outcome, A the action, U the common cause. The first weights road conditions by p(u|a) — the mix the old policy selected. The second weights by p(u) — the mix in the target population. They’re equal only under extra conditions: randomized action, U not affecting the action, or Y not varying with U. In a sequential model we want a belief-conditioned causal transition, p(S_t+1 | b_t, do(A_t=a)) — “start from this belief, then force this action.” Dropping the starting state (p(s'|a)) is usually too crude: the same throttle does very different things uphill, downhill, or against a wall.

4 · Do the numbers: hard braking looks harmful

Let U ∈ {dry, icy}, A ∈ {hard, gentle}, and Y=1 mean “stops within 20 metres.” Instead of a probability grid, build it the way nature and the driver do:

1. Nature picks the road. 80% dry, 20% icy — the population we care about.
2. The driver reads the road and brakes. Hard braking is rare on dry roads (probability 0.10) but common on ice (0.90). So the action itself leaks information about difficulty.
3. Physics turns road + action into outcome. Dry: success 0.90 hard, 0.70 gentle. Icy: both worse, but hard still beats gentle, 0.40 vs 0.10.

Notice the physical truth before any arithmetic: hard braking is better on each road. Yet the log can reverse that, because hard braking was aimed disproportionately at the harder road. Follow both computations so their different meanings stay visible.

What the log reports. Hard braking happens with probability 0.8×0.1 + 0.2×0.9 = 0.26; successful hard-brake cases have probability 0.8×0.1×0.9 + 0.2×0.9×0.4 = 0.144. So:

Gentle braking happens with probability 0.74; successful gentle cases have probability 0.8×0.9×0.7 + 0.2×0.1×0.1 = 0.506. So:

The log says hard braking is 13 points worse. Now intervene, holding the population’s road mix fixed:

The causal answer says hard braking is 22 points better. Nothing paradoxical happened — we answered two different questions with two different road mixtures. The old policy reserved hard braking for hard cases, so the action smuggled in the difficulty. If road condition is observed and blocks the backdoor, the second formula recovers the truth; if it’s hidden, the effect may not be recoverable at all — and a bigger transformer just estimates the observational distribution more precisely while staying precisely wrong about the intervention.

5 · Identification: when does data even determine the answer?

A causal effect is identifiable when the observed-data distribution plus stated assumptions pins it down uniquely. This is not the same as having enough samples: with few samples an identifiable effect just has wide error bars; with hidden confounding, infinite observational data can still leave several causal stories equally compatible. The usual routes to identification:

• Randomized actions. Assign the action independently of pre-action causes, and p(u|a)=p(u) — clean, but often unsafe or wasteful.
• Backdoor adjustment. Observe a pre-action set X that blocks every non-causal path, then estimate Σ_xp(y|a,x)p(x). Never adjust for descendants of the action — that erases part of the effect.
• Multiple behavior policies. Data from controllers with different habits helps separate stable physics from policy-specific correlation, though diversity alone doesn’t guarantee identification.
• Instrumental variables. An instrument shifts the action, has no direct path to the outcome, and shares no hidden cause with it — strong, domain-specific assumptions, not a generic neural trick.
• Known mechanisms or simulators. Conservation laws, geometry, and calibrated experiments restrict the possible models; simulation adds cheap interventions, transferring only as far as its mechanisms match reality.

6 · Positivity: you can’t learn an untried choice for free

Even with no confounding, estimation needs support (positivity, overlap). For every situation the target policy may visit, the data must have actually tried each action whose effect you want:

In words: if your new policy might choose a at belief b, the logger must have sometimes chosen it there. And “nonzero” is only the mathematical floor — if the probability is tiny, estimates have enormous variance and the network extrapolates instead of inferring.

This is why expert demonstrations are both precious and dangerous. Experts give competent behavior in common states, but they deliberately avoid collisions, unstable grasps, extreme torques, and recovery states — so action support is narrow and failures are missing. A planner optimizing against such a model searches far harder than any test set, and may discover an unsupported action with a gloriously high predicted reward. That is model exploitation, not intelligence.

The naive reflex is “collect more data,” but volume isn’t the missing ingredient — which alternatives were tried from which situations is. That dictates a staged strategy:

1. Start with expert logs to learn the ordinary operating region cheaply — knowing their competence is also their blind spot.
2. Add controlled random exploration where consequences are reversible. Randomization breaks the action↔difficulty link and gives locally clean effects; it’s unsafe as a universal policy, so keep it bounded.
3. Mix multiple purposeful policies to widen state and action coverage — though each can still use hidden information, so diversity reduces one weakness without erasing confounding.
4. Use a simulator to repeat dangerous or rare interventions cheaply via matched-start resets — inheriting a sim-to-real gap wherever contact, friction, or human response differ.
5. Finish with guarded online exploration aimed at decision-relevant uncertainty, under constraints, monitoring, and a fallback controller.

Order matters: experts ground normal behavior, controlled diversity exposes alternative effects, simulation expands the safe envelope, and guarded reality corrects simulator and model bias. No source suffices alone — each trades one kind of evidence for one kind of risk. And inverse-propensity weighting can correct observed selection by reweighting a sample by roughly π(a|b)/π_b(a|b) — but when π_b is near zero the weight explodes, when it’s zero nothing helps, and propensity methods never touch hidden confounding.

7 · Counterfactuals ask an even sharper question

An intervention is population-level: what outcome distribution would result if action a were forced? A counterfactual is about this specific episode: given that this vehicle actually braked gently and stopped after 24 metres, what would its stopping distance have been had it braked hard? In a structural causal model, that takes three steps:

1. Abduction: infer a posterior over the exogenous variables ε from the factual history — this episode’s particular friction, delay, disturbances.
2. Action: replace the action equation with A:=a'.
3. Prediction: re-run the modified mechanisms using the same inferred ε.

That “same noise” is what makes it counterfactual. Sampling a fresh road and fresh disturbances under each candidate action compares two different possible episodes — enough for expected-value planning, but not the individual counterfactual. Individual effects are generally harder to identify than average intervention effects, because you never observe both Y_a and Y_a′ for the same real episode.

8 · Some actions change the world; others reveal it

Actions reach utility by two routes. A world-changing action alters physical state: push the box, brake, deliver medicine. An information action alters what the agent will observe: move the camera past an occluder, tap an object to gauge stiffness, ask a clarifying question. One action can do both. For active perception, write the two mechanisms separately:

A camera rotation barely changes the scene state but strongly changes the next observation, which updates belief:

(predict the next state from the old belief, then reweight by how well each state explains the new observation). An information action is worth taking when that belief update improves later decisions — its immediate physical reward may be negative (it costs time) while its value of information is positive:

Term by term: expected value after seeing the action’s observation, minus value if you act now without it, minus sensing cost. A world model that predicts physical reward but never models belief change will systematically undervalue looking, probing, and asking.

9 · Train and plan without pretending uncertainty is knowledge

A practical system pairs causal design with uncertainty-aware control. Learn a belief representation from history, but pressure it to keep variables that predict outcomes under varied policies — not just variables that identify the logger. Train ensembles or a Bayesian approximation to expose disagreement in poorly supported regions, record behavior-policy probabilities when you can, and add intervention data where it matters. At planning time, score reward while penalizing unsupported or disputed transitions:

belief = infer_belief(observation_history, action_history)
for candidate_sequence in action_sequences:
    imagined = belief
    score = 0
    for action in candidate_sequence:
        next_dist = world_model.predict(imagined, do=action)
        uncertainty = ensemble_disagreement(next_dist)
        support = behavior_density(imagined, action)
        if violates_hard_constraint(next_dist):
            score = -infinity
            break
        score += expected_reward(next_dist)
        score -= beta * uncertainty
        score -= gamma * support_penalty(support)
        imagined = next_dist
    record (candidate_sequence, score)
choose first action of the highest-scoring candidate_sequence
execute, observe reality, update belief, and replan

The explicit do=action is a semantic reminder — an API flag cannot make observational training causal. β governs caution about ignorance; γ discourages straying from demonstrated support; hard constraints protect outcomes too costly to trade for reward; receding-horizon execution limits how long the agent trusts an uncorrected model. These safeguards have real costs: a big support penalty is safe but blocks improvement past the demonstrator; a small one allows discovery but invites exploitation. The right setting depends on reversibility, severity, monitoring latency, and how fast real observations correct the belief.

10 · Diagnostics: did it learn control, or just correlation?

Ordinary held-out likelihood is necessary but not sufficient, because train and test logs can share the same policy. Add tests that isolate the action’s effect:

• Matched-start intervention test. From the same measured or inferred state, execute different randomized actions and compare predicted vs actual outcome distributions.
• Action-swap test. Hold belief and exogenous seed fixed, swap only the candidate action, and check the predicted difference has the right sign, size, delay, and affected variables.
• Policy-shift test. Evaluate on data from a controller with different action frequencies; stable physics should transfer after accounting for changed state coverage.
• Negative control. Shuffle action labels or use an action known to have no effect; unexpected predictive power reveals leakage or confounding.
• Support map. Plot state–action density, disagreement, calibration error, and planner visitation together — average uncertainty can hide a narrow, dangerous blind spot.
• Constraint test. Under adversarially optimized actions, check conservation, collision geometry, actuator limits, and causal delay.
• Information-action test. Check that sensing actions reduce posterior uncertainty and improve later decisions — not merely make the next frame easy to predict.

The sharpest test freezes the start belief, optimizes actions against the model, then executes only in a high-fidelity simulator or guarded real setup. If predicted return rises while actual return falls, the planner found a model error — track this optimization gap as you turn up the optimization pressure.

11 · Failure modes and what they imply

Read each failure as a causal trace — symptom, then the shortcut or missing mechanism, then a test that can falsify it before you apply the repair.

1. Different actions produce nearly identical rollouts. Passive state already predicts most frames, so the lazy loss-minimizer ignores the action. Balance intervention data, add action-sensitivity tests, and weight task-relevant action consequences. Success = changing only the action produces the right sign, size, delay, and variables.
2. An action “works” only under the training controller. Policy identity or hidden state stayed confounded with the action. Collect multi-policy or randomized data, improve the causal state, and test under a controller with different frequencies. If the effect vanishes after the policy shift, the model learned the logger, not the physics.
3. The planner picks extreme, unnatural controls. Optimization crossed the support boundary into optimistic extrapolation. Constrain actions, penalize epistemic uncertainty, collect targeted safe data — and watch the predicted-vs-realized gap widen as pressure rises; that widening is the signature.
4. Prediction leans on camera shake or dashboard cues. A post-action cue leaks the command through an easier route than physics. Mask nuisance channels, alter overlays, randomize camera effects, test temporal order. A real action model keeps its prediction after those correlates are broken.
5. Counterfactual videos change unrelated background details. Each branch drew fresh latent noise, so it’s a different episode, not a different action in the same one. Share inferred exogenous variables across branches; unrelated background should then hold fixed while true descendants change.
6. The agent never looks before a risky manipulation. The model omits action-dependent observations or the value of information. Model belief updates and train sensing actions on later decision value, not immediate reward. Test whether a look reduces posterior uncertainty and improves the next choice.
7. Offline metrics are great, online return collapses. The planner induced a new state–action distribution the passive test set never measured. Evaluate closed-loop, map support along chosen plans, re-observe reality, and iterate collection. The decisive test is not another random holdout — it’s whether error concentrates where optimization chooses to go.

12 · A first-principles checklist

1. Name the causal query. Average intervention, individual counterfactual, or just forecasting the current policy? Specify outcome, horizon, starting population, and action.
2. Draw the mechanisms. Separate latent environment, sensor, behavior policy, reward, and possible hidden common causes; mark which variables are pre-action, post-action, or selected into the dataset.
3. Audit identification. State why the effect follows from the data: randomization, sufficient adjustment, a valid instrument, multiple environments plus invariance, or trusted structure.
4. Audit overlap. Compare target-policy actions with logger support in the states the target will visit; treat weak support as epistemic uncertainty, not a license to interpolate.
5. Design interventions. Spend experiment budget where effects are uncertain and decision-relevant; respect constraints, use simulation or guarded trials for high stakes.
6. Evaluate under optimization. Test matched starts, action swaps, policy shifts, long rollouts, and adversarial plans — prediction on untouched logs doesn’t simulate deployment.
7. Close the loop. Execute a short prefix, observe, update, detect mismatch, fall back safely. Causal learning cuts model error; feedback caps the cost of what remains.

Where this points next

We can now state the contract imagination needs: a belief-conditioned model that predicts the consequences of chosen actions, preserves uncertainty, and flags when a plan leaves identifiable support. Lesson 08 puts that model to work as a training environment — generating synthetic trajectories to train a policy inside imagination — while carrying the caution forward: imagined experience is only worth as much as the causal mechanism it visits.

Interview prompts

• Why can expert logs estimate the wrong braking effect? Experts choose strong braking in dangerous states, so danger causes both action and outcome. The observational comparison weights road conditions by p(u|a); the intervention weights them by p(u). Adjust for a sufficient pre-action state, randomize safely, or state that the effect is not identified.
• What exactly does do(A=a) change? It replaces the action-generating structural equation with A:=a and cuts incoming causal arrows to A. It does not erase action's effects, resample the starting state merely because the old policy preferred another action, or mean “filter logged rows where A=a.”
• When is p(s′|s,a) a valid causal transition? When s is a causally sufficient pre-action state that blocks common causes of action and next state, the action has adequate support at s, and the mechanism is stable across the policy change. A learned latent is not sufficient merely because it predicts held-out logs well.
• Distinguish identifiability, overlap, and uncertainty. Identifiability asks whether assumptions and infinite observed data uniquely determine the effect. Overlap asks whether relevant actions occur in relevant states. Statistical uncertainty asks how precisely finite data estimates an identifiable effect; epistemic uncertainty should grow when overlap is weak.
• How does a counterfactual differ from an intervention? An intervention describes outcomes under a forced action for a population. An individual counterfactual holds the factual episode's inferred exogenous conditions fixed, changes only the action equation, and predicts that same episode's alternative outcome.
• Why will a planner expose errors missed by test likelihood? Optimization actively searches action space for high predicted return and therefore selects rare combinations where approximation error is favorable. The induced distribution differs from passive test logs; support penalties, ensembles, constraints, adversarial evaluation, and replanning reduce the risk.
• What is an information action, and how should a world model value it? It is an action whose observation changes belief, such as moving a camera or probing stiffness. Its value is the expected improvement in later decisions after the belief update, minus sensing cost—not just immediate physical reward.
• Design a causal action-effect evaluation. Match or randomize starting states, execute several supported actions, compare calibrated outcome distributions, and repeat under a shifted policy. Add shared-noise action swaps, negative controls, constraint checks, support maps, and an optimization-gap test using plans selected by the model.