Flow matching — pick a path, not a chain

A different design choice with a shockingly short derivation: the entire training objective is one regression line.

The frame shift

DDPM built a path by designing a noise process: pick β_t, induce q(x_t), derive everything else. The path is implicit, curved, and tied to Gaussian noising.

Flow matching flips it: design the path directly. Pick any one-parameter family of distributions {p_t}_t∈[0,1] with p₀ = N(0, I) and p₁ = p_data. Learn the velocity field v_θ(x, t) that pushes mass along it. Sample by ODE integration.

The objective doesn’t depend on the path being Gaussian, Markov, or anything in particular. That freedom is the design lever — we’ll use it to pick a path that’s easy to integrate (lesson 6).

The continuity equation

The compatibility condition: a time-varying density p_t and velocity u_t are compatible (the field actually moves mass along the path) iff they satisfy

This is just conservation of probability mass written in PDE form — the same equation as fluid dynamics. It says: the rate of change of density at x equals the negative divergence of the flux p_t u_t.

If we knew the “marginal velocity” u_t(x) at every point, sampling would be easy: integrate dx/dt = u_t(x) starting at x₀ ∼ p₀; you land in p₁. The trouble: u_t(x) has no closed form for any non-trivial p_data. We can’t evaluate it, so we can’t regress against it directly.

The CFM trick: conditional paths

Pick a conditional path that’s trivial: for each pair (x₀, x₁) with x₀ ∼ p₀, x₁ ∼ p_data, define a path from x₀ to x₁. The linear (OT) path is

This is a straight segment; the conditional velocity is its time derivative, which is constant:

Both quantities are evaluable: given a sampled (x₀, x₁, t), the conditional x_t is one linear combination and the conditional velocity is one subtraction. Zero networks needed.

The marginal-conditional equivalence

Why does regressing against the easy conditional thing give us the impossible marginal one? Because conditional expectation is the minimizer of squared error. The marginal velocity that’s compatible with p_t is

That’s the average of the constant conditional velocities, weighted by which (x₀, x₁) pairs are likely to have produced this particular x_t. And the regression

has minimizer exactly u_t(x) = 𝔼[x₁ − x₀ | x_t = x] — the same thing! So we can regress against the easy target and recover the impossible marginal one. This is the entire CFM identity.

The training objective in one line

That’s the whole thing. In Python (FlowMatching.loss):

x0 = randn_like(x1)
t  = rand(B)
xt = (1 - t) * x0 + t * x1
target = x1 - x0
pred   = self.model(xt, t)
return ((pred - target) ** 2).mean()

Five lines. Compare to the half-page ELBO derivation behind DDPM’s identical-looking but conceptually different MSE.

Interactive · the conditional velocity field

The widget below visualizes Eq. ●. Pick a t; we sample many (x₀, x₁) pairs, place each x_t at (1 − t) x₀ + t x₁, and draw an arrow showing the constant conditional velocity x₁ − x₀. Notice how, at each point in the cloud, several arrows of different directions exist — the average of those arrows is the marginal velocity field, which is what the net learns.

Interactive · train a velocity net, see the field

The previous widget shows the target field (averaged from sampled pairs). Here’s what a trained network actually learns: a smooth field v_θ(x, t) defined on the whole plane. Hit train, then drag t: the arrows are what the model would push samples toward if they were at that grid point at that time. At t = 0 the field points outward from the origin (move noise toward data); at t = 1 the field shrinks toward zero (we’re already at data).

Why this is “better” than diffusion (sometimes)

• Path is straight. Each conditional trajectory is a line. The marginal field inherits much of that straightness, so a low-order ODE integrator (Euler with 20–50 steps) is enough. Lesson 6 demos this.
• Time is just U(0, 1). No β schedule, no choice of T, no special β-spacing tricks. The path is the design choice; everything else is the path.
• Scale-stable targets. ‖x₁ − x₀‖ is O(1) at every t, just like ε in DDPM. No reweighting.
• Composes cleanly with anything. Want a different path? Swap one line. Want video? Replace (x₀, x₁) with (z₀, z₁) in some latent space. Want guidance? Same trick as classifier-free guidance in DDPM, mechanically.

Trade-offs vs. DDPM, summarized