Why this topic
Flow matching (Lipman et al., 2023) trains a generative model by regressing a velocity field. There is no noise schedule to tune, no variational bound with many terms, and no ODE to integrate during training. At sampling time you start from Gaussian noise and follow the learned velocity with an ordinary ODE solver. It now underlies several large image and video models, and its training loop fits in five lines.
The difficulty is that those five lines rest on a stack of ideas from different fields: diffeomorphisms, the continuity equation from fluid mechanics, Neural ODEs, optimal transport and continuous normalizing flows. I wrote these notes to build that stack from the bottom up, in the order I needed it, before deriving the flow matching theorems. This post introduces the idea and maps out the pages; the derivations are in the notes below.
The core idea
Picture a cloud of particles drawn from a Gaussian. Give every point in space, at every moment $t \in [0, 1]$, a velocity, and let the particles drift. If the velocity field is chosen well, by $t = 1$ the cloud has been rearranged into the shape of your data. Generative modelling then reduces to learning that velocity field.
Formally, a time-dependent vector field $v_t : [0,1] \times \mathbb{R}^d \to \mathbb{R}^d$ defines a flow $\phi_t$ through the ODE
$$ \frac{d}{dt}\phi_t(x) = v_t(\phi_t(x)), \qquad \phi_0(x) = x, $$and pushing the noise distribution $p_0$ through $\phi_t$ gives a probability path $p_t = [\phi_t]_\# p_0$. Probability mass is neither created nor destroyed as it moves, which is the continuity equation linking a path $p_t$ to a vector field $u_t$ that generates it:
$$ \frac{\partial p_t(x)}{\partial t} + \nabla \cdot \big(p_t(x)\, u_t(x)\big) = 0 . $$Continuous normalizing flows train $v_t$ by maximum likelihood, which requires integrating the divergence $\nabla \cdot v_t$ along every trajectory during training. Flow matching instead regresses the model $v_t$ directly onto a target field $u_t$ that generates a chosen path $p_t$ from noise to data:
$$ \mathcal{L}_{FM}(\theta) = \mathbb{E}_{t,\, p_t(x)} \big\| v_t(x) - u_t(x) \big\|^2 . $$The catch is that neither the marginal path $p_t$ nor $u_t$ is known. The resolution is to build everything from conditional paths $p_t(x \mid x_1)$, each one ending at a single data point $x_1 \sim q(x_1)$. The marginal path $p_t(x) = \int p_t(x \mid x_1)\, q(x_1)\, dx_1$ is generated by an average of conditional fields (Theorem 1), and the conditional objective
$$ \mathcal{L}_{CFM}(\theta) = \mathbb{E}_{t,\, q(x_1),\, p_t(x \mid x_1)} \big\| v_t(x) - u_t(x \mid x_1) \big\|^2 $$differs from $\mathcal{L}_{FM}$ only by a constant independent of $\theta$, so both have the same gradients (Theorem 2). Every quantity in $\mathcal{L}_{CFM}$ is now tractable.
Choosing a Gaussian conditional path $p_t(x \mid x_1) = \mathcal{N}(x \mid \mu_t(x_1), \sigma_t(x_1)^2 I)$ with the flow $\psi_t(x) = \sigma_t(x_1)\,x + \mu_t(x_1)$ gives the field $u_t(x \mid x_1) = \frac{\sigma_t'(x_1)}{\sigma_t(x_1)}(x - \mu_t(x_1)) + \mu_t'(x_1)$ (Theorem 3). The optimal transport choice $\mu_t = t x_1$, $\sigma_t = 1 - (1 - \sigma_{\min})t$ moves each noise sample along a straight line, and the loss becomes
$$ \mathcal{L}_{CFM}(\theta) = \mathbb{E}_{t,\, q(x_1),\, p(x_0)} \big\| v_t(\psi_t(x_0)) - \big(x_1 - (1 - \sigma_{\min})\,x_0\big) \big\|^2 . $$With $\sigma_{\min} = 0$, the point is $x_t = (1-t)\,x_0 + t\,x_1$ and the regression target is simply $x_1 - x_0$.
A guide to the notes
| Pages | What is derived |
|---|---|
| 1 | Title page. |
| 2 | Diffeomorphisms and the change-of-variables formula $\tilde p(x) = p(\Phi^{-1}(x)) / \lvert \det \partial\Phi/\partial x \rvert$; invariance of KL under diffeomorphisms; the continuity equation in physics (mass density) and for probability densities. |
| 3 | Differential equations (pendulum, projectile, two-body examples); vector fields and phase space; ODE solution $x(t) = x_0 + \int_0^t v(x(\tau), \tau)\,d\tau$; the flow map $\Phi$ and phase flow. |
| 4 | Neural ODEs: a ResNet block $x_{k+1} = x_k + f(x_k)$ as an Euler step, its continuous-depth limit $\dot x = f_\theta(x)$, and the flow map as the hidden-state trajectory. |
| 5 | Optimal transport: particle view with a Lipschitz velocity field, empirical measures converging to a density, the Kantorovich formulation over couplings $\Pi(\mu,\nu)$, the Wasserstein-2 distance, and McCann's displacement interpolation $((1-t)\,\text{id} + tT)_\# \mu$. |
| 6 | Continuous normalizing flows: vector field, flow, probability path as a push-forward; objective $-\log p_1(z(1))$; from the continuity equation to $\frac{d}{dt}\log p_t = -\nabla \cdot v_t$ and $\mathcal{L}_{CNF} = -\log p(z(0)) + \int_0^1 \nabla \cdot v\, dt$. |
| 7 | The Jacobian trace is expensive in high dimension; Hutchinson's estimator $\nabla \cdot v = \mathbb{E}_\epsilon[\epsilon^\top J \epsilon]$ via autograd; integrating backwards from $z(1)$ to $z(0)$ for training, forwards for inference. |
| 8 | Flow matching setup and $\mathcal{L}_{FM}$; why it is not directly computable ($u_t$ and $p_t$ unknown); contrast with CNFs, which only constrain the endpoints. |
| 9 | Data distribution $q$ and target $p_1 \approx q$; many paths connect noise to data; conditional paths $p_t(x \mid x_1)$, the marginal path, and the start of the marginal vector field derivation from the continuity equation. |
| 10 | Marginal field $u_t(x) = \int u_t(x \mid x_1)\frac{p_t(x \mid x_1)\, q(x_1)}{p_t(x)}\,dx_1$; Theorem 1 (marginal field generates marginal path). |
| 11 | Expanding $\mathcal{L}_{FM}$ term by term and rewriting each expectation over $p_t(x)$ as one over $p_t(x \mid x_1)\,q(x_1)$. |
| 12 | Completing the square to obtain $\mathcal{L}_{CFM}$; Theorem 2 (equal gradients up to a $\theta$-independent constant). |
| 13 | Gaussian conditional paths, boundary conditions $\mu_0 = 0, \sigma_0 = 1$ and $\mu_1 = x_1, \sigma_1 = \sigma_{\min}$; the canonical affine flow $\psi_t(x) = \sigma_t(x_1)\,x + \mu_t(x_1)$ and its endpoint checks. |
| 14 | Differentiating $\psi_t$ to get $u_t(x \mid x_1)$; Theorem 3; rewriting $\mathcal{L}_{CFM}$ in terms of noise $x_0$ and data $x_1$. |
| 15 | Example 1: diffusion paths (variance exploding and variance preserving) as special cases. Example 2: optimal transport paths, their straight-line flow and vector field, and the resulting $\mathcal{L}_{CFM}$; the $\sigma_{\min} = 0$ case $x_t = (1-t)x_0 + t x_1$. |
| 16 | Summary and key ideas; training pseudocode and Euler-integration inference pseudocode with $\sigma_{\min} = 0$. |
From derivation to code
The repository trains a class-conditional flow matching model on MNIST (codes/FlowMatching.ipynb, mirrored in codes/flow_matching.py). It is the page 16 pseudocode almost verbatim:
- Training. For each image $x_1$ (scaled to $[-1, 1]$) the code draws $x_0 \sim \mathcal{N}(0, I)$ and $t \sim \mathcal{U}(0, 1)$, forms
x_t = (1 - t) * x_noise + t * x_real, setsv_target = x_real - x_noise, and minimises the mean squared error betweenmodel(x_t, t, y)and that target. This is $\mathcal{L}_{CFM}$ for the optimal transport path with $\sigma_{\min} = 0$. - Network. The same conditional U-Net used for the diffusion model, here predicting a velocity rather than noise; $t$ and the digit label $y$ are embedded and added.
- Sampling. Start from Gaussian noise and take 50 explicit Euler steps,
x = x + v * dtwithdt = 1/50, querying the model at $t = i \cdot dt$. No noise is injected along the way: the sampler is a deterministic ODE solve.
How it relates to diffusion
Page 15 makes the connection explicit: diffusion models correspond to particular Gaussian probability paths, so they fit inside the same framework. What changes is the path and the regression target. DDPM fixes a noise schedule and predicts the noise $\epsilon$ added at a discrete step; flow matching with optimal transport paths picks the straight line between noise and data and predicts the constant velocity along it. The notes also record that the diffusion paths only approach pure noise asymptotically, whereas the flow matching path hits $\mathcal{N}(0, I)$ exactly at $t = 0$.
In the companion repository the two models share the architecture and data pipeline, which makes the comparison clean: the diffusion sampler uses 500 stochastic steps, the flow sampler 50 deterministic ones. The Hugging Face Space shows both trajectories side by side. For the derivation of the discrete-time side, see Understanding DDPM.
The notes
Can’t see the viewer? Open the PDF in a new tab.