Denoising Markov Models Generalize Diffusion Through Reverse-Time Generators

Yinuo Ren frames diffusion-based generative modeling as a problem of designing a Markovian probability path from data to an easy reference law, then learning how to run that path backward. In the familiar continuous case, the forward process moves data toward a simple reference distribution such as a Gaussian; the reverse-time process moves samples from the reference distribution back toward the data distribution. The learned object is usually the score, the gradient of the log intermediate density.

Ren’s broader motivation also includes stochastic interpolants and flow matching, which he presents as evidence that generative models can learn along prescribed probability paths. His proposed framework is narrower and more specific: choose a forward Markov process that sends the data law toward a simple reference law, ensure the process can be simulated explicitly, and train the reverse process with a tractable loss. In continuous diffusion, that forward process is an SDE on Euclidean space. In discrete diffusion, it is more naturally a continuous-time Markov chain with jumps on a finite state space. For applications involving multimodal data, scientific data, geometric constraints, or hard constraints, Ren argues that the natural object to study is not “diffusion” narrowly understood, but a general Markov process.

That shift matters because the learned quantity changes with the process. In continuous diffusion, the reverse generator depends on $\nabla \log p_t$, the score. In discrete diffusion, the reverse jump rates depend on ratios such as $p_t(y)/p_t(x)$. The loss also changes: continuous diffusion gives the familiar squared-error score-matching loss, while discrete diffusion gives a score-entropy-like objective. Ren’s framework treats both as coordinate choices inside a single generator-level construction.

The conceptual picture is simple, even if the mathematical machinery is not. One starts with samples $x_0\sim p_0$ from the data distribution. A forward noising process with generator ${\mathcal{L}}_t$ pushes those samples toward an easy-to-sample reference distribution at time $T$. That same forward process induces a true reverse-time process, but this reverse process depends on unknown intermediate densities. The practical model therefore introduces an approximate backward generator ${\mathcal{K}}_t$, learned from data and used to run an approximate reverse process from an initial law $q_0$ that is intended to be close to $p_T$.

When Carles Domingo-Enrich asks how to interpret the measure $\mathbb{Q}$ in the diagram, Ren clarifies that $\mathbb{P}$ and $\mathbb{Q}$ are path measures, not initial distributions. The distinction is central: the framework compares entire stochastic paths, not merely endpoints.

The generator of a Markov process is the infinitesimal expected change of a test function. For a Markov process $(x_t{)}_{t\in [0,T]}$ on a state space $E$, Ren defines the time-dependent generator as

This generator compactly describes the evolution of the marginal densities through the Kolmogorov forward equation ${\partial }_t{p}_t={\mathcal{L}}_t^{\ast }{p}_t$. Dynkin’s formula plays the corresponding integral role, linking expected function changes along the process to integrals of the generator. Ren also introduces the carré du champ operator,

which measures the extent to which the stochastic generator fails to behave like an ordinary derivative. If the operator were merely a derivative, this bilinear term would vanish. In stochastic settings it is the object that captures perturbations appearing in time reversal, functional inequalities, nonequilibrium dynamics, and Doob-type transforms.

The first of the three gaps is time reversal: what is the relationship between the forward generator ${\mathcal{L}}_t$ and the true backward generator ${\tilde{\mathcal{L}}}_t$? Under suitable regularity assumptions on the densities $p_t$, the time-reversal theorem states that

The reverse generator is therefore the adjoint forward generator plus a density-dependent perturbation. This is the abstract version of what happens in continuous and discrete diffusion. In continuous diffusion, the unknown density enters through a score. In discrete diffusion, it enters through score ratios. At the generator level, both are forms of the same perturbation term.

The second gap is parameterization. Since the true density $p_t$ is unknown, Ren restricts the approximate backward generator to the same structural family but replaces $p_t$ with a strictly positive function ${\phi }_t:E\to {\mathbb{R}}_{+}$:

In practice, ${\phi }_t$, or an equivalent coordinate representation, is parameterized by a neural network. This is not an arbitrary search over all possible backward Markov processes. It is a search over density-dependent perturbations of the adjoint generator, deliberately mirroring the exact time-reversal formula. If ${\phi }_t=p_t$, then ${\mathcal{K}}_t$ equals the true reverse generator. Ren calls this the “holy grail” of the parameterization.

The third gap is the mismatch between the approximate backward generator and the true backward generator. Algebraically, this gap can be written in terms of the mismatch ratio

With this definition, the approximate generator is itself a perturbation of the true backward generator:

Thus ${\mathcal{K}}_t$ being a ${\phi }_t$-perturbation of the adjoint forward generator is equivalent to being an ${\eta }_t$-perturbation of the true reverse generator. The ideal case is ${\eta }_t\equiv 1$, meaning perfect recovery of the true backward process.

The framework becomes trainable once the generator mismatch is connected to a quantitative path-space discrepancy. Ren does this through a generalized Doob’s $h$-transform.

Given a Markov generator ${\mathcal{L}}_t$, a positive function $h_t$, and a function ${\lambda }_t$, the generalized transform defines

Under the theorem Ren cites, the transformed process is absolutely continuous with respect to the original path measure, and the Radon–Nikodym derivative has an explicit pathwise form. Ren identifies the estimated backward generator ${\mathcal{K}}_{T-t}$ as the generalized Doob transform of the true backward generator ${\tilde{\mathcal{L}}}_{T-t}$, with $h_t$ equal to the mismatch ratio ${\eta }_t$. Choosing

makes the transformed generator conservative, so the approximate backward process is a valid Markov process rather than an operator that loses or creates total probability mass.

The training implication comes from the resulting change-of-measure theorem. By plugging the time-reversal formula and Dynkin’s formula for $\log {\eta }_t$ into the generalized transform, Ren obtains a path-space KL expression. Under suitable assumptions, there exists a path measure $\mathbb{Q}$ such that the time-reversal process under $\mathbb{Q}$ is governed by ${\mathcal{K}}_t$, and

Here $\mathbb{P}$ is the path law of the forward process and its true reverse; $\mathbb{Q}$ is the reweighted path law inducing the approximate backward generator. The functional $\mathcal{L}[{\eta }_t]$ is the exact path-space discrepancy between the true and approximate backward processes.

When Carles Domingo-Enrich asks whether, in the purely continuous case, this recovers the Girsanov change of measure, Ren answers yes. If one plugs in the generator corresponding to a diffusion process, the expression reduces to the familiar squared-error loss. In Ren’s phrasing, it “magically” reduces to the $L^2$ loss.

The algorithmic consequence is the error bound. By the tower property of KL divergence, the terminal law $q_T$ generated by the approximate backward process satisfies

Ren interprets the first term as terminal mismatch: how well the chosen forward process reaches the easy reference distribution. The second is estimation error: how well the learned backward process matches the true backward process.

The point is that the loss controls an upper bound on the generative KL error, together with the terminal mismatch created by the forward design. That is the connection between the abstract path-space construction and the endpoint distribution produced by the model.

There remains a practical problem: ${\eta }_t$ contains the unknown density $p_t$. Ren’s generalized score-matching result replaces the marginal ratio by a conditional ratio,

Up to constants independent of model parameters, minimizing the path-space KL is equivalent to minimizing a generalized score-matching loss involving the data distribution $p_0$, the forward transition kernel $p_{t|0}$, and the forward generator ${\mathcal{L}}_t$. This is the point where the abstract generator construction becomes an algorithm: sample data, sample an intermediate time, simulate the forward process to get $x_t$, evaluate the generalized score-matching integrand, and update the parameters of ${\phi }_t^{\theta }$.

At inference time, the procedure reverses in the qualified sense shown on Ren’s meta-algorithm slide: sample $y_0$ from $q_0\approx p_T$, simulate the learned backward generator ${\mathcal{K}}_t^{\theta }$ from time 0 to $T$, and output $y_T$ with the goal that $q_T\approx p_0$.

Ren also adds numerical simulation error to the bound. If the numerical simulation of the approximate backward process has one-step error $\mathcal{O}({\kappa }^{r+1})$, the generated law ${\hat{q}}_T$ satisfies a bound of the form

This separates three errors: truncation error from imperfect convergence of the forward process, estimation error from learning the wrong reverse dynamics, and numerical error from discretizing the backward process.

The framework recovers standard continuous diffusion when the state space is ${\mathbb{R}}^d$ and the forward generator has diffusion form,

Plugging this generator into the time-reversal and parameterization formulas gives a backward generator of the same diffusion type, with score coordinates

The resulting generalized score-matching loss involves the diffusion matrix $D_t$:

Ren notes that when $D_t$ is proportional to the identity, this becomes the standard diffusion score-matching objective, up to time reweighting.

Discrete diffusion appears when the state space is finite and the forward generator is a jump process,

In that case, the backward rates are reweighted by the ratio of the positive parameterization function at two states:

The natural score coordinate is therefore the ratio

This recovers the discrete diffusion objective Ren associates with score-entropy losses. The significance is not that continuous and discrete objectives look identical; they do not. The significance is that both arise from the same generator-level parameterization and the same path-space KL principle.

Lévy-type models show why expanding beyond Gaussian diffusion is not just a formal generalization. The general KL guarantee contains the terminal mismatch term $D_{\mathrm{KL}}(p_T\Vert q_0)$. If the forward process and reference law are poorly matched to the data, this term can break the guarantee regardless of how well the reverse process is learned.

The obstruction Ren highlights is heavy tails. For affine Gaussian forward diffusions, if the data law $p_0$ is heavy-tailed with infinite second moment and the reference $q_0$ is Gaussian, then for every positive $T$,

Conceptually, affine Gaussian forward dynamics do not repair a tail mismatch against a Gaussian terminal reference. Even if the learning objective were driven to zero, the terminal mismatch would remain uncontrolled. Ren takes this as motivation for forward processes with matching tails and jump behavior, such as $\alpha$-stable processes.

The general form he uses for Lévy-type models is the Courrège form. Under suitable regularity assumptions, the forward generator can be written as a sum of deterministic transport, local Gaussian diffusion, and nonlocal jumps:

The transport term moves mass deterministically. The diffusion term accounts for local Gaussian noise. The integral term describes jumps from $x$ to $y$, with a small-jump compensation term because infinitely many small jumps can be indistinguishable from local transport.

Plugging this generator into Ren’s framework yields a backward Lévy triplet. The reverse jumps are reweighted by ${\phi }_t(y)/{\phi }_t(x)$, the diffusion coefficient remains $D_t$, and the backward drift includes the usual diffusion-score term plus an additional small-jump compensation term. The path-space KL decomposes into a diffusion term and a jump term:

with

Ren’s conclusion from this decomposition is that one can design generative models from broader Lévy processes, provided the corresponding loss can be learned and the backward dynamics can be simulated.

Ren’s examples illustrate what changes when the forward process is chosen for the geometry of the problem rather than treated as a default implementation detail.

The first uses geometric Brownian motion as the forward process. The setup preserves the positive orthant ${\mathbb{R}}_{++}^d$ by construction. The forward process has multiplicative noise,

with diffusion matrix

The target data are mixtures of Gaussians restricted to the positive orthant, and the model is trained with the generalized diffusion score-matching loss. Ren reports that training remains stable even though the score can be singular near the boundary, and generated samples match the target without boundary violations. The example supports his broader point: choosing dynamics consistent with the state space can enforce constraints structurally rather than by post hoc correction.

The second experiment is a pure jump model in continuous space. Here the drift and diffusion coefficient are both zero, and the domain is the torus ${\mathbb{T}}^2={\mathbb{R}}^2/{\mathbb{Z}}^2$. The forward Lévy kernel is an isotropic Gaussian jump kernel, and the initial backward law is approximately uniform on the torus. Samples start from the uniform law and progressively jump into structured target distributions such as chessboard, Swiss roll, and moons patterns.

Ren emphasizes that this is not simply discrete diffusion translated to another notation. Discrete diffusion uses jump processes on a finite state space. The pure jump experiment uses jumps for generative modeling in continuous space. It produces probability paths that are neither ordinary deterministic transport nor diffusion. He contrasts this denoising setup with generator matching: here one defines a forward process and searches for its corresponding backward process, rather than prescribing a probability path first and learning the generator that realizes it.

In the Q&A, Carles Domingo-Enrich asks whether the Lévy-type section is motivated mainly by heavy-tailed distributions or by the possibility of modeling systems with both continuous and discrete aspects. Ren answers that he does have multimodal distributions in mind. Although the diffusion and jump terms appear decoupled in the KL loss, the pure jump example suggests to him that jump terms can also be imposed on continuous components in generative models. His safer design implication is that jumps may help model multimodal structure and create probability paths not available to pure diffusion.

Domingo-Enrich also asks whether typical modeled distributions, such as images, audio, video, or scientific data, are heavy-tailed in practice. Ren points to prior work using $\alpha$-stable processes for generative modeling and notes that these models require care. The backward Lévy triplet can be complicated, so practical backward simulation often involves approximations. For Ren, an important open direction is to understand and improve those approximations through further investigation and better numerical schemes.

The recipe Ren leaves is four-part: choose a forward Markov process with generator ${\mathcal{L}}_t$ that sends the data distribution toward an easy reference; use the time-reversal theorem to identify the exact backward generator, which depends on the intermediate density $p_t$; parameterize the approximate backward generator by a positive function ${\phi }_t$ in the same structural form as the true reverse generator; and learn ${\phi }_t$ through the change-of-measure theorem connecting Doob’s $h$-transform and the path-space KL objective.

The framework leaves several design questions open. Ren singles out the need to understand how forward dynamics affect trainability, efficiency, and robustness; to develop scalable inference and numerical methods for general Markov and Lévy-type processes; and to select forward processes in a principled way for constraints, topology, and distributional characteristics.

He also notes that the framework itself has assumptions. It assumes the process is Markovian, though generative modeling may not require Markovianity. It also forces the forward and backward processes to live in the same space, which may not be necessary either. For Ren, those assumptions mark the boundary of the current framework, not the boundary of the design space.