Generative Models#

nvalchemi provides an inference driver, AtomisticGenerator, for generative workflows across any model family. Whether you work with diffusion models, flow matching, generative adversarial networks (GANs), variational autoencoders (VAEs), or heuristic search algorithms, the driver provides a unified execution loop. It handles batching, seeds, devices, CUDA streams, lifecycle hooks, and pipelining into dynamics simulations.

Tip

nvalchemi follows a batch-first principle. Generative workflows produce batches of structures per call, not individual structures one at a time.

The mental model#

Generative inference separates model execution from workflow orchestration:

  1. You write a generating function. This callable encapsulates the family-specific sampling logic and owns the model.

  2. The driver runs the pipeline. AtomisticGenerator manages conditioning, device placement, random number generation, lifecycle hooks, and validation.

  3. The output is a Batch. Returning a Batch lets hooks filter structures and allows immediate composition with dynamics engines.

BEFORE_CONDITION   hooks                      (only when conditioning is present)
                   inputs = condition(inputs, num_samples=..., rng=...)
AFTER_CONDITION    hooks                      (only when conditioning is present)
                   sample = generator_func(inputs, num_samples=..., rng=...)
if sample is a Batch:                         (the contract path)
    ctx.batch = sample
    AFTER_GENERATE hooks                      (filtering: ctx.batch = ctx.batch[keep])
    return ctx.batch
return sample                                 (other containers pass through raw)

The table below summarizes the core generative components:

Component

Role

AtomisticGenerator

The driver

GeneratingFunction

The sampling callable

ConditionFunction

Optional input transform

GenerationStage

The lifecycle stages

GenerationContext

Per-call hook state

GenerationPipeline

Composition via |

GenerativeModelMixin

The model-side mixin

GenerativeModelConfig

Model-side declaration

The generating function and output contract#

A GeneratingFunction is any callable that accepts inputs and returns generated structures. It receives the following arguments:

  • inputs: Conditioning data, an existing Batch, or None for unconditional generation.

  • num_samples: Number of independent draws requested for the call.

  • rng: An optional torch.Generator instance for reproducible draws.

  • **kwargs: Additional runtime options forwarded from the caller.

The function owns the model, the sampling loop, and data materialization. Returning a Batch follows the output contract:

  • The driver runs AFTER_GENERATE filtering hooks on ctx.batch.

  • The driver verifies device residency and declared batch fields.

  • Downstream dynamics engines can consume the batch directly.

If a function returns another container, the driver passes it through as raw output. The driver skips AFTER_GENERATE hooks, and downstream dynamics stages will reject the non-Batch output.

To signal total rejection, a generating function can return empty(). Downstream pipeline stages then skip execution for that batch.

Here is a minimal generating function producing random carbon clusters:

import torch
from nvalchemi.data import AtomicData, Batch

def random_cluster_generate(
    inputs=None,
    *,
    num_samples: int = 1,
    rng: torch.Generator | None = None,
    num_atoms: int = 8,
    **kwargs,
) -> Batch:
    device = rng.device if rng is not None else "cpu"
    positions = torch.randn(num_samples, num_atoms, 3, generator=rng, device=device)
    atomic_numbers = torch.full((num_atoms,), 6, dtype=torch.long, device=device)
    return Batch.from_data_list(
        [AtomicData(positions=p, atomic_numbers=atomic_numbers) for p in positions]
    )

Driving generation#

Wrap your generating function in AtomisticGenerator to run it:

from nvalchemi.gen import AtomisticGenerator

gen = AtomisticGenerator(generator_func=random_cluster_generate, seed=42)

# Generate a batch of 4 structures
batch = gen.sample(num_samples=4)  # gen(num_samples=4) is equivalent

Per-call lifecycle#

Every call to sample() follows a fixed sequence:

  1. Resolve conditioning: If a condition function exists (condition_func or generator_func.condition), the driver runs BEFORE_CONDITION hooks, transforms inputs, and runs AFTER_CONDITION hooks.

  2. Execute sampling: The driver calls generator_func with the inputs, the resolved num_samples, and the active rng.

  3. Inspect batch output: When generator_func returns a Batch, the driver runs AFTER_GENERATE hooks, tests device residency, and tests declared batch fields.

  4. Pass through raw output: Any other return type passes through directly without batch tests or AFTER_GENERATE hooks.

Calling gen(inputs, ...) is direct syntactic sugar for gen.sample(inputs, ...).

Driver configuration#

You configure the driver at initialization. The table below lists the available fields:

Field

Purpose

generator_func

The generating callable that runs the model.

condition_func

Optional input transform run before generation.

hooks

List of hooks that run at generation stages.

num_samples

Default draw count per call. A call argument overrides it.

seed

Base seed for random number generation.

device

Target device, tested at initialization.

dedicated_stream

Creates a dedicated CUDA stream inside sessions.

enable_inference_mode

Disables gradient tracking inside sessions.

compile_generate, compile_kwargs

Options to compile with torch.compile.

required_inputs, outputs

Declared input and output batch fields.

Explicit arguments override attributes on generator_func. If device, required_inputs, or outputs are unset, the driver reads them from generator_func. The driver tests device and compile_kwargs at initialization. For the full list, see the API reference.

Streaming#

stream() yields batches one at a time across an iterable of inputs. For unconditional generation, pass inputs=None along with max_batches:

for batch in gen.stream(None, max_batches=10, num_samples=4):
    print(f"Generated batch with {batch.num_graphs} structures")

Calling iter(gen) or for batch in gen: creates an unbounded stream. Use max_batches in stream() whenever you need a bounded loop.

Sessions: streams, seeds, and compilation#

AtomisticGenerator acts as a context manager:

with gen:
    first_batch = gen.sample(num_samples=4)
    second_batch = gen.sample(num_samples=4)

Bare calls (gen.sample(...) or gen(...)) are sufficient for quick draws. Outside a session, each call seeds independently using seed + step_count.

Sessions (with gen:) manage state across multiple calls. Entering a session performs four setup actions:

  1. Dedicated CUDA stream: Enters a private CUDA stream on CUDA devices (disable with dedicated_stream=False). The stream waits on pending work at entry.

  2. Session-scoped RNG: Initializes a torch.Generator from seed that advances across draws.

  3. Inference mode: Enters torch.inference_mode when enable_inference_mode=True (off by default). Enable it when your sampler does not need gradients.

  4. Hook lifecycles: Calls __enter__ on context-manager hooks for clean teardown on exit.

You can compile the generating function with torch.compile via gen.compile(**compile_kwargs) or by setting compile_generate=True. Compilation wraps the function call. Hook dispatch stays eager.

Building a Batch runs Python and pydantic code that TorchDynamo cannot trace, so a function that constructs a Batch graph-breaks at construction. For end-to-end capture, compile the model inside your generating function (torch.compile(model, ...)) and keep the wrapper eager.

Devices, streams, and compile#

The driver resolves its target device at construction time. The driver reads the explicit device field first, and then falls back to the device attribute on the generating function, with the following conditions:

  • If you specify a CUDA device, that device must exist on the host.

  • When the generating function returns a Batch, the batch tensors must reside on the resolved device (i.e. the function cannot change devices).

A device mismatch raises a ValueError during the call. In a pipeline, each stage resolves its own device, and its outputs pass to the next stage without automatic device migration.

Using the context manager pattern (i.e. with gen: ...), the generation workflow will run on a CUDA stream unless dedicated_stream is set to False. At session entry, the new stream waits for active work on the current caller stream, so session operations do not race caller tasks. A pipeline session creates one CUDA stream and shares that stream with every stage that follows the _stream convention. These stages include generators, dynamics engines, and fused stages. The pipeline shares the stream only when the stage device matches the pipeline stream device, so mixed-device pipelines do not share streams across devices.

The shared stream serializes stage execution within a pipeline call without cross-stream synchronization. One caveat however is that exiting a session does not synchronize the CUDA stream: if you dispatch operations on other streams, you must synchronize or enqueue a wait_stream call before you consume the results. As the session overview explains, compile_generate and compile_kwargs compile the generating function call. Because batch construction causes graph breaks in TorchDynamo, you must compile the neural network model inside the generating function for end-to-end graph capture.

Input conditioning#

Conditioning prepares inputs before generation runs. It converts raw inputs (such as class labels, compositions, or parent structures) into the representation expected by the generating function.

You supply conditioning through condition_func on the generator, or via a condition attribute on the generating function callable. The driver resolves conditioning by priority:

  1. Explicit condition_func passed to AtomisticGenerator.

  2. A condition attribute on generator_func.

  3. None (unconditional; inputs pass directly to generator_func).

The BEFORE_CONDITION and AFTER_CONDITION hook stages fire only when a condition step is present.

Conditioning prepares the request; it does not enforce physical constraints on the output. Output constraints belong in guidance terms inside the sampler, in filtering hooks, or in downstream relaxation stages.

Here is a common pattern: tiling a conditioning batch so each structure receives num_samples draws:

def tile_condition(inputs, *, num_samples: int = 1, rng=None):
    if isinstance(inputs, Batch):
        idx = torch.arange(inputs.num_graphs).repeat_interleave(num_samples)
        return inputs[idx.to(inputs.device)]
    return inputs

Lifecycle hooks and GenerationContext#

Generative workflows use the same Hook protocol as dynamics and training. A hook defines stage, frequency, and __call__(ctx, stage).

All hooks in a call share a single GenerationContext instance:

Field

Description

ctx.batch

The generated Batch (or a Batch input).

ctx.inputs

Raw inputs before conditioning, conditioned inputs after.

ctx.sample

Raw sample returned by the generating function.

ctx.accepted_mask

Optional boolean tensor indicating accepted candidates.

ctx.intermediates

Scratch dictionary for passing data between hooks.

ctx.step_count

Counter of generation calls driving frequency gating.

ctx.global_rank

Distributed process rank for multi-GPU reporting.

ctx.workflow

Back-reference to the driving generator.

Hooks mutate state by replacing context fields. For example, filtering at AFTER_GENERATE subsets ctx.batch:

import torch

from nvalchemi.data import Batch
from nvalchemi.gen import GenerationStage


class CentroidSpreadFilter:
    def __init__(self, max_spread: float = 3.0) -> None:
        self.max_spread = max_spread
        self.stage = GenerationStage.AFTER_GENERATE
        self.frequency = 1

    def __call__(self, ctx, stage) -> None:
        batch = ctx.batch
        counts = batch.num_nodes_per_graph
        idx = torch.repeat_interleave(
            torch.arange(batch.num_graphs, device=batch.positions.device), counts
        )
        centroid = torch.zeros(batch.num_graphs, 3, device=batch.positions.device)
        centroid.index_add_(0, idx, batch.positions)
        centroid = centroid / counts[:, None]

        dist = (batch.positions - centroid[idx]).norm(dim=-1)
        max_dist = torch.zeros(batch.num_graphs, device=batch.positions.device)
        max_dist.scatter_reduce_(0, idx, dist, reduce="amax")

        keep = max_dist <= self.max_spread
        ctx.accepted_mask = keep
        if not keep.any():
            # Batch does not support zero-graph selections; signal total
            # rejection with an explicitly empty batch instead
            ctx.batch = Batch.empty(
                num_systems=0,
                num_nodes=0,
                num_edges=0,
                device=batch.positions.device,
            )
            return
        ctx.batch = batch[keep]

Attach the hook during generator construction:

gen = AtomisticGenerator(
    generator_func=random_cluster_generate,
    hooks=[CentroidSpreadFilter(max_spread=2.5)],
)

The model side: mixin and config#

When generation relies on a neural network, the model subclasses GenerativeModelMixin and declares its capabilities via GenerativeModelConfig.

This is the non-energy counterpart to BaseModelMixin. It does not require energy, forces, or neighbor lists.

Every subclass must set self.model_config in __init__:

import torch
from torch import nn
from nvalchemi.data import AtomicData, Batch
from nvalchemi.models.gen import GenerativeModelConfig, GenerativeModelMixin

class ToyDecoder(nn.Module, GenerativeModelMixin):
    def __init__(self, num_atoms: int, latent_dim: int = 16) -> None:
        super().__init__()
        self.latent_dim = latent_dim
        self.num_atoms = num_atoms
        self.net = nn.Sequential(
            nn.Linear(latent_dim, 64),
            nn.SiLU(),
            nn.Linear(64, num_atoms * 3),
        )
        self.model_config = GenerativeModelConfig(
            required_inputs=frozenset(),  # unconditional
            outputs=frozenset({"positions", "atomic_numbers"}),
            prediction_outputs={"positions"},
        )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.net(x)

Model-owning samplers#

In nvalchemi, the model does not own the sampler. Instead, a sampler callable owns the model:

class ToyGenerate:
    def __init__(self, model: ToyDecoder) -> None:
        self.model = model
        self.required_inputs = model.model_config.required_inputs
        self.outputs = model.model_config.outputs

    @property
    def device(self) -> torch.device:
        return next(self.model.parameters()).device

    def __call__(
        self,
        inputs=None,
        *,
        num_samples: int = 1,
        rng: torch.Generator | None = None,
        **kwargs,
    ) -> Batch:
        z = torch.randn(
            num_samples, self.model.latent_dim, generator=rng, device=self.device
        )
        positions = self.model(z).reshape(num_samples, self.model.num_atoms, 3)
        numbers = torch.full(
            (self.model.num_atoms,), 6, dtype=torch.long, device=self.device
        )
        return Batch.from_data_list(
            [AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
        )

The driver reads required_inputs, outputs, and device off the sampler object as default declarations.

For testing and rapid prototyping, nvalchemi provides ready-made demo models: DemoGANModel and DemoDiffusionModel.

Composing pipelines with GenerationPipeline#

You can chain multiple generative stages, custom transformations, and dynamics simulations using the | operator:

from nvalchemi.dynamics import FIRE2, ConvergenceHook
from nvalchemi.gen import AtomisticGenerator
from nvalchemi.models.demo import DemoModel, DemoModelWrapper

# Stage 1: Generate initial candidate structures
gen_stage = AtomisticGenerator(generator_func=ToyGenerate(ToyDecoder(num_atoms=8)))

# Stage 2: Relax candidates with an ML potential optimizer
relax_stage = FIRE2(
    model=DemoModelWrapper(DemoModel()),  # swap in your own model here
    dt=0.1,
    n_steps=200,
    convergence_hook=ConvergenceHook.from_fmax(0.05),
)

# Chain into a sequential pipeline; per-stage options go through stage_kwargs
pipeline = gen_stage | relax_stage
with pipeline:  # session: shared stream, seeded RNG, stage sessions
    relaxed_batch = pipeline(stage_kwargs=[{"num_samples": 8}, None])

Pipeline execution rules#

GenerationPipeline executes stages sequentially:

  • Stage invocation: Stages with a run() method (such as dynamics engines) run to completion via stage.run(batch, **kwargs). Other callables run via stage(batch, **kwargs).

  • Field contract validation: At construction, adjacent AtomisticGenerator stages are validated: downstream.required_inputs must be a subset of upstream.outputs.

  • Shared CUDA stream: Entering with pipeline: creates a single dedicated CUDA stream shared across all stages honoring the _stream convention.

  • Stage arguments: Pass per-stage keyword arguments using stage_kwargs as a list of dictionaries aligned with pipeline stages.

out = pipeline(stage_kwargs=[{"num_samples": 8}, {"n_steps": 100}])

Integrations and model families#

Sampling procedures differ across model families. Below are common integration patterns.

PhysicsNeMo diffusion#

nvalchemi integrates directly with NVIDIA PhysicsNeMo diffusion abstractions (physicsnemo.diffusion). Noise schedulers, preconditioners, and ODE/SDE samplers plug into a generating function without adapter layers:

from typing import Any
import torch
from torch import nn
from physicsnemo.diffusion.noise_schedulers import EDMNoiseScheduler
from physicsnemo.diffusion.preconditioners import EDMPreconditioner
from physicsnemo.diffusion.samplers import sample as pn_sample

from nvalchemi.data import AtomicData, Batch
from nvalchemi.gen import AtomisticGenerator

class PositionDenoiser(nn.Module):
    def __init__(self, num_atoms: int) -> None:
        super().__init__()
        self.num_atoms = num_atoms
        self.net = nn.Sequential(
            nn.Linear(num_atoms * 3 + 1, 64),
            nn.SiLU(),
            nn.Linear(64, num_atoms * 3),
        )

    def forward(
        self,
        x: torch.Tensor,
        sigma: torch.Tensor,
        class_labels: torch.Tensor | None = None,
    ) -> torch.Tensor:
        b = x.shape[0]
        s = sigma.reshape(b, 1).expand(b, 1)
        return self.net(torch.cat([x.reshape(b, -1), s], dim=-1)).reshape_as(x)

class EDMGenerate:
    def __init__(
        self,
        model: PositionDenoiser,
        *,
        num_steps: int = 18,
        sigma_max: float = 5.0,
    ) -> None:
        self.model = model
        self.num_steps = num_steps
        self.scheduler = EDMNoiseScheduler(sigma_max=sigma_max)
        self.required_inputs = frozenset()
        self.outputs = frozenset({"positions", "atomic_numbers"})

    @property
    def device(self) -> torch.device:
        return next(self.model.parameters()).device

    def __call__(
        self,
        inputs: Any = None,
        *,
        num_samples: int = 1,
        rng: torch.Generator | None = None,
        **kwargs: Any,
    ) -> Batch:
        denoiser = self.scheduler.get_denoiser(
            x0_predictor=EDMPreconditioner(self.model)
        )
        xN = torch.randn(
            num_samples, self.model.num_atoms, 3, generator=rng, device=self.device
        )
        xN = xN * self.scheduler.sigma_max
        x0 = pn_sample(
            denoiser, xN, self.scheduler, num_steps=self.num_steps, solver="heun"
        )
        numbers = torch.full(
            (self.model.num_atoms,), 6, dtype=torch.long, device=self.device
        )
        return Batch.from_data_list(
            [AtomicData(positions=p, atomic_numbers=numbers) for p in x0]
        )

diffusion = AtomisticGenerator(
    generator_func=EDMGenerate(PositionDenoiser(num_atoms=32), num_steps=18),
    seed=42,
)

with diffusion:
    batch = diffusion.sample(num_samples=16)

Deterministic solvers (like "heun" or "euler") derive all randomness from the initial noise tensor xN. Drawing xN with the session’s rng ensures exact reproducibility from seed.

Generative Adversarial Networks (GAN)#

GAN generation decodes a single latent noise vector in one forward pass:

class GANGenerate:
    def __init__(self, model) -> None:
        self.model = model

    @property
    def device(self) -> torch.device:
        return next(self.model.parameters()).device

    def __call__(
        self,
        inputs=None,
        *,
        num_samples: int = 1,
        rng: torch.Generator | None = None,
        **kwargs,
    ) -> Batch:
        z = torch.randn(
            num_samples, self.model.latent_dim, generator=rng, device=self.device
        )
        positions = self.model.decode(z).reshape(num_samples, -1, 3)
        numbers = torch.full(
            (positions.shape[1],), 6, dtype=torch.long, device=self.device
        )
        return Batch.from_data_list(
            [AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
        )

from nvalchemi.models.gen import DemoGANModel

# the demo model satisfies this sketch's interface (latent_dim + decode)
gan = AtomisticGenerator(generator_func=GANGenerate(DemoGANModel()))
samples = gan(num_samples=4)

Variational Autoencoders (VAE)#

A VAE decodes a latent draw like a GAN, but the latent can also come from the encoder — which is how a VAE conditions on input structures:

class VAEGenerate:
    def __init__(self, model) -> None:
        self.model = model

    @property
    def device(self) -> torch.device:
        return next(self.model.parameters()).device

    def __call__(
        self,
        inputs=None,
        *,
        num_samples: int = 1,
        rng: torch.Generator | None = None,
        **kwargs,
    ) -> Batch:
        if inputs is None:
            # unconditional: sample the prior
            z = torch.randn(
                num_samples, self.model.latent_dim, generator=rng, device=self.device
            )
            n = num_samples
        else:
            # conditional: encode the inputs, then reparametrize —
            # one draw per input graph; for repeated draws per input, add a
            # condition step that tiles the batch (the driver does not tile
            # on its own)
            mu, logvar = self.model.encode(inputs)
            n = mu.shape[0]
            eps = torch.randn(n, self.model.latent_dim, generator=rng, device=mu.device)
            z = mu + eps * torch.exp(0.5 * logvar)
        positions = self.model.decode(z).reshape(n, -1, 3)
        numbers = torch.full(
            (positions.shape[1],), 6, dtype=torch.long, device=z.device
        )
        return Batch.from_data_list(
            [AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
        )

vae = AtomisticGenerator(generator_func=VAEGenerate(vae_model))
samples = vae(num_samples=4)

For a complete executable script demonstrating model-owning samplers with both GAN and diffusion architectures, see examples/intermediate/09_generative_samplers.py.

What’s next#

  • Dynamics — Relax or run MD on generated structures.

  • Hooks — Learn the shared hook protocol and reporting system.

  • Training — Train or fine-tune neural network backbones.

  • AtomicData and Batch — Understand graph data representations, node and edge features, and batching.

  • Generative API reference — Explore the complete class and method reference.

See also#