Generative Models#
nvalchemi provides an inference driver,
AtomisticGenerator, for generative workflows across
any model family. Whether you work with diffusion models, flow matching, generative
adversarial networks (GANs), variational autoencoders (VAEs), or heuristic search
algorithms, the driver provides a unified execution loop. It handles batching, seeds,
devices, CUDA streams, lifecycle hooks, and pipelining into dynamics simulations.
Tip
nvalchemi follows a batch-first principle. Generative workflows produce
batches of structures per call, not individual structures one at a time.
The mental model#
Generative inference separates model execution from workflow orchestration:
You write a generating function. This callable encapsulates the family-specific sampling logic and owns the model.
The driver runs the pipeline.
AtomisticGeneratormanages conditioning, device placement, random number generation, lifecycle hooks, and validation.The output is a
Batch. Returning aBatchlets hooks filter structures and allows immediate composition with dynamics engines.
BEFORE_CONDITION hooks (only when conditioning is present)
inputs = condition(inputs, num_samples=..., rng=...)
AFTER_CONDITION hooks (only when conditioning is present)
sample = generator_func(inputs, num_samples=..., rng=...)
if sample is a Batch: (the contract path)
ctx.batch = sample
AFTER_GENERATE hooks (filtering: ctx.batch = ctx.batch[keep])
return ctx.batch
return sample (other containers pass through raw)
The table below summarizes the core generative components:
Component |
Role |
|---|---|
The driver |
|
The sampling callable |
|
Optional input transform |
|
The lifecycle stages |
|
Per-call hook state |
|
Composition via |
|
The model-side mixin |
|
Model-side declaration |
The generating function and output contract#
A GeneratingFunction is any callable that accepts
inputs and returns generated structures. It receives the following arguments:
inputs: Conditioning data, an existingBatch, orNonefor unconditional generation.num_samples: Number of independent draws requested for the call.rng: An optionaltorch.Generatorinstance for reproducible draws.**kwargs: Additional runtime options forwarded from the caller.
The function owns the model, the sampling loop, and data materialization. Returning a
Batch follows the output contract:
The driver runs
AFTER_GENERATEfiltering hooks onctx.batch.The driver verifies device residency and declared batch fields.
Downstream dynamics engines can consume the batch directly.
If a function returns another container, the driver passes it through as raw output. The
driver skips AFTER_GENERATE hooks, and downstream dynamics stages will reject the
non-Batch output.
To signal total rejection, a generating function can return
empty(). Downstream pipeline stages then skip execution for
that batch.
Here is a minimal generating function producing random carbon clusters:
import torch
from nvalchemi.data import AtomicData, Batch
def random_cluster_generate(
inputs=None,
*,
num_samples: int = 1,
rng: torch.Generator | None = None,
num_atoms: int = 8,
**kwargs,
) -> Batch:
device = rng.device if rng is not None else "cpu"
positions = torch.randn(num_samples, num_atoms, 3, generator=rng, device=device)
atomic_numbers = torch.full((num_atoms,), 6, dtype=torch.long, device=device)
return Batch.from_data_list(
[AtomicData(positions=p, atomic_numbers=atomic_numbers) for p in positions]
)
Driving generation#
Wrap your generating function in AtomisticGenerator to
run it:
from nvalchemi.gen import AtomisticGenerator
gen = AtomisticGenerator(generator_func=random_cluster_generate, seed=42)
# Generate a batch of 4 structures
batch = gen.sample(num_samples=4) # gen(num_samples=4) is equivalent
Per-call lifecycle#
Every call to sample() follows a fixed
sequence:
Resolve conditioning: If a condition function exists (
condition_funcorgenerator_func.condition), the driver runsBEFORE_CONDITIONhooks, transformsinputs, and runsAFTER_CONDITIONhooks.Execute sampling: The driver calls
generator_funcwith the inputs, the resolvednum_samples, and the activerng.Inspect batch output: When
generator_funcreturns aBatch, the driver runsAFTER_GENERATEhooks, tests device residency, and tests declared batch fields.Pass through raw output: Any other return type passes through directly without batch tests or
AFTER_GENERATEhooks.
Calling gen(inputs, ...) is direct syntactic sugar for gen.sample(inputs, ...).
Driver configuration#
You configure the driver at initialization. The table below lists the available fields:
Field |
Purpose |
|---|---|
|
The generating callable that runs the model. |
|
Optional input transform run before generation. |
|
List of hooks that run at generation stages. |
|
Default draw count per call. A call argument overrides it. |
|
Base seed for random number generation. |
|
Target device, tested at initialization. |
|
Creates a dedicated CUDA stream inside sessions. |
|
Disables gradient tracking inside sessions. |
|
Options to compile with |
|
Declared input and output batch fields. |
Explicit arguments override attributes on generator_func. If device,
required_inputs, or outputs are unset, the driver reads them from
generator_func. The driver tests device and compile_kwargs at initialization.
For the full list, see the API reference.
Streaming#
stream() yields batches one at a time
across an iterable of inputs. For unconditional generation, pass inputs=None along
with max_batches:
for batch in gen.stream(None, max_batches=10, num_samples=4):
print(f"Generated batch with {batch.num_graphs} structures")
Calling iter(gen) or for batch in gen: creates an unbounded stream. Use
max_batches in stream() whenever you need a bounded loop.
Sessions: streams, seeds, and compilation#
AtomisticGenerator acts as a context manager:
with gen:
first_batch = gen.sample(num_samples=4)
second_batch = gen.sample(num_samples=4)
Bare calls (gen.sample(...) or gen(...)) are sufficient for quick draws. Outside
a session, each call seeds independently using seed + step_count.
Sessions (with gen:) manage state across multiple calls. Entering a session
performs four setup actions:
Dedicated CUDA stream: Enters a private CUDA stream on CUDA devices (disable with
dedicated_stream=False). The stream waits on pending work at entry.Session-scoped RNG: Initializes a
torch.Generatorfromseedthat advances across draws.Inference mode: Enters
torch.inference_modewhenenable_inference_mode=True(off by default). Enable it when your sampler does not need gradients.Hook lifecycles: Calls
__enter__on context-manager hooks for clean teardown on exit.
You can compile the generating function with torch.compile via
gen.compile(**compile_kwargs) or by setting compile_generate=True. Compilation
wraps the function call. Hook dispatch stays eager.
Building a Batch runs Python and pydantic code that TorchDynamo cannot trace, so a
function that constructs a Batch graph-breaks at construction. For end-to-end
capture, compile the model inside your generating function
(torch.compile(model, ...)) and keep the wrapper eager.
Devices, streams, and compile#
The driver resolves its target device at construction time. The driver reads
the explicit device field first, and then falls back to the device attribute
on the generating function, with the following conditions:
If you specify a CUDA device, that device must exist on the host.
When the generating function returns a
Batch, the batch tensors must reside on the resolved device (i.e. the function cannot change devices).
A device mismatch raises a ValueError during the call. In a pipeline,
each stage resolves its own device, and its outputs pass to the next stage
without automatic device migration.
Using the context manager pattern (i.e. with gen: ...), the generation
workflow will run on a CUDA stream unless dedicated_stream is set to False.
At session entry, the new stream waits for active work
on the current caller stream, so session operations do not race caller tasks. A
pipeline session creates one CUDA stream and shares that stream with every stage
that follows the _stream convention. These stages include generators, dynamics
engines, and fused stages. The pipeline shares the stream only when the stage
device matches the pipeline stream device, so mixed-device pipelines do not
share streams across devices.
The shared stream serializes stage execution within a pipeline call without
cross-stream synchronization. One caveat however is that exiting a session
does not synchronize the CUDA stream: if you dispatch operations on other
streams, you must synchronize or
enqueue a wait_stream call before you consume the results. As the session
overview explains, compile_generate and compile_kwargs compile the
generating function call. Because batch construction causes graph breaks in
TorchDynamo, you must compile the neural network model inside the generating
function for end-to-end graph capture.
Input conditioning#
Conditioning prepares inputs before generation runs. It converts raw inputs (such as class labels, compositions, or parent structures) into the representation expected by the generating function.
You supply conditioning through condition_func on the generator, or via a condition
attribute on the generating function callable. The driver resolves conditioning by
priority:
Explicit
condition_funcpassed toAtomisticGenerator.A
conditionattribute ongenerator_func.None(unconditional; inputs pass directly togenerator_func).
The BEFORE_CONDITION and AFTER_CONDITION hook stages fire only when a condition step
is present.
Conditioning prepares the request; it does not enforce physical constraints on the output. Output constraints belong in guidance terms inside the sampler, in filtering hooks, or in downstream relaxation stages.
Here is a common pattern: tiling a conditioning batch so each structure receives
num_samples draws:
def tile_condition(inputs, *, num_samples: int = 1, rng=None):
if isinstance(inputs, Batch):
idx = torch.arange(inputs.num_graphs).repeat_interleave(num_samples)
return inputs[idx.to(inputs.device)]
return inputs
Lifecycle hooks and GenerationContext#
Generative workflows use the same Hook protocol as
dynamics and training. A hook defines stage,
frequency, and __call__(ctx, stage).
All hooks in a call share a single GenerationContext instance:
Field |
Description |
|---|---|
|
The generated |
|
Raw inputs before conditioning, conditioned inputs after. |
|
Raw sample returned by the generating function. |
|
Optional boolean tensor indicating accepted candidates. |
|
Scratch dictionary for passing data between hooks. |
|
Counter of generation calls driving frequency gating. |
|
Distributed process rank for multi-GPU reporting. |
|
Back-reference to the driving generator. |
Hooks mutate state by replacing context fields. For example, filtering at
AFTER_GENERATE subsets ctx.batch:
import torch
from nvalchemi.data import Batch
from nvalchemi.gen import GenerationStage
class CentroidSpreadFilter:
def __init__(self, max_spread: float = 3.0) -> None:
self.max_spread = max_spread
self.stage = GenerationStage.AFTER_GENERATE
self.frequency = 1
def __call__(self, ctx, stage) -> None:
batch = ctx.batch
counts = batch.num_nodes_per_graph
idx = torch.repeat_interleave(
torch.arange(batch.num_graphs, device=batch.positions.device), counts
)
centroid = torch.zeros(batch.num_graphs, 3, device=batch.positions.device)
centroid.index_add_(0, idx, batch.positions)
centroid = centroid / counts[:, None]
dist = (batch.positions - centroid[idx]).norm(dim=-1)
max_dist = torch.zeros(batch.num_graphs, device=batch.positions.device)
max_dist.scatter_reduce_(0, idx, dist, reduce="amax")
keep = max_dist <= self.max_spread
ctx.accepted_mask = keep
if not keep.any():
# Batch does not support zero-graph selections; signal total
# rejection with an explicitly empty batch instead
ctx.batch = Batch.empty(
num_systems=0,
num_nodes=0,
num_edges=0,
device=batch.positions.device,
)
return
ctx.batch = batch[keep]
Attach the hook during generator construction:
gen = AtomisticGenerator(
generator_func=random_cluster_generate,
hooks=[CentroidSpreadFilter(max_spread=2.5)],
)
The model side: mixin and config#
When generation relies on a neural network, the model subclasses
GenerativeModelMixin and declares its capabilities
via GenerativeModelConfig.
This is the non-energy counterpart to BaseModelMixin. It
does not require energy, forces, or neighbor lists.
Every subclass must set self.model_config in __init__:
import torch
from torch import nn
from nvalchemi.data import AtomicData, Batch
from nvalchemi.models.gen import GenerativeModelConfig, GenerativeModelMixin
class ToyDecoder(nn.Module, GenerativeModelMixin):
def __init__(self, num_atoms: int, latent_dim: int = 16) -> None:
super().__init__()
self.latent_dim = latent_dim
self.num_atoms = num_atoms
self.net = nn.Sequential(
nn.Linear(latent_dim, 64),
nn.SiLU(),
nn.Linear(64, num_atoms * 3),
)
self.model_config = GenerativeModelConfig(
required_inputs=frozenset(), # unconditional
outputs=frozenset({"positions", "atomic_numbers"}),
prediction_outputs={"positions"},
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)
Model-owning samplers#
In nvalchemi, the model does not own the sampler. Instead, a sampler callable owns the
model:
class ToyGenerate:
def __init__(self, model: ToyDecoder) -> None:
self.model = model
self.required_inputs = model.model_config.required_inputs
self.outputs = model.model_config.outputs
@property
def device(self) -> torch.device:
return next(self.model.parameters()).device
def __call__(
self,
inputs=None,
*,
num_samples: int = 1,
rng: torch.Generator | None = None,
**kwargs,
) -> Batch:
z = torch.randn(
num_samples, self.model.latent_dim, generator=rng, device=self.device
)
positions = self.model(z).reshape(num_samples, self.model.num_atoms, 3)
numbers = torch.full(
(self.model.num_atoms,), 6, dtype=torch.long, device=self.device
)
return Batch.from_data_list(
[AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
)
The driver reads required_inputs, outputs, and device off the sampler object as
default declarations.
For testing and rapid prototyping, nvalchemi provides ready-made demo models:
DemoGANModel and
DemoDiffusionModel.
Composing pipelines with GenerationPipeline#
You can chain multiple generative stages, custom transformations, and dynamics simulations using the | operator:
from nvalchemi.dynamics import FIRE2, ConvergenceHook
from nvalchemi.gen import AtomisticGenerator
from nvalchemi.models.demo import DemoModel, DemoModelWrapper
# Stage 1: Generate initial candidate structures
gen_stage = AtomisticGenerator(generator_func=ToyGenerate(ToyDecoder(num_atoms=8)))
# Stage 2: Relax candidates with an ML potential optimizer
relax_stage = FIRE2(
model=DemoModelWrapper(DemoModel()), # swap in your own model here
dt=0.1,
n_steps=200,
convergence_hook=ConvergenceHook.from_fmax(0.05),
)
# Chain into a sequential pipeline; per-stage options go through stage_kwargs
pipeline = gen_stage | relax_stage
with pipeline: # session: shared stream, seeded RNG, stage sessions
relaxed_batch = pipeline(stage_kwargs=[{"num_samples": 8}, None])
Pipeline execution rules#
GenerationPipeline executes stages sequentially:
Stage invocation: Stages with a
run()method (such as dynamics engines) run to completion viastage.run(batch, **kwargs). Other callables run viastage(batch, **kwargs).Field contract validation: At construction, adjacent
AtomisticGeneratorstages are validated:downstream.required_inputsmust be a subset ofupstream.outputs.Shared CUDA stream: Entering
with pipeline:creates a single dedicated CUDA stream shared across all stages honoring the_streamconvention.Stage arguments: Pass per-stage keyword arguments using
stage_kwargsas a list of dictionaries aligned with pipeline stages.
out = pipeline(stage_kwargs=[{"num_samples": 8}, {"n_steps": 100}])
Integrations and model families#
Sampling procedures differ across model families. Below are common integration patterns.
PhysicsNeMo diffusion#
nvalchemi integrates directly with NVIDIA PhysicsNeMo diffusion abstractions
(physicsnemo.diffusion). Noise schedulers, preconditioners, and ODE/SDE samplers plug
into a generating function without adapter layers:
from typing import Any
import torch
from torch import nn
from physicsnemo.diffusion.noise_schedulers import EDMNoiseScheduler
from physicsnemo.diffusion.preconditioners import EDMPreconditioner
from physicsnemo.diffusion.samplers import sample as pn_sample
from nvalchemi.data import AtomicData, Batch
from nvalchemi.gen import AtomisticGenerator
class PositionDenoiser(nn.Module):
def __init__(self, num_atoms: int) -> None:
super().__init__()
self.num_atoms = num_atoms
self.net = nn.Sequential(
nn.Linear(num_atoms * 3 + 1, 64),
nn.SiLU(),
nn.Linear(64, num_atoms * 3),
)
def forward(
self,
x: torch.Tensor,
sigma: torch.Tensor,
class_labels: torch.Tensor | None = None,
) -> torch.Tensor:
b = x.shape[0]
s = sigma.reshape(b, 1).expand(b, 1)
return self.net(torch.cat([x.reshape(b, -1), s], dim=-1)).reshape_as(x)
class EDMGenerate:
def __init__(
self,
model: PositionDenoiser,
*,
num_steps: int = 18,
sigma_max: float = 5.0,
) -> None:
self.model = model
self.num_steps = num_steps
self.scheduler = EDMNoiseScheduler(sigma_max=sigma_max)
self.required_inputs = frozenset()
self.outputs = frozenset({"positions", "atomic_numbers"})
@property
def device(self) -> torch.device:
return next(self.model.parameters()).device
def __call__(
self,
inputs: Any = None,
*,
num_samples: int = 1,
rng: torch.Generator | None = None,
**kwargs: Any,
) -> Batch:
denoiser = self.scheduler.get_denoiser(
x0_predictor=EDMPreconditioner(self.model)
)
xN = torch.randn(
num_samples, self.model.num_atoms, 3, generator=rng, device=self.device
)
xN = xN * self.scheduler.sigma_max
x0 = pn_sample(
denoiser, xN, self.scheduler, num_steps=self.num_steps, solver="heun"
)
numbers = torch.full(
(self.model.num_atoms,), 6, dtype=torch.long, device=self.device
)
return Batch.from_data_list(
[AtomicData(positions=p, atomic_numbers=numbers) for p in x0]
)
diffusion = AtomisticGenerator(
generator_func=EDMGenerate(PositionDenoiser(num_atoms=32), num_steps=18),
seed=42,
)
with diffusion:
batch = diffusion.sample(num_samples=16)
Deterministic solvers (like "heun" or "euler") derive all randomness from the
initial noise tensor xN. Drawing xN with the session’s rng ensures exact
reproducibility from seed.
Generative Adversarial Networks (GAN)#
GAN generation decodes a single latent noise vector in one forward pass:
class GANGenerate:
def __init__(self, model) -> None:
self.model = model
@property
def device(self) -> torch.device:
return next(self.model.parameters()).device
def __call__(
self,
inputs=None,
*,
num_samples: int = 1,
rng: torch.Generator | None = None,
**kwargs,
) -> Batch:
z = torch.randn(
num_samples, self.model.latent_dim, generator=rng, device=self.device
)
positions = self.model.decode(z).reshape(num_samples, -1, 3)
numbers = torch.full(
(positions.shape[1],), 6, dtype=torch.long, device=self.device
)
return Batch.from_data_list(
[AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
)
from nvalchemi.models.gen import DemoGANModel
# the demo model satisfies this sketch's interface (latent_dim + decode)
gan = AtomisticGenerator(generator_func=GANGenerate(DemoGANModel()))
samples = gan(num_samples=4)
Variational Autoencoders (VAE)#
A VAE decodes a latent draw like a GAN, but the latent can also come from the encoder — which is how a VAE conditions on input structures:
class VAEGenerate:
def __init__(self, model) -> None:
self.model = model
@property
def device(self) -> torch.device:
return next(self.model.parameters()).device
def __call__(
self,
inputs=None,
*,
num_samples: int = 1,
rng: torch.Generator | None = None,
**kwargs,
) -> Batch:
if inputs is None:
# unconditional: sample the prior
z = torch.randn(
num_samples, self.model.latent_dim, generator=rng, device=self.device
)
n = num_samples
else:
# conditional: encode the inputs, then reparametrize —
# one draw per input graph; for repeated draws per input, add a
# condition step that tiles the batch (the driver does not tile
# on its own)
mu, logvar = self.model.encode(inputs)
n = mu.shape[0]
eps = torch.randn(n, self.model.latent_dim, generator=rng, device=mu.device)
z = mu + eps * torch.exp(0.5 * logvar)
positions = self.model.decode(z).reshape(n, -1, 3)
numbers = torch.full(
(positions.shape[1],), 6, dtype=torch.long, device=z.device
)
return Batch.from_data_list(
[AtomicData(positions=p, atomic_numbers=numbers) for p in positions]
)
vae = AtomisticGenerator(generator_func=VAEGenerate(vae_model))
samples = vae(num_samples=4)
For a complete executable script demonstrating model-owning samplers with both GAN and
diffusion architectures, see examples/intermediate/09_generative_samplers.py.
What’s next#
Dynamics — Relax or run MD on generated structures.
Hooks — Learn the shared hook protocol and reporting system.
Training — Train or fine-tune neural network backbones.
AtomicData and Batch — Understand graph data representations, node and edge features, and batching.
Generative API reference — Explore the complete class and method reference.