Adaptive Hints#

Use Adaptive Hints when downstream model calls or provider adapters can safely receive guidance metadata from the adaptive runtime.

Adaptive hints register as LLM request intercepts. Lower numeric priority values run earlier in the intercept chain. The default priority is chosen relative to other middleware rather than as a standalone importance score.

`plugins.toml` Example#

version = 1

[[components]]
kind = "adaptive"
enabled = true

[components.config]
version = 1
agent_id = "planner"

[components.config.state.backend]
kind = "in_memory"

[components.config.telemetry]
subscriber_name = "adaptive.telemetry"
learners = ["tool_parallelism"]

[components.config.adaptive_hints]
priority = 100
break_chain = false
inject_header = true
inject_body_path = "nvext.agent_hints"

This configuration injects adaptive guidance into outgoing model requests while allowing later request intercepts to continue running.

Plugin Configuration#

Use plugin configuration when the application should let NeMo Relay own the Adaptive Hints request-intercept lifecycle.

Python

import nemo_relay

adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
    agent_id="planner",
    state=nemo_relay.adaptive.StateConfig(
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
    ),
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
        inject_body_path="nvext.agent_hints",
    ),
)

plugin_config = nemo_relay.plugin.PluginConfig(
    components=[nemo_relay.adaptive.ComponentSpec(adaptive_config)]
)

report = nemo_relay.plugin.validate(plugin_config)
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
    raise RuntimeError(report["diagnostics"])

await nemo_relay.plugin.initialize(plugin_config)
try:
    # Run instrumented application work here.
    pass
finally:
    nemo_relay.plugin.clear()

Node.js

const adaptive = require("nemo-relay-node/adaptive");
const plugin = require("nemo-relay-node/plugin");

const adaptiveConfig = adaptive.defaultConfig();
adaptiveConfig.agent_id = "planner";
adaptiveConfig.state = { backend: adaptive.inMemoryBackend() };
adaptiveConfig.telemetry = adaptive.telemetryConfig({ learners: ["tool_parallelism"] });
adaptiveConfig.adaptive_hints = adaptive.adaptiveHintsConfig({
  inject_body_path: "nvext.agent_hints",
});

const pluginConfig = plugin.defaultConfig();
pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)];

const report = plugin.validate(pluginConfig);
if (report.diagnostics.some((diagnostic) => diagnostic.level === "error")) {
  throw new Error(JSON.stringify(report.diagnostics));
}

await plugin.initialize(pluginConfig);
try {
  // Run instrumented application work here.
} finally {
  plugin.clear();
}

Rust

use nemo_relay::plugin::{initialize_plugins, validate_plugin_config, PluginConfig};
use nemo_relay_adaptive::plugin_component::ComponentSpec;
use nemo_relay_adaptive::{
    AdaptiveConfig, AdaptiveHintsComponentConfig, BackendSpec, StateConfig, TelemetryComponentConfig,
};

let mut adaptive = AdaptiveConfig::default();
adaptive.agent_id = Some("planner".into());
adaptive.state = Some(StateConfig {
    backend: BackendSpec::in_memory(),
});
adaptive.telemetry = Some(TelemetryComponentConfig {
    learners: vec!["tool_parallelism".into()],
    ..TelemetryComponentConfig::default()
});
adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig {
    inject_body_path: "nvext.agent_hints".into(),
    ..AdaptiveHintsComponentConfig::default()
});

let mut plugin_config = PluginConfig::default();
plugin_config.components.push(ComponentSpec::new(adaptive).into());

let report = validate_plugin_config(&plugin_config);
assert!(!report.has_errors());

let active = initialize_plugins(plugin_config).await?;

Manual API#

Use the manual runtime API when an integration needs to own adaptive lifecycle directly instead of activating the top-level plugin component.

Python

import nemo_relay

adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
    agent_id="planner",
    state=nemo_relay.adaptive.StateConfig(
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
    ),
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
        inject_body_path="nvext.agent_hints",
    ),
)

runtime = nemo_relay.adaptive.AdaptiveRuntime(adaptive_config.to_dict())
await runtime.register()
try:
    # Run instrumented application work here.
    nemo_relay.adaptive.set_latency_sensitivity(8)
finally:
    await runtime.shutdown()

Node.js

The Node.js binding exposes Adaptive Hints through the adaptive plugin component helpers. Use the Plugin Configuration example above when activating Adaptive Hints from Node.js.

Rust

use nemo_relay_adaptive::{
    set_latency_sensitivity, AdaptiveConfig, AdaptiveHintsComponentConfig, AdaptiveRuntime,
    BackendSpec, StateConfig, TelemetryComponentConfig,
};

let mut adaptive = AdaptiveConfig::default();
adaptive.agent_id = Some("planner".into());
adaptive.state = Some(StateConfig {
    backend: BackendSpec::in_memory(),
});
adaptive.telemetry = Some(TelemetryComponentConfig {
    learners: vec!["tool_parallelism".into()],
    ..TelemetryComponentConfig::default()
});
adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig {
    inject_body_path: "nvext.agent_hints".into(),
    ..AdaptiveHintsComponentConfig::default()
});

let mut runtime = AdaptiveRuntime::new(adaptive).await?;
runtime.register().await?;

// Run instrumented application work here.
set_latency_sensitivity(8).ok();

runtime.shutdown().await?;

Fields#

Field	Default	Notes
`priority`	`100`	Request intercept priority. Lower values run earlier.
`break_chain`	`false`	Whether this intercept stops later request intercepts.
`inject_header`	`true`	Whether to add adaptive hints as request header metadata.
`inject_body_path`	`nvext.agent_hints`	JSON body path for request-body hint injection.

Disable break_chain unless the adaptive hint should be the final request transform. Adjust priority only when adaptive hints need to run before or after known application middleware.

Expected Output#

Outgoing managed LLM requests receive adaptive hint metadata in the configured header and body location. The hints do not replace the application callback or change the returned value by themselves. Downstream code must explicitly interpret the metadata before behavior changes.

Common Validation Failures#

Unknown adaptive hint fields when unknown fields are treated as errors.
inject_body_path does not match the request shape expected by downstream provider adapters.
Hint injection is enabled before downstream model paths can consume or ignore the metadata safely.