Observability Configuration#

Use this page when an application should install standard observability exporters from one plugin configuration document instead of manually registering each subscriber.

The plugin kind is observability. It is registered by the core runtime, so applications do not need to register a plugin implementation before validation or initialization.

For plugin file discovery, precedence, merge behavior, editor controls, and gateway conflict rules, see Plugin Configuration Files.

Note

Observability plugin configuration uses the generic NeMo Flow plugin document shape, so field names are snake_case in every binding. This differs from Node.js runtime classes such as OpenTelemetrySubscriber, which use Node-native camelCase option names outside the plugin system.

What It Installs#

Every exporter section is optional and defaults to disabled. A section is active only when it includes enabled: true.

Section	Runtime behavior
`atof`	Registers a global Agent Trajectory Observability Format (ATOF) JSONL exporter for raw lifecycle events.
`atif`	Registers one Agent Trajectory Interchange Format (ATIF) dispatcher that writes one trajectory file for each top-level agent scope.
`opentelemetry`	Registers a global OpenTelemetry OTLP subscriber.
`openinference`	Registers a global OpenInference OTLP subscriber.

subscriber_name is not part of this config. The runtime infers component-local subscriber names and registers them under the observability plugin namespace:

Agent Trajectory Observability Format (ATOF): __nemo_flow_plugin__observability__atof
Agent Trajectory Interchange Format (ATIF) dispatcher: __nemo_flow_plugin__observability__atif
Per-agent ATIF scope subscriber: __nemo_flow_plugin__observability__atif-{agent_scope_uuid}
OpenTelemetry: __nemo_flow_plugin__observability__opentelemetry
OpenInference: __nemo_flow_plugin__observability__openinference

`plugins.toml` Example#

version = 1

[[components]]
kind = "observability"
enabled = true

[components.config]
version = 1

[components.config.atof]
enabled = true
output_directory = "logs"
filename = "events.jsonl"
mode = "overwrite"

[components.config.atif]
enabled = true
output_directory = "logs"
filename_template = "trajectory-{session_id}.json"

[components.config.opentelemetry]
enabled = true
transport = "http_binary"
endpoint = "http://localhost:4318/v1/traces"
service_name = "nemo-flow"
service_namespace = "agent"
service_version = "0.2.0"
instrumentation_scope = "nemo-flow-observability"
timeout_millis = 3000

[components.config.opentelemetry.headers]
authorization = "Bearer <token>"

[components.config.opentelemetry.resource_attributes]
"deployment.environment" = "dev"
"service.instance.id" = "local"

[components.config.openinference]
enabled = true
transport = "http_binary"
endpoint = "http://localhost:6006/v1/traces"
service_name = "nemo-flow"
service_namespace = "agent"
service_version = "0.2.0"
instrumentation_scope = "nemo-flow-openinference"
timeout_millis = 3000

[components.config.openinference.headers]
authorization = "Bearer <token>"

[components.config.openinference.resource_attributes]
"deployment.environment" = "dev"
"service.instance.id" = "local"

[components.config.policy]
unknown_component = "warn"
unknown_field = "warn"
unsupported_value = "error"

Include only the sections you want to configure. In layered plugins.toml files, omission inherits lower-precedence values; write enabled = false to disable an inherited section.

Per-Language Plugin Configuration#

Python

from nemo_flow import plugin, scope, ScopeType
from nemo_flow.observability import (
    AtifConfig,
    AtofConfig,
    ComponentSpec,
    ObservabilityConfig,
    OtlpConfig,
)

config = plugin.PluginConfig(
    components=[
        ComponentSpec(
            ObservabilityConfig(
                atof=AtofConfig(
                    enabled=True,
                    output_directory="logs",
                    filename="events.jsonl",
                    mode="overwrite",
                ),
                atif=AtifConfig(
                    enabled=True,
                    output_directory="logs",
                    filename_template="trajectory-{session_id}.json",
                ),
                opentelemetry=OtlpConfig(
                    enabled=True,
                    endpoint="http://localhost:4318/v1/traces",
                    service_name="nemo-flow",
                    service_namespace="agent",
                    service_version="0.2.0",
                    instrumentation_scope="nemo-flow-observability",
                    resource_attributes={"deployment.environment": "dev"},
                ),
                openinference=OtlpConfig(
                    enabled=True,
                    endpoint="http://localhost:6006/v1/traces",
                    service_name="nemo-flow",
                    service_namespace="agent",
                    service_version="0.2.0",
                    instrumentation_scope="nemo-flow-openinference",
                    resource_attributes={"deployment.environment": "dev"},
                ),
            )
        )
    ]
)

report = plugin.validate(config)
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
    raise RuntimeError(report["diagnostics"])

await plugin.initialize(config)
try:
    with scope.scope("agent", ScopeType.Agent):
        pass
finally:
    plugin.clear()

Node.js

const plugin = require("nemo-flow-node/plugin");
const observability = require("nemo-flow-node/observability");

await plugin.initialize({
  version: 1,
  components: [
    observability.ComponentSpec({
      version: 1,
      atof: observability.atofConfig({
        enabled: true,
        output_directory: "logs",
        filename: "events.jsonl",
        mode: "overwrite",
      }),
      atif: observability.atifConfig({
        enabled: true,
        output_directory: "logs",
        filename_template: "trajectory-{session_id}.json",
      }),
      opentelemetry: observability.otlpConfig({
        enabled: true,
        endpoint: "http://localhost:4318/v1/traces",
        service_name: "nemo-flow",
        service_namespace: "agent",
        service_version: "0.2.0",
        instrumentation_scope: "nemo-flow-observability",
        resource_attributes: {
          "deployment.environment": "dev",
        },
      }),
      openinference: observability.otlpConfig({
        enabled: true,
        endpoint: "http://localhost:6006/v1/traces",
        service_name: "nemo-flow",
        service_namespace: "agent",
        service_version: "0.2.0",
        instrumentation_scope: "nemo-flow-openinference",
        resource_attributes: {
          "deployment.environment": "dev",
        },
      }),
    }),
  ],
});

try {
  // Run instrumented application work here.
} finally {
  plugin.clear();
}

Rust

use nemo_flow::observability::plugin_component::{
    AtifSectionConfig, AtofSectionConfig, ComponentSpec, ObservabilityConfig,
    OtlpSectionConfig,
};
use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig};

let component = ComponentSpec::new(ObservabilityConfig {
    atof: Some(AtofSectionConfig {
        enabled: true,
        output_directory: Some("logs".into()),
        filename: Some("events.jsonl".into()),
        mode: "overwrite".into(),
    }),
    atif: Some(AtifSectionConfig {
        enabled: true,
        output_directory: Some("logs".into()),
        filename_template: "trajectory-{session_id}.json".into(),
        ..AtifSectionConfig::default()
    }),
    opentelemetry: Some(OtlpSectionConfig {
        enabled: true,
        endpoint: Some("http://localhost:4318/v1/traces".into()),
        service_name: "nemo-flow".into(),
        service_namespace: Some("agent".into()),
        service_version: Some("0.2.0".into()),
        instrumentation_scope: Some("nemo-flow-observability".into()),
        resource_attributes: [("deployment.environment".into(), "dev".into())].into(),
        ..OtlpSectionConfig::default()
    }),
    openinference: Some(OtlpSectionConfig {
        enabled: true,
        endpoint: Some("http://localhost:6006/v1/traces".into()),
        service_name: "nemo-flow".into(),
        service_namespace: Some("agent".into()),
        service_version: Some("0.2.0".into()),
        instrumentation_scope: Some("nemo-flow-openinference".into()),
        resource_attributes: [("deployment.environment".into(), "dev".into())].into(),
        ..OtlpSectionConfig::default()
    }),
    ..ObservabilityConfig::default()
});

let config = PluginConfig {
    version: 1,
    components: vec![component.into()],
    policy: Default::default(),
};

let report = validate_plugin_config(&config);
assert!(!report.has_errors());

let active = initialize_plugins(config).await?;

Validation And Teardown#

Validate plugin configuration before activating it. The plugin reports unsupported transports, unsupported ATOF modes, unsafe ATIF filename templates, unknown fields according to policy, and enabled exporters that are unavailable in the current build or target.

Call plugin.clear() or clear_plugin_configuration() during teardown. Clearing the plugin config deregisters inferred subscribers, flushes file exporters, and shuts down owned OTLP subscribers.

Use manual subscriber/exporter APIs instead of the plugin when you need custom subscriber names, explicit per-run exporter objects, or direct control over the collection window.

Observability Configuration#

What It Installs#

plugins.toml Example#

Per-Language Plugin Configuration#

Validation And Teardown#

`plugins.toml` Example#