Safety and limits¶

Privacy Guard bounds request text, engine output, detections, regex execution, transport output, and concurrency. Limits are part of the architecture, not optional tuning defaults.

Package-wide values live in constants.py.

Validation ownership¶

Each layer validates facts it can establish:

Keep request-wide policy decisions in the processor and algorithm-specific behavior in the concrete engine. The service adapts transport but does not reimplement either.

Text, processing, and transport limits¶

The gRPC receive allowance is 1 MiB larger than the request body limit for the protobuf envelope. The service still enforces the advertised 4 MiB body bound.

One monotonic Timeout is shared by all stages and final result validation. RegexEngine passes the remaining duration into every backend search. Third-party calls that accept a timeout should receive the same remaining duration; calls that cannot be interrupted continue to occupy a service worker until they exit.

Operators configure the internal bound with --timeout-seconds or the programmatic PrivacyGuardServer(timeout_seconds=...) argument. OpenShell's outer middleware timeout must include additional headroom for worker queueing and configuration or engine preparation beyond this internal bound.

Diagnostic and result limits¶

The diagnostic-string bound applies to stage names, entity names, metadata keys and values, model-profile names, and other audit-safe identifiers built from the shared domain field type. These values must be non-empty printable Unicode without control or bidirectional formatting characters. Untrusted request IDs use the same content and size rules for logging; an invalid request ID is represented by a constant placeholder and does not change the evaluation result. The human-readable log message quotes request IDs and escapes ASCII spaces so their text cannot introduce another key/value token; structured log records retain the exact validated ID. Regex entity and supplied rule names have a stricter ASCII grammar and limit described below.

The processor aggregates occurrences before the service applies protobuf limits. If a safe result cannot be represented, the service returns a limit deny with no partial findings.

Regex and catalog limits¶

Entity and rule names use [A-Za-z_][A-Za-z0-9_-]*. Rule names are optional; their deterministic derived diagnostic identities are not serialized back into configuration.

The per-evaluation config Struct is limited to 64 KiB. A catalog may satisfy Privacy Guard's engine limits yet remain too large for the current upstream protocol. Internal caching does not raise that transport ceiling.

NER configuration and execution limits¶

NER labels must be non-empty printable Unicode without leading or trailing whitespace and must be unique after Unicode case folding. Model identity, endpoint, and execution chunking are deployment resources rather than policy fields.

The endpoint facade bounds response bytes before JSON decoding and performs one request without retry or fallback. The local facade processes every supported input in overlapping code-point windows and rebases returned offsets. Its overlap must be smaller than its chunk length. Operators should choose overlap using the longest entity spans expected in the deployment corpus.

Both facades return only normalized label, start, end, and score values. The engine rejects unknown labels, invalid or out-of-bounds spans, non-finite or out-of-range scores, and excessive result cardinality. Request text, returned matched text, raw response bodies, endpoint URLs, credentials, and collaborator exception messages are not logged or included in errors.

Regex execution safety¶

RegexEngine validates and compiles the complete catalog before accepting configuration. It:

• rejects invalid and empty pattern strings
• rejects patterns that match empty input
• rejects user-defined named groups and inline flags
• verifies its private trailing marker on every match
• treats a context-dependent zero-width match as an atomic configuration failure
• evaluates patterns independently to retain overlaps
• uses the timeout-capable third-party regex backend
• caps detections per stage
• projects exact UTF-8 replacement size before rendering

No timeout, limit, or pattern failure returns partial stage detections or mutated text.

When a limit is exceeded¶

The processor returns a successful deny with privacy_guard_limit_exceeded and no partial findings or replacement when:

• the shared timeout expires
• an engine exceeds its detection or output limit
• intermediate text exceeds the UTF-8 byte limit
• aggregate request detections exceed the limit

The service returns the same bounded deny when:

• findings exceed the protobuf group or encoded-size limit
• replacement text cannot be encoded within the body limit
• a deny reason code is not safely representable

The processor emits a content-safe timeout or resource limit kind, and the service adapter emits resource for representation limits. Neither log includes request content or collaborator error text. The deny reason directs users to check that log, then reduce the request or replacement size, simplify the configured stages and rules, or increase --timeout-seconds or PrivacyGuardServer(timeout_seconds=...) up to 30 seconds. When increasing the processing timeout, users must give OpenShell's middleware timeout additional headroom for worker queueing and configuration or engine preparation. These options reduce bounded output pressure or provide appropriate time without weakening the fail-closed behavior.

Malformed input and engine contract or execution failures instead abort the RPC with a cataloged error. OpenShell then applies the middleware registration's failure behavior.

The distinction is deliberate:

• an exhausted declared processing bound is a closed, successful deny
• invalid user input is an INVALID_ARGUMENT RPC failure
• an engine or middleware defect is an INTERNAL RPC failure

Error catalog¶

Production failures use PrivacyGuardError and a stable ErrorCode. Each error spec defines:

• invalid-input or internal classification
• responsible component
• failed operation
• content-safe summary
• remediation hint

Successful policy and safety-limit denials use stable reason codes plus content-safe recovery guidance; they do not add domain fields solely for diagnostic logging. CLI validation failures name the rejected option and state the accepted form or next action. Cataloged server-startup failures print one content-safe message and exit nonzero rather than exposing a traceback. Internal engine and third-party failures remain concise because the owning boundary translates them before they reach users.

Caught extension and third-party exceptions are translated at their trust boundary. Raw exception messages and chains are not exposed. Configuration errors do not include rejected pattern strings, paths, credentials, provider endpoints, or request text.

Use a new catalog entry for a genuinely new externally observable failure. Do not return raw Pydantic, regex-backend, engine, or protobuf exceptions.

Logging¶

Default operational logs include:

• request ID
• evaluation duration
• allow or deny action
• aggregate finding count
• stable error code
• stage source and invocation strategy at debug level

They do not include:

• request or replacement text
• matched or surrounding text
• offsets
• patterns or catalog contents
• headers, targets, or credentials
• model endpoints
• arbitrary exception text

Sensitive domain fields use repr=False to reduce accidental exposure. This does not replace safe exception handling.

--debug enables content-safe processing diagnostics. --debug-log-content deliberately logs complete input and processed text and emits a warning; use it only in a controlled development environment.

Extension requirements¶

Custom engines should:

• use the base constructor and public run() wrapper
• return the exact TextProcessingResult contract
• return stable, declared entity identifiers rather than values derived from request text
• keep request data local to _run()
• keep initialized state immutable and make resources concurrency-safe
• pass the shared remaining timeout to delegated APIs when they support it
• fail the stage on native partial failure
• use TextProcessingResult.from_detections() to bound lazy detection streams
• add manual bounds only when a low-level collaborator allocates proportional output before Privacy Guard can consume it
• translate expected collaborator failures to the content-safe engine exception hierarchy
• avoid logging input or caught exception text

Privacy Guard keeps framework-controlled fields and identifiers that pass its shared validation content-safe for findings and operational logs. Operator-installed Python engine code is trusted deployment code: it can access request text and must honor the declared identifier and logging contract. The framework cannot make arbitrary trusted Python code non-exfiltrating, so operators must review and install custom engines with the same care as other code in the middleware process.

Do not add retries, fallback providers, persistent request artifacts, or extra validation without a concrete failure mode and a clear owning layer.

State and retention¶

Cross-request entity memory is not implemented. Privacy Guard retains one active validated policy and its immutable configured engine state, but never request text, detections, or replacement mappings. The active processor is deliberate process state, not an LRU entry: it remains until successful replacement or shutdown.

During an update, the fully prepared candidate may coexist with the active processor. Evaluations that already captured the prior processor may keep it alive until they finish. A failed candidate leaves active state unchanged.

Regex's supporting compiled-rule cache remains bounded:

The compiled-rule allowance accounts for retained backend state that is much larger than short pattern text. One entry that exceeds the budget remains valid and usable for the current operation but is not retained there. This does not invalidate an active or candidate processor holding those rules. The active policy is not rejected by cache admission; policy and schema limits bound its shape, while custom-engine prepared memory remains an operator responsibility. Regex cache skip and eviction records contain only the cache name and weights, plus eviction counts.

Changing a limit¶

A limit change may affect policy schema, processor behavior, engine contracts, protobuf serialization, tests, examples, and the OpenShell middleware registration.

Before changing a limit:

1. identify every layer that enforces or advertises it
2. update exact-boundary tests
3. check the failure result immediately above the boundary
4. run the complete project validation
5. benchmark changes that affect regex compilation or request processing
6. coordinate upstream when the copied OpenShell protocol is involved

Back to the architecture overview