Provider Control Features

This page covers the provider controls that change how the gateway picks, logs, and retries traffic, beyond the target fields in Provider Configuration:

• model_groups give clients a stable model name while you change the targets behind it.
• logging keeps message bodies and keys out of traces and events.
• max_context_tokens on a target compresses requests that are too long for it.
• zero_completion_insurance retries an empty completion on the next target.
• The top-level health_monitor section probes provider endpoints in the background.

Syntax

For a working target with id: openai-primary, add a group and logging next to targets in the same providers section:

Inside providers

model_groups:
  - name: fast
    aliases: [default-fast]
    targets: [openai-primary]
logging:
  redact_message_bodies: true

Send a request with "model": "fast". The runtime event shows that openai-primary served it.

Body redaction changes traces and event payloads only. The upstream model still receives the full content.

Fields

Model group fields

Field	Type	Description
name	string	Name that a request model can select. Required.
targets	array of strings	Target IDs of the group, in order. Each must exist in providers.targets. Required.
aliases	array of strings	Other model names for the group
description	string	Description
fallback_group	string	Group whose targets come next. Must exist and must not create a cycle.

Logging fields

Field	Type	Default	Description
redact_message_bodies	boolean	false	Writes [REDACTED] instead of message content in traces and event payloads
redact_api_keys	boolean	true	Writes [REDACTED] instead of API keys in traces and event payloads

Context compression field

Field	Type	Description
targets[].max_context_tokens	integer	Above this size, the gateway compresses the request from the middle before it sends it to the target.

Zero-completion insurance fields

Field	Type	Default	Description
enabled	boolean	true	Sends an empty completion to the next eligible target

Health monitor fields

Field	Type	Default	Description
providers[].name	string		Provider ID that the probe reports. Required.
providers[].endpoint	string		URL that the probe calls with a plain GET. Required.
providers[].interval_seconds	integer	30	Validated, but not used by the runtime
providers[].timeout_ms	integer	5000	Validated, but not used by the runtime
unhealthy_threshold	integer	3	Validated (must be above 0), but not used by the runtime
alert_callback_urls	array of strings		Validated, but not used by the runtime

Examples

Complete example

Use two models that your account serves, and export both keys in the gateway terminal:

policy-config.yaml

pack:
  name: advanced-provider-controls
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai-prod
    provider: openai
    provider_type: openai
    format: openai
    model: your-openai-model
    max_context_tokens: 128000
    secret_key_ref:
      env: VERDICTAN_OPENAI_API_KEY
  - id: openai-secondary
    provider: openai
    provider_type: openai
    format: openai
    model: your-secondary-openai-model
    secret_key_ref:
      env: VERDICTAN_OPENAI_SECONDARY_API_KEY
  model_groups:
  - name: standard
    targets:
    - openai-prod
    - openai-secondary
  logging:
    redact_message_bodies: true
  zero_completion_insurance:
    enabled: true
policies:
  chain:
  - audit-logger
policy:
  audit-logger: {}

Health monitor

Inside policy-config.yaml

health_monitor:
  providers:
    - name: openai-primary
      endpoint: https://health.example.com/ready

Limits and failure behavior

Model groups

• When the request model matches a group, the group targets go first in the candidate list, followed by the targets of each fallback_group.
• The provider attempt rules decide how the gateway moves through the list.
• An unknown target ID, an unknown fallback_group, or a cycle of fallback groups is a load error.
• A group name or alias that is also the name of a pipeline or of the auto provider is a load error.

Context compression

Compression happens only when the selected target sets max_context_tokens and the request is longer.

Zero-completion insurance

The check is on by default and applies only to buffered POST /v1/chat/completions responses. Turn it off if a second provider call is not approved.

Health monitor

• The gateway starts one probe task for each entry. VERDICTAN_HEALTH_PROBE_INTERVAL_SECS sets the interval: default 30 seconds, from 10 to 300.
• The results are not available on any public endpoint. To see provider health in GET /verdictan/providers/metrics, use a target health_probe instead.

Next steps

• Provider Configuration: targets, route strategies, retries, and pipelines.
• Cloud Provider Configuration: cloud-specific target fields.