CLI

verdictan gateway run

verdictan gateway run starts a gateway in the current terminal. The gateway runs until you press Ctrl+C or send it SIGTERM. Use it for a first gateway, for a test, or inside a container. To run a gateway as a service on a host, use verdictan gateway install.

Every gateway that this command starts is a connected gateway. It gets its configuration from the control plane, and it accepts only requests that have a valid application token.

Before you start

The verdictan binary, or the verdictan/verdictan container image. See Install the gateway.
A gateway runtime token. Give it to the gateway in VERDICTAN_API_TOKEN. Do not give this token to applications.
An agent and a runtime registration for the gateway. The quickstart creates both and shows the IDs.
An application token for the requests that you send through the gateway. It must be different from the gateway runtime token.
The provider credential variables that the deployed configuration names, set in the same environment as the gateway.

Start a connected gateway

For your first gateway, copy the command that the quickstart generates. It contains the correct IDs for your organization.

To start a gateway yourself, set the control-plane variables and run the command:

export VERDICTAN_API_URL="https://api.verdictan.com"
export VERDICTAN_API_TOKEN="replace-with-gateway-runtime-token"

verdictan gateway run \
  --agent <agent-id> \
  --runtime-registration-id <runtime-registration-id> \
  --listen 127.0.0.1:41002

When the listener is open, the command prints gateway listening on http://127.0.0.1:41002 to standard error. Keep the terminal open.

At startup, the gateway does these steps:

It reads the runtime registration ID from --runtime-registration-id or VERDICTAN_RUNTIME_REGISTRATION_ID. If neither is set, it tries to register itself with the control plane. It stores the result in /tmp/verdictan/registration.json and uses that file on the next start.
It pulls the configuration that the control plane assigned to this gateway. If no configuration is assigned, the gateway waits for a deployment.
It opens the listener and serves requests.

The default listen address is 0.0.0.0:41002, which accepts connections on all interfaces. The examples on this page bind 127.0.0.1. Bind all interfaces only when a firewall, TLS, and application tokens protect the gateway.

Usage

verdictan gateway run [OPTIONS]

Options

Flag	Description
`--listen <host:port>`	Listen address. The default is `0.0.0.0:41002`.
`--agent <name\|id>`	Agent for this gateway. Without `--policy-config`, the CLI looks up an agent that has this ID or name. With `--policy-config`, it is the agent name that owns the config.
`--runtime-registration-id <id>`	Runtime registration ID that the control plane issued for this gateway.
`--policy-config <path>`	Local policy config to send to the control plane before the gateway starts. Repeat the flag to add overlay files. Each later file overlays the earlier files. Requires `--agent`. See Bind a local policy config to an agent.
`--config-name <name>`	Configuration name for `--policy-config`. The default is the file name, without the extension, of the first `--policy-config` file.
`--fail-mode <allow\|block>`	Response when the upstream provider cannot be reached. The default is `block`. See Limits and failure behavior.
`--max-concurrency <n>`	Maximum concurrent upstream requests before adaptive backoff starts. The default is `16`. A value of `0` becomes `1`.
`--upstream <url>`	Fixed upstream base URL, such as `https://api.openai.com`. Without it, the gateway uses the provider targets in its configuration.
`--upstream-api-key-header <name>`	Header that carries `VERDICTAN_UPSTREAM_API_KEY` to the fixed upstream. The default is `Authorization`.
`--upstream-api-key-prefix <prefix>`	Text before the upstream key value. The default is `Bearer` . Set an empty value to send the key without a prefix.

The global flags --region <region> and --lang <tag> also apply. The command does not accept an API token or an upstream key as a flag. Use the environment variables in the next table.

Variable	Use
`VERDICTAN_API_URL`	Control-plane base URL. The default is `https://api.verdictan.com`.
`VERDICTAN_API_TOKEN`	Gateway runtime token. The gateway uses it for registration, configuration pull, token validation, telemetry, and History writeback.
`VERDICTAN_RUNTIME_REGISTRATION_ID`	Runtime registration ID when `--runtime-registration-id` is not set.
`VERDICTAN_AGENT_ID`	Exact agent ID. When it is set, the CLI does not look up `--agent` on the control plane.
`VERDICTAN_AGENT_NAME`	Agent name or ID to look up when `--agent` is not set.
`VERDICTAN_GATEWAY_MAX_CONCURRENCY`	Concurrency limit when `--max-concurrency` is not set. The gateway ignores a value that is not a whole number.
`VERDICTAN_UPSTREAM_URL`	Fixed upstream base URL when `--upstream` is not set.
`VERDICTAN_UPSTREAM_API_KEY`	Credential for the fixed upstream.
`VERDICTAN_UPSTREAM_API_KEY_HEADER`, `VERDICTAN_UPSTREAM_API_KEY_PREFIX`	Header name and prefix when the matching flags are not set.

Bind a local policy config to an agent

--policy-config sends a local file to the control plane before the gateway starts. You must also pass --agent <name> and set VERDICTAN_API_TOKEN. The CLI then does these steps:

It lists agents with GET /v1/agents. If no agent has the name, it creates one with POST /v1/agents.
It reads each --policy-config file and joins the files into one YAML document with --- between them.
It looks for a configuration with the --config-name value with GET /v1/configurations. If there is none, it creates one for the agent with POST /v1/configurations.
It saves the YAML as a new version with deploy: true with POST /v1/configurations/<config-id>/versions.

warning

The current control-plane API does not serve the /v1/configurations routes in steps 3 and 4. The sync stops with an API error, and the gateway does not start. To use a local config, deploy it to the agent in Configurations in the Console. Then start the gateway without --policy-config.

With CLI 0.1.14 or later, use verdictan policy push to deploy through the agent before you start the gateway. See Policy Lifecycle.

The local file does not become the active configuration of a connected gateway. When the configuration pull succeeds, the gateway uses the configuration from the control plane. It uses the local file only when the pull fails. Without --policy-config, a connected gateway does not read /etc/verdictan/policy-config.yaml.

Examples

Connected gateway from a local policy config

First, deploy the config to the agent in Configurations in the Console. Deploy a version has the steps. Then start the gateway with that agent and its registration:

export VERDICTAN_API_TOKEN="replace-with-gateway-runtime-token"
export VERDICTAN_OPENAI_API_KEY="replace-with-provider-credential"

verdictan gateway run \
  --agent <agent-id> \
  --runtime-registration-id <runtime-registration-id> \
  --listen 127.0.0.1:41002

The gateway pulls the deployed configuration when it starts, and checks again about every 30 seconds. When a refresh applies configuration, it logs connected read model refresh applied updated config. The line includes config_version and config_sha256.

With gateway 0.1.14 or later, the next heartbeat reports the applied hash. Check Sync in Gateways. See Verify the deployment for each status and the required request tests.

Example config with two provider targets

This config has a primary OpenAI target and an Azure OpenAI fallback target. It passes verdictan config validate and verdictan policy lint --mode runtime.

policy-config.yaml

pack:
  name: multi-provider
  version: 0.1.0
  enabled: true

providers:
  targets:
    - id: openai-primary
      provider: openai
      model: your-openai-model
      base_url: https://api.openai.com
      secret_key_ref:
        env: VERDICTAN_OPENAI_API_KEY
    - id: azure-fallback
      provider: azure
      provider_type: azure-openai
      format: openai
      model: your-azure-openai-model
      base_url: https://replace-with-resource-name.openai.azure.com
      secret_key_ref:
        env: VERDICTAN_AZURE_OPENAI_API_KEY
      azure_api_version: your-supported-api-version
      azure_deployment: your-azure-deployment

  routing:
    strategy: ordered

policies:
  chain:
    - prompt-injection
    - pii-detector

Replace your-openai-model and your-azure-openai-model with model IDs that your provider accounts serve. Set both key variables in the gateway environment.

With ordered routing, the gateway sends the request to the first target. A timeout or an upstream status 408, 429, 500, 502, 503, or 504 sends the request to the next target. The gateway returns other upstream errors to the client. To stop fallback, keep one target or set providers.routing.allow_fallbacks: false. Provider Configuration documents the routing fields.

Run the gateway in a container

The image entrypoint is verdictan. Pass the credentials as environment variables, and publish the port only on the host loopback address:

export VERDICTAN_API_TOKEN="replace-with-gateway-runtime-token"
export VERDICTAN_OPENAI_API_KEY="replace-with-provider-credential"

docker run --rm -p 127.0.0.1:41002:41002 \
  -e VERDICTAN_API_URL="https://api.verdictan.com" \
  -e VERDICTAN_API_TOKEN \
  -e VERDICTAN_OPENAI_API_KEY \
  -e VERDICTAN_RUNTIME_REGISTRATION_ID="<runtime-registration-id>" \
  verdictan/verdictan:latest gateway run \
    --listen 0.0.0.0:41002 \
    --agent <agent-id>

Inside the container, the gateway listens on all container interfaces. The -p 127.0.0.1:41002:41002 option keeps the port off the host network.

Send traffic through the gateway

Check that the gateway is ready. Then send a request from a second terminal with an application token:

curl -s http://127.0.0.1:41002/readyz

export VERDICTAN_CLIENT_TOKEN="replace-with-application-token"

curl http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-openai-model",
    "messages": [
      {"role": "user", "content": "Explain quantum computing"}
    ]
  }'

A ready gateway returns "status": "ready" from /readyz. The chat request returns a model response, or a policy decision from the configuration.

The gateway has three probe endpoints. They do not send a request to a provider.

Endpoint	Response
`GET /healthz`	`{"status":"ok"}` when the process responds.
`GET /livez`	`{"status":"alive"}` when the process responds.
`GET /readyz`	`{"status":"ready", ...}` when the gateway is ready. HTTP `503` with `"status": "not_ready"` when a dependency is not available, for example the control-plane connection.

Limits and failure behavior

The policy chain runs on Chat Completions, Responses, Messages, and WebSocket frames. Embeddings, audio, moderation, and legacy completions get authentication, provider access, capability, and budget checks, but no policy controls.
With --fail-mode block, a Chat Completions or Responses request that cannot reach the upstream returns HTTP 503 with reason code proxy.upstream_unreachable. For Chat Completions, a timeout returns HTTP 504.
With --fail-mode allow, the same condition returns HTTP 200 with a placeholder reply and reason code proxy.degraded_allow. The client cannot tell this from a success by the status code. Use this mode only if your application checks the reason code.
Fallback can send the same request to more than one target. Each attempt can cost money or change data.
Without VERDICTAN_API_TOKEN, the gateway starts, logs no API token provided — gateway telemetry and history writeback disabled, and has no configuration. /readyz returns HTTP 503.
A connected gateway does not serve a baked-in local configuration. It uses the configuration that the control plane assigns.

Troubleshooting

Symptom	Cause	Fix
`--policy-config requires --agent <name> to bind the configuration to an agent.`	`--policy-config` without `--agent`.	Remove `--policy-config`. Deploy the config to the agent in Configurations.
`--agent requires an API token to sync with the control plane. Set VERDICTAN_API_TOKEN.`	`--policy-config` and `--agent` without a token.	Set `VERDICTAN_API_TOKEN`.
`--agent or VERDICTAN_AGENT_NAME requires an API token to resolve the agent.`	An agent lookup without a token.	Set `VERDICTAN_API_TOKEN`, or set `VERDICTAN_AGENT_ID` to the exact ID.
`agent '<name>' was not found on the control plane.`	No agent has that ID or name.	Copy the agent ID from the Console, or set `VERDICTAN_AGENT_ID`.
`agent name '<name>' matched multiple agents`	Two or more agents have that name.	Set `VERDICTAN_AGENT_ID` to the exact agent ID.
`invalid --fail-mode (expected allow\|block)`	Another value for `--fail-mode`.	Use `allow` or `block`.
`failed to bind gateway listener`	The port is in use, or the address is not on this host.	Stop the other process, or select a different `--listen` address.
`/readyz` returns `503` with `dependency.connected_control_plane_unavailable`	The gateway has no token, or it cannot reach or authenticate to the control plane.	Check `VERDICTAN_API_URL` and `VERDICTAN_API_TOKEN`, and read the gateway log.
`/readyz` returns `503` with `audit.wal_quarantine_non_empty`	The local event spool has quarantined audit events that block readiness.	See Quarantined audit events.
`/readyz` returns `503` with `audit.wal_full`, `audit.wal_unwritable`, or `audit.wal_corrupt_checkpoint`	The local event spool is at its size limit, the gateway cannot write to it, or its checkpoint is not valid.	Read the gateway log. For a full spool, make sure that the gateway can deliver events to the control plane. For an unwritable spool, make the spool directory writable.

Quarantined audit events

A connected gateway keeps its audit events in a local spool before it sends them to the control plane. The spool is in $VERDICTAN_DATA_DIR/event-retry/. Without VERDICTAN_DATA_DIR, the spool is in the verdictan directory of the system temporary directory, for example /tmp/verdictan/event-retry/.

When the control plane rejects a spooled event, or a spooled record is malformed, the gateway quarantines the record and continues with the next record. With gateway 0.1.26 or later, the gateway puts each quarantined record in one of two directories:

Directory	Records	Effect on `/readyz`
`event-retry/quarantine/earlier-run/`	Records that were in the spool when the gateway process started, for example the events of an earlier runtime registration	None. `/readyz` stays ready.
`event-retry/quarantine/`	Records that this gateway process wrote and could not deliver	HTTP `503` with `audit.wal_quarantine_non_empty` until you remove the files

A rejected record of the current process shows that the control plane does not accept the audit events of this gateway. Thus the gateway is not ready. These conditions still block readiness:

Quarantined records that this gateway process wrote, with audit.wal_quarantine_non_empty.
Quarantine files that CLI 0.1.25 or earlier wrote. These files are all in event-retry/quarantine/, and the gateway cannot find the process that wrote them. You must remove them.
A full spool, a spool that the gateway cannot write, or a checkpoint that is not valid.

To monitor the quarantine, read the audit_wal_quarantine_files and audit_wal_quarantine_earlier_run_files fields of a ready /readyz response, or the gauges with the same names on /metrics. When the gateway starts with files in earlier-run/, it writes a WARN log line.

To examine the quarantined records, list the files:

ls "$VERDICTAN_DATA_DIR/event-retry/quarantine" "$VERDICTAN_DATA_DIR/event-retry/quarantine/earlier-run"

A permanent-*.json file is for a rejected record. A corrupt-*.json file is for a malformed record. The gateway does not send these records again.

When you no longer need the records, remove the files. The second command makes the gateway ready again when quarantine files block it:

rm -f "$VERDICTAN_DATA_DIR/event-retry/quarantine/earlier-run/"*.json
rm -f "$VERDICTAN_DATA_DIR/event-retry/quarantine/"*.json

Next steps

Quickstart in Docker: start a container gateway and send the first governed request.
verdictan gateway: check, install, reload, and revert gateways.
Policy Lifecycle: publish a policy config to a gateway and roll it out.
Declarative Config Reference: add provider targets and policy controls to a config.

Before you start​

Start a connected gateway​

Usage​

Options​

Bind a local policy config to an agent​

Examples​

Connected gateway from a local policy config​

Example config with two provider targets​

Run the gateway in a container​

Send traffic through the gateway​

Limits and failure behavior​

Troubleshooting​

Quarantined audit events​

Next steps​