Observability Configuration

Metrics tell you whether traffic flows. Request events tell you why one request got its result. This page shows where the gateway exposes each, and how to scrape the metrics with Prometheus. Start with the local gateway from the quickstart.

Syntax

The gateway serves Prometheus metrics on its listener. No configuration is necessary:

curl --fail --silent --show-error http://127.0.0.1:41002/metrics

The output is Prometheus text with metric names that start with verdictan_gateway_. Send one request through the gateway and read the metrics again: the request counter goes up. A healthy metrics endpoint does not prove that model requests succeed.

Where to look

Question	Look in
Is traffic flowing, and with what status?	GET /metrics on the gateway
How fast and healthy is each provider?	GET /verdictan/providers/metrics on the gateway
Why did this request get its result?	verdictan events tail, or GET /v1/events with since
What did the request and response contain?	History, when capture is on
Who changed a setting?	Trail

Metric families

/metrics exposes these families:

Metric	Labels	Meaning
verdictan_gateway_requests_total	method, status, provider	Request count
verdictan_gateway_policy_eval_duration_seconds		Policy chain latency (histogram)
verdictan_gateway_upstream_errors_total	provider, error_type	Upstream error count
verdictan_gateway_cache_hits_total, verdictan_gateway_cache_misses_total		Response cache hits and misses
verdictan_gateway_events_dropped_total		Dropped events
verdictan_gateway_active_connections		Active connections (gauge)
verdictan_gateway_usage_authorization_control_plane_total		Usage authorizations with the control plane
verdictan_gateway_relay_requests_total, verdictan_gateway_relay_duration_seconds		Relay request count and latency

GET /verdictan/providers/metrics returns a JSON snapshot for each provider: the sample count, the latest health sample, and percentiles of time to first token (TTFT) and throughput. It needs the gateway runtime token.

OpenTelemetry export

The gateway also exports traces, metrics, and logs over OTLP when you set the standard OpenTelemetry variables on its process, such as OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_PROTOCOL (grpc by default, or http/protobuf). The OTLP metrics include instruments that /metrics does not have, such as verdictan_gateway_provider_attempts_total, verdictan_gateway_provider_first_token_seconds, and verdictan_gateway_provider_tokens_total. Set OTEL_SDK_DISABLED=true to turn all export off.

Examples

Read metrics from a local gateway

For provider latency and throughput, read the JSON snapshot:

curl --fail --silent --show-error \
  http://127.0.0.1:41002/verdictan/providers/metrics \
  -H "Authorization: Bearer $VERDICTAN_API_TOKEN"

If you changed the listener address, use your address.

View traffic in Grafana

1. Configure Prometheus to scrape /metrics on the gateway. Use an address that Prometheus can reach. If Prometheus runs on another machine or in a container, 127.0.0.1 points to Prometheus itself.

2. In Grafana, open Explore with the Prometheus data source.

3. Query up for the scrape job. A value of 1 means that the scrape works.

4. Send a model request through Verdictan, then run:

   sum by (status) (increase(verdictan_gateway_requests_total[5m]))

The result shows traffic with the HTTP status of your request. Wait at least two scrape intervals so that increase has data. To investigate a failure, use the request ID in runtime events.

Declare Langfuse and Datadog sinks

The gateway accepts a callbacks section, but no callback type receives request events. Do not build dashboards or alerts on callbacks. The fields are listed here only so that existing files keep loading.

Inside policy-config.yaml

callbacks:
  - type: langfuse
    host: https://cloud.langfuse.com
    public_key_env: VERDICTAN_LANGFUSE_PUBLIC_KEY
    secret_key_env: VERDICTAN_LANGFUSE_SECRET_KEY
  - type: datadog
    secret_key_ref:
      env: VERDICTAN_DATADOG_API_KEY
    site: datadoghq.com
    tags:
      - env:prod

Callback entry fields

Field	Type	Default	Description
type	string		langfuse, datadog, prometheus, helicone, or braintrust. Required.
host	string	https://cloud.langfuse.com	Langfuse host
public_key_env	string		Variable with the Langfuse public key
secret_key_env	string		Variable with the Langfuse secret key
secret_key_ref	object		Credential reference with env, for datadog, helicone, and braintrust
site	string	datadoghq.com	Datadog site
service_name	string	verdictan-proxy	Datadog service name
tags	array of strings		Datadog tags
project_name	string	default	Braintrust project name

prometheus has no fields.

Limits and failure behavior

Metrics

• /metrics has no route-labeled latency, no policy verdict counts, no fallback counts, and no dedicated size or rate limit rejection metrics. Use verdictan events or the Events API for results and rejection causes.
• Build dashboards only on metric names and labels that appear in your live scrape. Watch scrape health with the up series.

Callbacks

• No callback type delivers request events. The section loads, but the sinks get no data.
• The gateway accepts at most 32 sinks.
• An unknown type, more than 32 sinks, or a SIEM type such as splunk, elastic, generic_https_json, splunk_hec, or datadog_logs silently turns off all callback sinks. Configure SIEM delivery in AI Usage SIEM Streaming.
• A callback secret_key_ref accepts only env.

Next steps

• verdictan events: the result of one request.
• Runtime Configuration: turn on History capture.
• Debug AI Requests: trace one request through events and Trail.