AI Usage SIEM Streaming

Stream a record of every AI interaction through your gateway into Splunk, Elasticsearch, Datadog, or your own HTTPS collector. Your security team can then investigate policy decisions and data-leakage findings next to the rest of their security events.

A destination can also receive Trail events, the audit log of the control plane. Verdictan keeps Trail events for 90 days. Forward them to a SIEM to keep them for longer.

Connected gateways capture AI Usage Records automatically. Add a destination in the Console to forward these records. Then send a test request and find its record in your SIEM.

Before you start

• Verdictan CLI 0.1.16 or later.
• A connected gateway with its bound gateway runtime token in VERDICTAN_API_TOKEN. A personal API token cannot send these records.
• The ai_usage_stream:write permission to create a destination. Use ai_usage_stream:read to examine delivery health. Use ai_usage_stream:admin to delete a destination.
• A destination that accepts HTTPS ingestion from the Verdictan control plane.

Capture

Capture is always on for connected gateways. You cannot disable capture or set a body capture limit.

• Capture includes chat completions, messages, responses, embeddings, audio, completions, moderations, WebSocket sessions, and MCP requests. Blocked requests and upstream errors also produce records.
• Each record contains the full request and response bodies. Binary bodies use lossy UTF-8 text. The SHA-256 hashes cover the raw bytes.
• Before dispatch, the gateway checks its write-ahead log (WAL). If the WAL cannot accept records, the gateway returns 503 with audit.wal_unavailable.
• If capture fails for a buffered response, the gateway returns 503 with ai_usage.capture_unavailable.
• If capture fails after streaming starts, the gateway logs the error. It cannot replace output that the client already received.
• The gateway sends records to POST /v1/machine/ai-usage-stream/ingest.
• The control plane stores records even when no destination exists. It has no retention period for these records.

Large bodies increase memory and disk use. The WAL has a total capacity of 1 GiB by default. Network proxies and destinations can have separate request size limits.

A gateway without a control-plane connection logs a startup warning. It serves traffic without capture.

The policy config has no capture settings. policy lint and the control plane reject the ai_usage_streaming section. A running gateway ignores this legacy section and logs a warning.

Set the destination and its redaction mode in the Console.

Configure a destination

1. In the Console, open Settings → SIEM Streaming and select Add Destination.
2. Enter a Name, and pick a Destination type: splunk, elastic, datadog, or generic_https_json.
3. Enter the destination's HTTPS ingestion URL in Endpoint URL, and its credential. The control plane encrypts the credential before storing it.
4. Under Record Streams, keep AI usage records selected. To also send Trail events, select Trail events. See Forward Trail events.
5. Keep Redacted Mode (Default) for the first test; request and response bodies are replaced with [REDACTED]. Read Redaction modes before you choose raw.
6. Leave Event Filter empty until delivery works. See Event filters.
7. Select Create Destination.

The SIEM destination form with synthetic data and redacted mode.

The API checks the type, URL, credential, redaction mode, and filter. Raw mode without the acknowledgment fails with destination.raw_requires_acknowledgement. A URL that breaks the outbound URL policy fails with destination.ssrf_denied.

Verify

1. Send one clearly labeled request through the gateway, such as Reply with Hello. SIEM delivery check., and note its request ID and time.
2. Open the destination in the Console. For AI usage records, the Delivery Health card shows Pending, Sent, Retrying, Quarantined, Last Delivery, Delivery Lag, and the last error. For Trail events, it shows Delivered Events, Last Event Time, Last Delivery, Failed Batches, Next Attempt, Last HTTP Status, and the last error.
3. From the API, GET /v1/ai-usage-stream/delivery-health returns, for each destination, pending_count, sent_count, retryable_count, quarantined_count, last_sent_at, last_error, and lag_seconds.
4. For attempt details, GET /v1/ai-usage-stream/destinations/{id}/delivery-status returns the 50 most recent attempts with phase, http_status, error_code, error_summary, and delivery_duration_ms.
5. In the SIEM, search for the record's record_id, and check that redaction_mode_applied and the body fields match the destination's mode.

Supported destinations

Type	Target system	Credential header	Content type
splunk	Splunk HTTP Event Collector	Authorization: Splunk <token>	application/json
elastic	Elasticsearch bulk API	Authorization: ApiKey <key>	application/x-ndjson
datadog	Datadog logs intake	DD-API-KEY: <key>	application/json
generic_https_json	Any HTTPS JSON collector	Authorization: Bearer <token>	application/json

Any other type is rejected, including syslog, s3, splunk_hec, and datadog_logs.

Every delivery is an HTTPS POST with an Idempotency-Key: <record_id>:<destination_id> header. The body depends on the type:

• splunk: one NDJSON line per record, {"event": <record>, "sourcetype": "verdictan:ai_usage", "source": "verdictan-api"}.
• elastic: bulk NDJSON, an index action with _index: verdictan-ai-usage and _id: <record_id>, then the record.
• datadog: a JSON array of {"ddsource": "verdictan", "service": "verdictan-ai-usage", "message": <record>}.
• generic_https_json: one object, {"source": "verdictan", "schema_version": "1", "stream": "ai_usage", "records": [<record>]}.

The endpoint must use https:// with no user information in the URL. The control plane checks it against its outbound URL policy when you save the destination and again before each delivery, and rejects private, loopback, link-local, and other restricted addresses.

Destination endpoints

Type	Endpoint URL example	Credential
splunk	https://splunk.example.com:8088/services/collector/event	An HTTP Event Collector token
elastic	https://elastic.example.com:9200/_bulk	An API key that can write to the verdictan-ai-usage index
datadog	https://http-intake.logs.datadoghq.com/api/v2/logs	A Datadog API key for the site that owns the intake URL
generic_https_json	Your collector's HTTPS ingestion URL	A Bearer token your collector accepts

Redaction modes

Each destination has one mode, which decides what request and response content leaves Verdictan:

Mode	What the destination receives
redacted (default)	request_body and response_body are [REDACTED], and redaction_mode_applied is redacted. Detector findings and SHA-256 hashes stay.
raw	The full captured bodies. The acknowledgment time is stored in raw_acknowledged_at.

Redacted mode is enough for most investigations: each finding keeps its category, entity type, confidence, matched policies, and masked evidence locations, so an investigator can see what was detected and what the gateway did.

warning

Raw mode sends unredacted prompts and responses to an external system. Confirm the destination's access, retention, and data-handling controls before you save it.

Event filters

An optional event_filter decides which records a destination receives. Every record is stored regardless; the filter only controls whether a delivery is created for that destination.

• An empty, missing, null, or {} filter matches everything.
• With several fields, all must match.
• An array matches when the record's value equals one of its entries.
• Unknown fields, empty arrays, empty strings, and http_status values outside 100–599 are rejected.

Field	Match
request_family, transport, provider, model, upstream_host, outcome, gateway_id, agent_id, actor_type, currency, source	Exact match against a string array
http_status	Exact match against an integer array
streamed, scanned, secrets_detected, potential_data_leakage	Exact Boolean match

Only blocked or flagged OpenAI records with a data-leakage finding:

{
  "provider": ["openai"],
  "outcome": ["block", "flag"],
  "potential_data_leakage": true
}

Only upstream throttling and server errors:

{
  "http_status": [429, 500, 502, 503, 504]
}

AI Usage Record schema

Each record carries identifiers, content metadata, governance findings, and usage. Use request_id to find an interaction, record_id to deduplicate deliveries, and trail_event_id to join Trail evidence.

Expand the complete record field reference

Envelope and identity

Field	Description
schema_version	The record schema version
record_id	The unique record identifier (UUID)
org_id	The organization identifier
sequence	The monotonic sequence number in the organization
captured_at	The server-owned capture time
source	The capture source. The value is gateway
gateway_id	The gateway that processed the interaction
agent_id	The agent identifier
subject_token_id	The client API token that the request used
actor_id, actor_type	The actor identity and type
session_id	The session identifier
correlation_id	The cross-system correlation ID
request_id	The unique request identifier

Interaction and content

Field	Description
request_family	The request family, such as Chat, Responses, Messages, WebSocket, or MCP
transport	The transport: buffered, sse, websocket, or mcp
provider, model	The provider and model
upstream_host	The upstream host without credentials
streamed	true when the gateway streamed the response
started_at, completed_at	The start and completion times
latency_ms	The total latency in milliseconds
outcome	The governance outcome: allow, block, flag, or error
http_status	The upstream HTTP status code
request_body, response_body	The captured content. In redacted mode the value is [REDACTED]
request_bytes, response_bytes	The content byte counts
request_sha256_raw, response_sha256_raw	The SHA-256 hashes of the raw content
request_sha256_redacted, response_sha256_redacted	The SHA-256 hashes of the redacted content
request_truncated, response_truncated	Always false. The gateway captures the full content

Governance and data leakage

Field	Description
matched_policies	The policy controls that matched the interaction
decisions	The applied governance decisions
applied_mutations, applied_redactions	The changes that the gateway applied to the request or response
output_stage_evaluations	The output-stage evaluation results
human_oversight, escalation, flagged_review	The oversight references
detectors[]	The detector findings. Each entry has detector, category, entity_type, match_count, confidence, matched_policy_ids, action, evidence_locations, and sample_hash
scanned	true when the detector check completed
secrets_detected	true when a scanner found a credential or secret pattern
attachments[]	The file metadata: type, size, sha256, and scanned
potential_data_leakage	true when the gateway detected possible data leakage
redaction_mode_applied	redacted when the control plane removed the bodies before delivery

The evidence_locations value contains only offsets and hashes. The sample_hash value is a non-reversible hash of the matched content. An empty detectors[] with scanned: true shows that the check found no reportable match in its configured limits.

Financial and integrity

Field	Description
prompt_tokens, completion_tokens, cached_input_tokens, total_tokens	The token counts
cost, currency	The interaction cost and currency
wallet_transaction_id, budget_id, reservation_request_id, spend_log_id	The financial references
canonical_sha256	The RFC 8785 canonical hash of the record
trail_event_id	The Trail event that correlates with the record

Limits

• Delivery is at-least-once. Failed deliveries retry with exponential backoff; after five failed attempts the delivery is quarantined, and later records keep flowing.
• Records are sent in order per destination, but a retried record can arrive after a newer one. Deduplicate on record_id and canonical_sha256.
• The record and its deliveries are stored in one transaction. No delivery is created for a destination whose filter doesn't match.
• Records have no size limit. A destination that limits event size, such as Splunk HEC or Datadog logs, can reject a large record. The delivery then retries and is quarantined.
• The destination credential never appears in an API response.

Forward Trail events

The streams field of a destination selects the records that it receives:

streams	The destination receives
["ai_usage"] (default)	AI Usage Records
["trail"]	Trail events
["ai_usage", "trail"]	Both

A destination with the trail stream needs the org:trail:admin permission in addition to the AI usage stream permissions. You need it to create, change, or delete such a destination.

In the Console, select Trail events under Record Streams when you create or edit the destination. From the API, set streams:

curl -X POST "$VERDICTAN_API_URL/v1/ai-usage-stream/destinations" \
  -H "Authorization: Bearer $VERDICTAN_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "splunk",
    "name": "Splunk audit",
    "endpoint_url": "https://splunk.example.com:8088/services/collector/event",
    "secret": "<HEC token>",
    "streams": ["trail"]
  }'

To add Trail events to an existing destination, send PUT /v1/ai-usage-stream/destinations/{id} with "streams": ["ai_usage", "trail"].

How Trail forwarding works

• A new Trail destination starts at the oldest retained Trail event. It receives all the retained events, and then each new event within a few seconds.
• Verdictan sends Trail events in sequence order, in batches of up to 100 events. The Idempotency-Key of a batch is trail:<destination_id>:<first_sequence>-<last_sequence>.
• When a batch fails, the destination keeps its position. Verdictan sends the same batch again after a backoff that increases from 5 seconds to 1 hour. Later events wait for the failed batch.
• Verdictan deletes Trail events after 90 days. If a destination fails for longer, the events that expire are not forwarded.
• The event_filter applies only to AI Usage Records. A Trail destination receives all Trail events.
• Changing the endpoint URL does not send the earlier events again. To send all retained events to a new system, create a new destination.

Trail record format

Each Trail record has the fields of GET /v1/trail/events/{id}, and these fields:

Field	Value
record_id	The event_id of the event
canonical_sha256	The record_hash of the event
stream	trail

Use sequence_number, previous_hash, and record_hash to check the hash chain in your SIEM. Deduplicate on record_id.

The redaction mode of the destination controls personal data:

Mode	Trail record
redacted (default)	actor_email, source_ip, peer_ip, and user_agent are null, and redaction_mode_applied is redacted. actor_id stays.
raw	All fields.

The wire format is the same as for AI Usage Records, with Trail labels:

Type	Trail label
splunk	"sourcetype": "verdictan:trail"
elastic	_index: verdictan-trail. The API key must be able to write to this index.
datadog	"service": "verdictan-trail"
generic_https_json	"stream": "trail"

Check Trail delivery

GET /v1/ai-usage-stream/delivery-health returns a trail object for each Trail destination:

Field	Meaning
last_sequence	The last Trail sequence number that the destination acknowledged
last_event_time	The event time of that event
last_delivered_at	The time of the last successful batch
delivered_count	The Trail events that the destination acknowledged
attempt_count	The consecutive failed batches
next_attempt_at	The earliest time of the next batch
last_error, last_http_status	The result of the last failed batch

Compare last_sequence with the newest sequence_number in Trail to see how far the destination is behind.

Next steps

• Trail and Audit Evidence: join a record to its Trail event with trail_event_id.
• Reviewing Alerts and Evidence: investigate the flagged and blocked interactions your SIEM surfaces.