Skip to main content

SDKs

Python Integration

Point the OpenAI Python client at your Verdictan gateway, and your model calls go through your policies. Your script only holds a Verdictan application token; the provider key stays on the gateway.

Before you start​

  • A running gateway from the quickstart, with its application token and a configured model.
  • Python 3.11 or newer.

Create a Python environment​

In your project directory, create and activate a virtual environment, then install the client:

python -m venv .venv
source .venv/bin/activate
python -m pip install openai

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1. If your system calls it python3, use that for the first command.

Configure the client​

Set three variables in the same terminal:

export VERDICTAN_GATEWAY_URL="http://127.0.0.1:41002/v1"
export VERDICTAN_CLIENT_TOKEN="<application-token>"
export VERDICTAN_MODEL="<configured-model>"

If the gateway runs on another host, use its base URL instead, still ending in /v1.

Send a request​

chat.py
import os
from openai import APIStatusError, OpenAI

client = OpenAI(
    base_url=os.environ["VERDICTAN_GATEWAY_URL"],
    api_key=os.environ["VERDICTAN_CLIENT_TOKEN"],
    max_retries=0,
    timeout=60.0,
)

try:
    raw = client.chat.completions.with_raw_response.create(
        model=os.environ["VERDICTAN_MODEL"],
        messages=[{"role": "user", "content": "Reply with Hello."}],
    )
    print("Request ID:", raw.headers.get("x-request-id"))
    if raw.headers.get("x-verdictan-verdict") == "ESCALATE":
        print("Waiting for human review. Do not resend yet.")
    else:
        response = raw.parse()
        answer = response.choices[0].message.content if response.choices else None
        if not answer:
            raise RuntimeError("No text answer. Inspect the request in Verdictan.")
        print(answer)
except APIStatusError as error:
    print("Request failed:", error.status_code, error.code or "unknown_error")
    raise SystemExit(1) from None
python chat.py

You see a request ID and Hello. with_raw_response gives you the headers, which carry the request ID and the gateway's verdict.

The same request with AsyncOpenAI
chat_async.py
import asyncio
import os
from openai import AsyncOpenAI

async def main():
    async with AsyncOpenAI(
        base_url=os.environ["VERDICTAN_GATEWAY_URL"],
        api_key=os.environ["VERDICTAN_CLIENT_TOKEN"],
        max_retries=0,
        timeout=60.0,
    ) as client:
        raw = await client.chat.completions.with_raw_response.create(
            model=os.environ["VERDICTAN_MODEL"],
            messages=[{"role": "user", "content": "Reply with Hello."}],
        )
        print("Request ID:", raw.headers.get("x-request-id"))
        if raw.headers.get("x-verdictan-verdict") == "ESCALATE":
            print("Waiting for human review.")
            return
        response = raw.parse()
        answer = response.choices[0].message.content if response.choices else None
        if not answer:
            raise RuntimeError("No text answer. Inspect the request in Verdictan.")
        print(answer)

asyncio.run(main())

In an async server, await the call from the running event loop instead of calling asyncio.run in a handler.

Stream a response​

Replace the try block in chat.py with this loop, keeping the same client:

stream = client.chat.completions.create(
    model=os.environ["VERDICTAN_MODEL"],
    messages=[{"role": "user", "content": "Explain what a gateway does in two sentences."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        text = chunk.choices[0].delta.content
        if text:
            print(text, end="", flush=True)
print()

The if chunk.choices check skips usage-only chunks. With some policy chains, the gateway holds the whole answer and sends it at the end, so the first token can take a while. See Streaming and SSE for errors and escalations in a stream.

Handle gateway decisions​

Every response carries x-verdictan-verdict and x-verdictan-reason-code headers. Handle these cases:

ResponseWhat it meansWhat to do
200 with x-verdictan-verdict: ESCALATEA policy sent the request for human review. Content is None and finish_reason is content_filter.Tell the user it is under review. Don't resend.
400 with code content_policy_violationA policy blocked the request.Show a refusal. Other 400 codes are malformed requests.
401 or 403The token is missing, revoked, or not allowed.Check the application token and its access.
402A budget or balance limit.Check the error code.
429A rate limit.Honor Retry-After if present, within your retry limit.
Timeout or 5xxThe gateway or provider failed.Investigate before a retry: the provider may have already processed the request.

A 200 without text can also be a tool call, so don't treat every empty answer as an escalation. SDK retries are off in the example so that you decide when to retry. See Error Handling for the full list.

Find the request in Verdictan​

Open History in your organization and search for the printed request ID. Request content is visible only when capture is on in the assigned configuration. Without capture, find the decision in the delivered gateway events.

Next steps​