Skip to main content

Frameworks

Gradio Chat Interface

Build a small Gradio chat page whose Python server sends every message through your Verdictan gateway. The browser only sees answers; the application token stays in the Python process.

Before you start​

Configure the client​

Set three variables in the same terminal:

export VERDICTAN_GATEWAY_URL="http://127.0.0.1:41002/v1"
export VERDICTAN_CLIENT_TOKEN="<application-token>"
export VERDICTAN_MODEL="<configured-model>"

If the gateway runs on another host, use its base URL instead, still ending in /v1.

Install the packages:

python -m pip install gradio openai

Send a request​

app.py
import os
import gradio as gr
from openai import APIConnectionError, APIStatusError, OpenAI

client = OpenAI(
    base_url=os.environ["VERDICTAN_GATEWAY_URL"],
    api_key=os.environ["VERDICTAN_CLIENT_TOKEN"],
    max_retries=0,
    timeout=60.0,
)

def text_of(content):
    if isinstance(content, str):
        return content
    return "".join(part.get("text", "") for part in content if part.get("type") == "text")

def to_messages(message, history):
    messages = []
    for item in history:
        text = text_of(item["content"])
        if item["role"] in ("user", "assistant") and text:
            messages.append({"role": item["role"], "content": text})
    messages.append({"role": "user", "content": message})
    return messages

def respond(message, history):
    try:
        raw = client.chat.completions.with_raw_response.create(
            model=os.environ["VERDICTAN_MODEL"], messages=to_messages(message, history),
        )
        print("Request ID:", raw.headers.get("x-request-id"))
        if raw.headers.get("x-verdictan-verdict") == "ESCALATE":
            return "Waiting for human review. Do not resend yet."
        response = raw.parse()
        answer = response.choices[0].message.content if response.choices else None
        return answer or "No text answer. Check this request in Verdictan."
    except APIStatusError as error:
        if error.status_code == 400 and error.code == "content_policy_violation":
            return "This request was blocked by policy."
        return f"Request failed (HTTP {error.status_code}). Check the gateway decision."
    except APIConnectionError:
        return "The gateway could not be reached. Check it before retrying."

demo = gr.ChatInterface(respond, title="Chat through Verdictan")

if __name__ == "__main__":
    demo.launch(server_name="127.0.0.1", share=False)
python app.py

Open the local URL printed in the terminal and send Reply with Hello. The answer appears in the chat, and the request ID in the terminal.

Gradio passes each earlier turn's content as a list of content blocks; text_of keeps the text blocks so the model sees the conversation. The example handles text only. See the Gradio ChatInterface reference for files and other content.

Stream a response​

Once the non-streaming version works, replace respond with a generator. Gradio shows each partial answer as it arrives:

def respond(message, history):
    try:
        stream = client.chat.completions.create(
            model=os.environ["VERDICTAN_MODEL"],
            messages=to_messages(message, history),
            stream=True,
        )
        answer = ""
        for chunk in stream:
            if chunk.choices and chunk.choices[0].delta.content:
                answer += chunk.choices[0].delta.content
                yield answer
        if not answer:
            yield "No text answer. Check this request in Verdictan."
    except APIStatusError as error:
        if error.status_code == 400 and error.code == "content_policy_violation":
            yield "This request was blocked by policy."
        else:
            yield f"Request failed (HTTP {error.status_code}). Check the gateway decision."
    except APIConnectionError:
        yield "The gateway could not be reached. Check it before retrying."

With output policies that need the complete answer, the text arrives in one piece at the end. An escalated stream has no text, so it shows the "No text answer" message. See Streaming and SSE to tell escalations and errors apart in a stream.

Handle gateway decisions​

The callback covers blocks, other HTTP errors, connection failures, and escalations, and SDK retries are off. Read Error Handling before you add retries or show error details to users.

warning

Everyone who uses this page shares one application token. Before you open it to more people, add sign-in and give each user or service its own token. The example listens on 127.0.0.1 and does not create a public share link.

Find the request in Verdictan​

Search for the printed request ID in History (with capture on) or in the delivered gateway events, and check that the model and decision match your message. A page that loads does not prove its chat goes through Verdictan.

Next steps​