Skip to main content

Frameworks

Instructor Structured Output

Use Instructor to turn model output into a validated Pydantic object, while Verdictan applies your policies to the model call. This page extracts a typed incident summary from a short report and shows how to keep Instructor's retries from fighting the gateway's decisions.

The example uses Instructor's TOOLS mode, so your Verdictan target must support tool calls, as an OpenAI target does.

Before you start​

  • A running gateway from the quickstart, with its application token and a model that supports tools.
  • Python 3.11 or newer in a virtual environment.

Configure the client​

Set three variables in the same terminal:

export VERDICTAN_GATEWAY_URL="http://127.0.0.1:41002/v1"
export VERDICTAN_CLIENT_TOKEN="<application-token>"
export VERDICTAN_MODEL="<configured-model>"

If the gateway runs on another host, use its base URL instead, still ending in /v1.

Install the packages:

python -m pip install instructor pydantic openai

Send a request​

extract.py
import os
import instructor
from openai import OpenAI
from pydantic import BaseModel
from typing import Literal
from tenacity import Retrying, stop_after_attempt

class IncidentSummary(BaseModel):
    system: str
    severity: Literal["low", "medium", "high"]
    summary: str

client = instructor.from_openai(
    OpenAI(
        base_url=os.environ["VERDICTAN_GATEWAY_URL"],
        api_key=os.environ["VERDICTAN_CLIENT_TOKEN"],
        max_retries=0,
        timeout=60.0,
    ),
    mode=instructor.Mode.TOOLS,
)

incident = client.chat.completions.create(
    model=os.environ["VERDICTAN_MODEL"],
    response_model=IncidentSummary,
    max_retries=Retrying(stop=stop_after_attempt(1)),
    messages=[{
        "role": "user",
        "content": (
            "Extract this incident: The checkout service was unavailable for "
            "ten minutes. No orders could be placed. Severity: high."
        ),
    }],
)
print(incident.model_dump_json(indent=2))
python extract.py

You get an object with system, severity, and summary. severity should be high; the summary wording varies. The schema checks shape and allowed values, not whether the facts are right, so compare the result with the input in your application.

Control retries​

Instructor retries when validation fails, and the OpenAI client retries on transport errors. The example turns both off:

SettingEffect
OpenAI max_retries=0No transport retries
Instructor Retrying(stop=stop_after_attempt(1))One extraction attempt, no validation retry

Keep it that way until you have seen how your policies respond. When all attempts fail, Instructor raises InstructorRetryException. Look at the underlying error and the gateway record before you decide what to tell the user:

  • A block, budget error, or escalation must not start a schema-repair loop.
  • An escalated request has no model answer, and an output policy can change the answer. Either one can fail validation; don't show that as a completed extraction.
  • If you later allow validation retries, each retry is a new governed model request with its own provider cost.

See Instructor retry behavior and Error Handling.

Find the request in Verdictan​

  1. Run the script once and check the three fields.
  2. Find the request in History (with capture on) or in the delivered gateway events.
  3. Confirm there was exactly one model call and that it was allowed.
  4. Before you turn on automatic retries, test once with output that fails validation and once with a request your policies refuse.

Next steps​