Skip to main content

Model Providers

OpenAI

Put Verdictan between your application and the OpenAI API. Your OpenAI key stays on the gateway, your application keeps its OpenAI SDK and authenticates with a Verdictan application token, and every request passes your policy chain first.

OpenAI is the provider with the widest gateway support: chat, Responses, embeddings, and audio. Start with one text request.

Before you start​

  • A working gateway from the quickstart.
  • An OpenAI API key for the project that owns your model, and one model ID that the project can use.
  • The gateway startup command, and a separate application token for test requests.

Configure the provider​

Set provider: openai, a model, and base_url: https://api.openai.com:

policy-config.yaml
pack:
  name: openai-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: openai-primary
      provider: openai
      model: "gpt-4o-mini"
      base_url: https://api.openai.com
      secret_key_ref:
        env: OPENAI_API_KEY

The gateway appends the request path, such as /v1/chat/completions, to base_url, and sends the key from OPENAI_API_KEY as a Bearer token. The key never appears in the file or in application requests.

Route to a fine-tuned model​

Put the complete fine-tuned model ID in model. The gateway passes it to OpenAI unchanged. Do not put it in a provider shorthand, which splits at every colon.

Inside policy-config.yaml
providers:
  targets:
    - id: openai-finetuned
      provider: openai
      model: "replace-with-fine-tuned-model-id"
      base_url: https://api.openai.com
      secret_key_ref:
        env: OPENAI_API_KEY
      headers:
        OpenAI-Organization: "replace-with-openai-organization-id"

Every entry in headers goes to OpenAI with the request. Use it when the key must select another organization or project, and leave it out otherwise. Verdictan does not validate the header names.

Fine-tuned models are not in the Verdictan model catalog, so add a pricing block to the target to get correct cost records. See Provider Configuration.

Start and verify​

You can deploy the OpenAI configuration with CLI 0.1.14 or later instead of step 4. Check In sync, then send the request in step 5. Configuration sync alone does not prove OpenAI access.

With the quickstart in Docker, follow the container steps instead of steps 1 to 4, then come back for the test request.

  1. In the gateway terminal, set the key:

    export OPENAI_API_KEY="<provider-api-key>"

  2. Lint the file:

    verdictan policy lint --file policy-config.yaml

  3. Restart the test gateway with the key in its environment.

  4. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

  5. In a second terminal, set the application token and send a request for the configured model to the local listener:

    export VERDICTAN_CLIENT_TOKEN="<application-token>"
curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply. With capture on, the request is in History with its provider, model, and decision. Otherwise, find it in the delivered events. A health check or model list does not prove a provider call.

Connect an OpenAI-compatible client​

Give your OpenAI application three values. It never needs the OpenAI URL or key.

Client settingValue
Base URLThe agent URL or gateway URL with /v1. For the gateway above: http://127.0.0.1:41002/v1.
API keyThe client API token
ModelA model ID from model discovery. It must equal a target model.

The agent URL is the published host of the agent in the Console. See Agents. The gateway URL is the --listen address of a gateway that you run.

Check the values with an authenticated model discovery request:

curl -fsS http://127.0.0.1:41002/v1/models \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}"

The data array has one object for each target model. Use one of the id values as the client model. The client then calls the routes in the capability table, and the gateway applies the policy chain before dispatch.

Supported requests​

TaskGateway route and requirement
Chat or Responses/v1/chat/completions or /v1/responses, with a text model
Anthropic-format client/v1/messages, translated to Chat Completions. See Format Translation.
Embeddings/v1/embeddings, with an embedding target
Speech or transcription/v1/audio/speech or /v1/audio/transcriptions, with an audio target

The policy chain runs on Chat Completions, Responses, and Messages. Embeddings and audio requests are authenticated and pass provider access and budget rules, but the chain does not run on them. This adapter does not accept image input.

Both the gateway and the model must support a feature. Test streaming, tool calls, and each request family on their own. Runtime Request Families lists the policy stages of each family.

Failure behavior​

ProblemWhat to check
401 from the gatewayThe application token is active, and the key is set on the gateway host.
Provider authentication errorOPENAI_API_KEY is set in the gateway process or container. Restart the gateway.
Model not found or not allowedThe request model matches the target, and the project can use the model.
capability.unsupported_family or capability.unsupported_modalityThe adapter supports the request format. See Supported requests.
Provider 429 or another upstream errorThe response body and your provider quota. See Error Handling.

Limits​

  • Model names, prices, context limits, and retirement dates are in the OpenAI API reference. Verdictan does not publish them.
  • One target serves one model. Add a target for each embedding, speech, or transcription model, and test each family with its own request.
  • Streaming, tools, and structured output can behave differently for each model. Test them with your configured model.
  • Policy controls do not inspect embeddings or audio input or output.
  • A listener on a non-loopback address needs TLS and network controls before remote clients use it.

Next steps​