Skip to main content

Cloud Platforms

Azure OpenAI

Serve an Azure OpenAI deployment through your Verdictan gateway. Your application calls the Verdictan URL with its application token. The gateway calls your Azure resource with the Azure key.

You need four values from one Azure OpenAI resource: the resource host, the deployment name, a supported API version, and an API key.

Before you start​

  • A working gateway from the quickstart.
  • The four Azure values above.
  • The gateway startup command, and a separate application token for test requests.

Configure the provider​

policy-config.yaml
pack:
  name: azure-openai-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: azure-openai-primary
      provider: azure-openai
      model: "replace-with-model-id"
      base_url: https://replace-with-resource-name.openai.azure.com
      secret_key_ref:
        env: VERDICTAN_AZURE_OPENAI_API_KEY
      azure_deployment: "replace-with-deployment-name"
      azure_api_version: "replace-with-supported-api-version"

Set both Azure fields yourself. Without azure_deployment, the gateway uses model as the deployment name, and Azure deployment names are often different from model names. Without azure_api_version, it uses 2024-02-01 on the deployment route, which your resource may not support. Responses requests use a different route. See Supported requests.

The gateway reads the key from VERDICTAN_AZURE_OPENAI_API_KEY. If Azure returns a deployment or API version error, compare both fields with the resource settings.

Start and verify​

For deployment from a file, use CLI 0.1.14 or later. Check In sync. Then test the Azure deployment name with the request below.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

  1. In the gateway terminal, set the key and lint the file:

    export VERDICTAN_AZURE_OPENAI_API_KEY="<provider-api-key>"
verdictan policy lint --file policy-config.yaml

  2. Restart the test gateway with the key in its environment.

  3. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

  4. In a second terminal, send a request for the configured model:

    export VERDICTAN_CLIENT_TOKEN="<application-token>"
curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-model-id","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply, and the request appears in History when capture is on, or in the delivered events.

Supported requests​

The gateway calls /openai/deployments/{deployment}/chat/completions?api-version={version}. It sends the key in one api-key header, without an Authorization header.

With gateway 0.1.26 or later, the gateway sends /v1/responses requests to the Azure v1 route, /openai/v1/responses:

  • The request body model is the deployment name: azure_deployment, or model without it.
  • The gateway does not send a dated azure_api_version, such as 2024-10-21, on this route. The 2024-02-01 default does not apply.
  • The gateway sends a value without a date, such as preview, as ?api-version=preview.
  • If base_url already ends with /openai/v1, the gateway does not add that path again.
RequestSupport
Chat CompletionsYes. Streaming, tools, and structured output work if the deployment supports them.
MessagesTranslated to Chat Completions
ResponsesYes, on the Azure v1 route. Streamed and buffered requests, output redaction, and output blocks work.
AudioThe gateway accepts it for Azure, but it is not verified against a real Azure resource. Test before you rely on it.
EmbeddingsYes, with gateway 0.1.26 or later. The gateway sends /v1/embeddings to /openai/deployments/{deployment}/embeddings?api-version={version} with one api-key header. The policy chain does not run on embeddings.
Image inputNo

Deployment operations use the path without v1/. If the target has no deployment, the gateway uses the model name. If the target has no API version, the gateway uses 2024-02-01. Embeddings are not verified against a real Azure resource. Test before you rely on them.

For a streamed Chat Completions or Messages request, the gateway asks Azure for the stream usage only with azure_api_version 2024-09-01 or later. With an earlier version, the request can record estimated usage.

Failure behavior​

ProblemWhat to check
401 from the gatewayThe application token is active, and the key is set on the gateway host.
Provider authentication errorVERDICTAN_AZURE_OPENAI_API_KEY is set in the gateway process and belongs to this resource.
Deployment not foundazure_deployment is the deployment name, not the model name.
API version errorazure_api_version is a version that the resource supports.
capability.unsupported_familyThe request uses a supported family.
Provider 429 or another upstream errorThe response body and your Azure quota. See Error Handling.

Limits​

  • Model availability, deployment types, API versions, prices, quotas, and data terms are in the Azure OpenAI documentation.
  • One target is one deployment. Add a target for each deployment your application uses.
  • A listener on a non-loopback address needs TLS and network controls.

Next steps​