Cloud Platforms

IBM watsonx.ai

Call IBM watsonx.ai chat models through your Verdictan gateway. Your application sends OpenAI Chat Completions or Responses requests with its application token. The gateway translates each request to the watsonx.ai chat API, adds your project or space, and authenticates with an IBM Cloud IAM token.

Before you start

A working gateway from the quickstart.
An IBM Cloud API key, a watsonx.ai project ID or deployment space ID, and the regional watsonx.ai endpoint for that project or space.
One chat model ID that the project or space can use, and an API version date from the watsonx.ai API reference.
The gateway startup command, and a separate application token for test requests.

Configure the provider

policy-config.yaml

pack:
  name: watsonx-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: watsonx-chat
      provider: watsonx
      provider_type: watsonx
      model: "replace-with-watsonx-model-id"
      base_url: https://us-south.ml.cloud.ibm.com
      watsonx_api_version: "replace-with-version-date"
      watsonx_project_id: "replace-with-project-id"
      secret_key_ref:
        env: VERDICTAN_WATSONX_API_KEY

Each field has a fixed rule:

Field	Rule
`provider`	Exactly `watsonx`. The gateway rejects aliases such as `watsonx-ai`.
`provider_type`	`watsonx`. `verdictan policy lint` checks the other watsonx fields only when this field is set.
`base_url`	Required. An `https://` URL whose host ends with `.ml.cloud.ibm.com`. Use the region of your project or space.
`watsonx_api_version`	Required. The gateway sends it as the `version` query parameter.
`watsonx_project_id` or `watsonx_space_id`	Set exactly one. The gateway adds it to every request body as `project_id` or `space_id`.
`model`	The gateway sends it as `model_id`.
`secret_key_ref`	An environment variable that holds an IBM Cloud API key or an IAM access token.

The gateway calls /ml/v1/text/chat, or /ml/v1/text/chat_stream for streaming requests, with an Authorization: Bearer header.

IAM tokens

The gateway gets the Bearer token in this order:

WATSONX_ACCESS_TOKEN from the gateway environment, when it is set. This applies to every watsonx target.
The secret_key_ref value, if it starts with eyJ. The gateway uses it as an access token.
Any other secret_key_ref value. The gateway exchanges it as an IBM Cloud API key at https://iam.cloud.ibm.com/identity/token and caches the token until 30 seconds before it expires.

The target needs a resolved secret_key_ref in all three cases. Without it, the target is inactive, even when WATSONX_ACCESS_TOKEN is set. An access token expires, and the gateway does not refresh it. Use an API key for a long-running gateway.

Start and verify

With CLI 0.1.14 or later, you can deploy the model and project configuration from your terminal. Check In sync. Then send the request below to verify IBM credentials and model access.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

In the gateway terminal, set the key, then check the file. policy lint checks the policy pack and the watsonx fields, and config validate checks the provider target fields. Neither command contacts IBM Cloud.
```
export VERDICTAN_WATSONX_API_KEY="<ibm-cloud-api-key>"
verdictan policy lint --file policy-config.yaml
verdictan config validate --file policy-config.yaml
```
Restart the test gateway with the key in its environment.
In Configurations in the Console, create or edit the test configuration. Import the YAML and select Next. Under Deploy to agent, select the test agent. Select Deploy, then confirm the gateway count. Wait about 30 seconds. Send the request below to check that the gateway uses this model and project.

In a second terminal, send a request for the configured model:

export VERDICTAN_CLIENT_TOKEN="<application-token>"
curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-watsonx-model-id","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply. A reply proves the whole path: application token, IAM token exchange, project or space, and model access. The request appears in History when capture is on, or in the delivered events.

Supported requests

Request	Support
Chat Completions	Yes. Streams come back as watsonx sends them.
Responses	Yes. Send `input` as a string or as messages. Streams come back in the Responses format.
Tools	Yes. Tools are forwarded unchanged.
`response_format` with `json_object` or `json_schema`	Yes, forwarded unchanged. A schema with `strict: true` is rejected.
Messages, embeddings, image input, `parallel_tool_calls: true`, strict tools	No. The gateway rejects the request with HTTP `422` before it calls watsonx.

Both supported request families go through your policy chain.

The gateway builds a new watsonx request from these fields: messages (or Responses input), stream, max_tokens, max_completion_tokens, temperature, top_p, presence_penalty, frequency_penalty, seed, response_format, stop, tools, and tool_choice. Messages keep their tool_calls and tool_call_id. A string tool_choice, such as auto, becomes tool_choice_option. The gateway drops all other fields, so your application cannot set project_id or space_id.

Failure behavior

Problem	Result or cause
Lint reports `provider 'watsonx-ai' must use exact provider id 'watsonx'`	`provider` is an alias. Set it to `watsonx`.
Lint or `config validate` names `watsonx_api_version`	Add a nonempty API version.
Lint or `config validate` reports `exactly one of watsonx_project_id or watsonx_space_id`	Set one of the two fields, not both.
Lint or `config validate` reports `requires a regional https://*.ml.cloud.ibm.com base_url`	Use the `https://` regional endpoint.
Lint reports `should set 'secret_key_ref' to an IBM Cloud API key or access token reference`	Add `secret_key_ref`.
The gateway reports `Provider target 'watsonx-chat' is inactive: environment variable 'VERDICTAN_WATSONX_API_KEY' is not set`	Set the key in the gateway process and restart it.
`502` with `provider_auth_failed` and `watsonx IAM token exchange failed`	IBM Cloud IAM rejected the API key, or the gateway could not reach `iam.cloud.ibm.com` within 5 seconds.
`401` from the gateway	The application token is not active.
`400` with `provider_request_invalid` and `watsonx request requires at least one message`	The request has no messages or an empty `input`.
`422` with `capability.unsupported_family` or `capability.unsupported_modality`	The request uses Messages, embeddings, or image input.
watsonx returns an error status	The client gets the same status with code `upstream_provider_error` and the message `Upstream provider returned HTTP <status>`. The watsonx error body is not returned. Check the token, the project or space, the model, and the API version. If another target is eligible, some errors move the request to that target first.

Limits

Model IDs, regions, version dates, and request fields are in the IBM watsonx.ai API reference.
One target serves one model in one project or space. Add a target for each.
If the watsonx call fails with a rate limit, a server error, an authentication error, or a transport error, the gateway tries the next eligible target. Provider attempts and fallback has the rules.
A listener on a non-loopback address needs TLS and network controls.

Next steps

Provider Configuration: target fields, credentials, and routing.
Format Translation: how the gateway converts between request formats.
Runtime Request Families: request families and their policy stages.

Before you start​

Configure the provider​

IAM tokens​

Start and verify​

Supported requests​

Failure behavior​

Limits​

Next steps​