Cloudera AI Inference

Serve a model that you deployed on Cloudera AI Inference through your Verdictan gateway. The gateway calls the deployment's OpenAI-compatible serving address with your Cloudera credential, and your application calls the gateway with its application token.

You need the serving endpoint of one deployment. The adapter cannot find it from a workspace name.

Before you start

• A working gateway from the quickstart.
• From one Cloudera AI Inference deployment: its HTTPS serving endpoint, its model ID, and an API key.
• The gateway startup command, and a separate application token for test requests.

Configure the provider

Set base_url to the serving base address, without /v1/chat/completions:

policy-config.yaml

pack:
  name: cloudera-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: cloudera-primary
      provider: cloudera
      model: "replace-with-cloudera-model-id"
      base_url: https://replace-with-cloudera-endpoint
      secret_key_ref:
        env: VERDICTAN_CLOUDERA_API_KEY

base_url must start with https://. The gateway appends /v1/chat/completions and sends the key from VERDICTAN_CLOUDERA_API_KEY as a Bearer token.

Start and verify

To publish the Cloudera target from your terminal, use CLI 0.1.14 or later. Check In sync, then test your configured endpoint with the request below.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

1. In the gateway terminal, set the key and lint the file:

   export VERDICTAN_CLOUDERA_API_KEY="<provider-api-key>"
   verdictan policy lint --file policy-config.yaml

2. Restart the test gateway with the key in its environment.

3. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

4. In a second terminal, send a request for the configured model:

   export VERDICTAN_CLIENT_TOKEN="<application-token>"
   curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
     -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"model":"replace-with-cloudera-model-id","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply, and the request appears in History when capture is on, or in the delivered events.

Supported requests

Request	Support
Chat Completions	Yes, sent to the serving address plus /v1/chat/completions. Streaming and tools work if the deployed model supports them.
Messages	Translated to Chat Completions
Responses, embeddings, image input, response_format, parallel tool calls	No

Check that your deployment accepts the Chat Completions request shape before you connect it.

Failure behavior

Problem	What to check
401 from the gateway	The application token is active, and the key is set on the gateway host.
Provider authentication error	VERDICTAN_CLOUDERA_API_KEY is set in the gateway process and belongs to this deployment.
404 or model not found	base_url is the serving address without a path, and the model ID belongs to the same deployment.
capability.unsupported_family	The request is Chat Completions or Messages.
Provider 429 or another upstream error	The response body and your deployment's limits. See Error Handling.

Limits

• Model names, prices, rate limits, and data terms are in the Cloudera AI Inference documentation.
• The endpoint, model ID, and credential must all belong to one deployment. A valid key does not reach a model in another deployment.
• The deployed model decides which request fields work. Test streaming and tools with your endpoint.
• A listener on a non-loopback address needs TLS and network controls.

Next steps

• Provider Configuration: target fields, routing, and fallback.
• Runtime Request Families: request families and their policy stages.