Skip to main content

Cloud Platforms

Databricks Model Serving

Serve a Databricks Model Serving endpoint through your Verdictan gateway. The gateway calls the endpoint with your Databricks token or OAuth2 credentials, and your application calls the gateway with its application token.

The endpoint must accept an OpenAI-style messages body. In the configuration, model is the serving endpoint name, not the name of the model behind it.

Before you start​

  • A working gateway from the quickstart.
  • Your HTTPS workspace host, and the name of one serving endpoint that accepts a Chat Completions body.
  • A Databricks token that can invoke that endpoint, or OAuth2 client credentials.
  • The gateway startup command, and a separate application token for test requests.

Configure the provider​

policy-config.yaml
pack:
  name: databricks-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: databricks-primary
      provider: databricks
      model: "replace-with-serving-endpoint-name"
      base_url: https://replace-with-workspace-host
      secret_key_ref:
        env: VERDICTAN_DATABRICKS_API_KEY

base_url is the workspace host, starting with https://. The gateway sends the token from VERDICTAN_DATABRICKS_API_KEY as a Bearer token. To use OAuth2 instead of a static token, configure the target's oauth2 field. See OAuth2 authentication.

Start and verify​

Use CLI 0.1.14 or later if you keep the Databricks target in a local file. After In sync, test the endpoint name below. The token must remain available to the gateway.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

  1. In the gateway terminal, set the token and lint the file:

    export VERDICTAN_DATABRICKS_API_KEY="<provider-api-key>"
verdictan policy lint --file policy-config.yaml

  2. Restart the test gateway with the Databricks token in its environment.

  3. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

  4. In a second terminal, send a request whose model is the endpoint name:

    export VERDICTAN_CLIENT_TOKEN="<application-token>"
curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-serving-endpoint-name","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply, and the request appears in History when capture is on, or in the delivered events.

Supported requests​

RequestSupport
Chat CompletionsYes, sent to /serving-endpoints/{model}/invocations. Streaming and tools work if the endpoint supports them.
MessagesTranslated to Chat Completions
Responses, embeddings, image input, response_format, parallel tool callsNo

The endpoint must return an OpenAI-compatible response. Verdictan does not translate the request into DataFrame or tensor input.

Failure behavior​

ProblemWhat to check
401 from the gatewayThe application token is active, and the token is set on the gateway host.
Provider authentication errorVERDICTAN_DATABRICKS_API_KEY, or the oauth2 settings, can invoke the endpoint.
404 from Databricksmodel is the serving endpoint name, and base_url is the workspace host.
Schema errors from the endpointThe endpoint accepts a messages body, not DataFrame or tensor input.
Provider 429 or another upstream errorThe response body and the endpoint's limits. See Error Handling.

Limits​

  • Endpoint types, prices, rate limits, and data terms are in the Databricks Model Serving documentation.
  • One target is one serving endpoint. Add a target for each endpoint your application uses.
  • A listener on a non-loopback address needs TLS and network controls.

Next steps​