Together AI

Call Together AI chat and embedding models through your Verdictan gateway. Your Together key stays on the gateway, and your application calls the gateway with its application token.

Each target serves one model. Use the full model ID from the Together catalog, not the display name. For chat and embeddings, configure one target for each.

Before you start

• A working gateway from the quickstart.
• A Together AI API key, and the full catalog ID of each model you want to use.
• The gateway startup command, and a separate application token for test requests.

Configure the provider

Set base_url explicitly to the API host in the Together AI documentation. Without it, the adapter falls back to https://api.together.xyz.

policy-config.yaml

pack:
  name: together-ai-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: together-ai-primary
      provider: togetherai
      model: "replace-with-together-model-id"
      base_url: https://api.together.ai
      secret_key_ref:
        env: VERDICTAN_TOGETHER_API_KEY
    - id: together-ai-embed
      provider: togetherai
      model: "replace-with-together-embedding-model-id"
      base_url: https://api.together.ai
      secret_key_ref:
        env: VERDICTAN_TOGETHER_API_KEY

The gateway appends the path for each request family, such as /v1/chat/completions or /v1/embeddings, and sends the key from VERDICTAN_TOGETHER_API_KEY as a Bearer token. Leave out together-ai-embed if you only need chat.

Start and verify

With CLI 0.1.14 or later, deploy the Together AI target from your local file. Check In sync before you test the model. The test request verifies provider access separately.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

1. In the gateway terminal, set the key and lint the file:

   export VERDICTAN_TOGETHER_API_KEY="<provider-api-key>"
   verdictan policy lint --file policy-config.yaml

2. Restart the test gateway with the key in its environment.

3. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

4. In a second terminal, send a chat request:

   export VERDICTAN_CLIENT_TOKEN="<application-token>"
   curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
     -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"model":"replace-with-together-model-id","messages":[{"role":"user","content":"Reply with Hello."}]}'

5. If you kept the embedding target, send an embeddings request:

   curl --fail-with-body http://127.0.0.1:41002/v1/embeddings \
     -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"model":"replace-with-together-embedding-model-id","input":["Hello"]}'

The chat reply is in choices[0].message.content, and the vectors are in data[].embedding. The chat request appears in History when capture is on, or in the delivered events.

Supported requests

Request	Support
Chat Completions	Yes. Streaming and tools work if the model supports them.
Messages	Translated to Chat Completions
Embeddings	Yes, with an embedding model
Responses, audio, image input, response_format, parallel tool calls	No

note

The gateway authenticates embeddings requests and applies provider access and budget rules, but your policy chain does not run on them. Only Chat Completions, Responses, and Messages go through the chain.

Failure behavior

Problem	What to check
401 from the gateway	The application token is active, and the key is set on the gateway host.
Provider authentication error	VERDICTAN_TOGETHER_API_KEY is set in the gateway process. Restart the gateway.
Model not found	The request uses the full catalog ID, and it matches one target.
Error from /v1/embeddings	The request names the embedding model, not the chat model.
Provider 429 or another upstream error	The response body and your Together limits. See Error Handling.

Limits

• Model names, prices, rate limits, and data terms are in the Together AI documentation.
• Streaming and tools vary by model. Test each one with your model.
• A listener on a non-loopback address needs TLS and network controls.

Next steps

• Provider Configuration: target fields, routing, and fallback.
• Runtime Request Families: request families and their policy stages.