Skip to main content

Hosts & Gateways

Groq

Call Groq models through your Verdictan gateway. Your Groq key stays on the gateway, and your application calls the gateway in the OpenAI format with its application token.

Before you start​

  • A working gateway from the quickstart.
  • A Groq API key, and one model ID that your account can use.
  • The gateway startup command, and a separate application token for test requests.

Configure the provider​

Set base_url to https://api.groq.com/openai, with no /v1. The Groq SDK base URL ends in /openai/v1, but the gateway adds /v1/... itself, so copying the SDK value gives a doubled /v1.

policy-config.yaml
pack:
  name: groq-integration
  version: 1.0.0
  enabled: true
policies:
  chain:
    - prompt-injection
providers:
  targets:
    - id: groq-primary
      provider: groq
      model: "replace-with-groq-model-id"
      base_url: https://api.groq.com/openai
      secret_key_ref:
        env: VERDICTAN_GROQ_API_KEY

The gateway appends the path for each request family, such as /v1/chat/completions, and sends the key from VERDICTAN_GROQ_API_KEY as a Bearer token.

Start and verify​

For a terminal deployment, use CLI 0.1.14 or later with this Groq configuration. Check In sync, then send the request below with your application token.

With the quickstart in Docker, follow the container steps instead of steps 1 to 3.

  1. In the gateway terminal, set the key and lint the file:

    export VERDICTAN_GROQ_API_KEY="<provider-api-key>"
verdictan policy lint --file policy-config.yaml

  2. Restart the test gateway with the key in its environment.

  3. Deploy the file to the test agent in Configurations. If that agent already runs pack.version 1.0.0, increase the version first. Then wait for the gateway log line connected read model refresh applied updated config with your version as config_version. Try the change on a test gateway has the steps.

  4. In a second terminal, send a request for the configured model:

    export VERDICTAN_CLIENT_TOKEN="<application-token>"
curl --fail-with-body http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"model":"replace-with-groq-model-id","messages":[{"role":"user","content":"Reply with Hello."}]}'

choices[0].message.content contains the reply, and the request appears in History when capture is on, or in the delivered events.

Supported requests​

RequestSupport
Chat CompletionsYes
ResponsesYes
MessagesTranslated to Chat Completions
Streaming, tools, response_formatYes, if the model supports them
Embeddings, audio, image inputNo

Field limits differ by model. See Groq OpenAI compatibility.

Failure behavior​

ProblemWhat to check
401 from the gatewayThe application token is active, and the key is set on the gateway host.
Provider authentication errorVERDICTAN_GROQ_API_KEY is set in the gateway process. Restart the gateway.
404 from Groqbase_url is https://api.groq.com/openai, without /v1.
Model not foundThe request model matches the target, and your account can use it.
capability.unsupported_familyThe request uses a family from the table.
Provider 429 or another upstream errorThe response body and your Groq rate limits. See Error Handling.

Limits​

  • Model names, prices, rate limits, and data terms are in the Groq documentation.
  • Streaming, tools, and structured output vary by model. Test each one with your model.
  • A listener on a non-loopback address needs TLS and network controls.

Next steps​