Compare Model Cost and Latency

Send the same task to two configured models. Then compare their response quality, estimated cost, and latency. Verdictan records the cost and runtime measurements. Compare each answer with your requirements.

Start with a gateway that operates correctly and two model routes. Run the comparison with a dedicated test application token to identify the test traffic.

Measurements

Measure	Source	Boundary
Prompt tokens, completion tokens, and estimated cost for each request	Decision events from verdictan events tail or verdictan events export	An estimate from the pricing block of the target, not the provider invoice
Cost grouped by model for a period	Usage, grouped by Model	The control-plane estimate from the price override or catalog price, aggregated. It can differ from the event estimate
Requests, success share, average latency, P95 latency, and cost for each provider and model	Gateways, gateway detail, Runtime tab, Provider performance section	Latency includes the configured policy chain
Response quality	Your application responses, or captured History when enabled	Examine the answers for errors. Make sure that they do the application task. The comparison view does not grade answers

Before you start:

• Start a gateway. See the quickstart in Docker.
• Add each model as a target or nested model in Provider Configuration.
• Make sure that your Console user or API token has gateways:read for access to provider metrics.

Comparison workflow

1. Send the same prompts to each candidate model. Put the model ID in the request body. Keep the prompts, message count, and request family the same:

   curl -fsS http://127.0.0.1:41002/v1/chat/completions \
     -H "Authorization: Bearer ${VERDICTAN_CLIENT_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"model":"replace-with-first-candidate-model","messages":[{"role":"user","content":"Summarize in one sentence: Checkout returned HTTP 503 for 10 minutes after a release. The team rolled back the release, and checkout recovered."}]}'

   Replace replace-with-first-candidate-model with a model ID from model discovery. Then send the same prompt with the second candidate model ID. Keep the same prompt set and policy configuration. Run more than one case. Include a usual short request and a longer request from your application. One response is not sufficient to compare models.

2. In your authenticated CLI terminal, read the per-request records. Use a profile or token with events:read, not the application token. Each decision event contains the model, the prompt tokens, the completion tokens, and the estimated cost:

   verdictan events tail --since 15m --json

3. Export a larger sample for analysis in a spreadsheet or a script:

   verdictan events export --since 24h --format csv > model-comparison.csv

4. Open Usage.

5. Set the time range.

6. Select Model in Group by to compare the total cost for each model.

7. Open Gateways.

8. Select the gateway.

9. Select the Runtime tab. The Provider performance table shows Requests, Success, Avg Latency, P95 Latency, and Cost for each Provider and Model.

10. For the failed requests that cause a low Success value, read the decision events. The table does not show the failure cause.

Select a model from the results

Record one row for each model for the same test set:

Model	Correct answers / total	Estimated cost	Average latency	P95 latency	Errors
First candidate	Your review result	From the test window	From Provider performance	From Provider performance	From events
Second candidate	Your review result	From the same window	From Provider performance	From Provider performance	From events

First, verify the model output against your quality requirement. Then try to decrease cost. A low price or fast response is not useful if the answer is incorrect. Keep the request IDs to examine unusual results.

Limits

• Verdictan records cost estimates. The provider invoice is the authoritative record. See Usage Costs and Budgets.
• For Usage amounts, the control plane selects one price for each request. An active administrator price override has priority. If there is no override, Verdictan uses the imported model catalog price for the same provider and model.
• If neither price is available, the control plane refuses the usage authorization with the error code usage_authorization_budget_estimate_unavailable. The gateway then refuses the request with 503.
• Verdictan does not apply reasoning, audio, batch, service-tier, or regional prices. For these requests, the estimate can be different from the provider charge.
• Latency values include the configured policy chain. A different chain gives different latency values. Compare candidates with the same chain.
• The provider metrics table shows a success share. It does not replace the event records for failure analysis.

Next steps

• Usage Costs and Budgets: set a budget after you select the model.
• Gateway Fleet: read the gateway detail view that holds the provider metrics.
• verdictan events: query and export the per-request records.
• Provider Configuration: add the pricing block and the candidate targets.