verdictan cache

The gateway response cache stores a model response and returns it for a matching request. Use verdictan cache to examine the cache storage, read the metadata of one entry, or delete all entries.

The commands open the storage directly from the VERDICTAN_LLM_CACHE_* environment variables of your shell. They do not connect to a running gateway. To see the cache of a gateway, use the same variables as the gateway process.

Before you start

• Set VERDICTAN_LLM_CACHE_BACKEND to the backend of the gateway: memory, filesystem, redis, valkey, s3, gcs, or qdrant. Without it, the commands stop with an error, except when VERDICTAN_ENV is development, dev, test, or testing. In that case they use memory.
• For the filesystem backend, set VERDICTAN_LLM_CACHE_DIR if the gateway sets it. For other backends, set the same connection variables as the gateway. The runtime configuration lists them.
• Do not set VERDICTAN_LLM_CACHE_CLEAR=true in your shell. With this variable, each verdictan cache command deletes all entries when it opens the cache.

What the commands can see depends on the backend:

Backend	What the commands can see
filesystem	All entries in the directory. stats shows counters and list shows entries.
redis, valkey, s3, gcs, qdrant	The shared storage. inspect and clear work. stats shows the backend only, and list prints Cache is empty.
memory	An empty cache in the CLI process only. The gateway memory cache is not visible.

When VERDICTAN_LLM_CACHE_ENABLED is false, the commands open an empty memory cache, for every backend.

verdictan cache stats

verdictan cache stats prints the backend and the Enabled value. For the filesystem backend, it also prints the directory and the counters.

Usage

verdictan cache stats

Examples

export VERDICTAN_LLM_CACHE_BACKEND=filesystem
verdictan cache stats

On Linux, without VERDICTAN_LLM_CACHE_DIR, the output is similar to this:

Cache Backend: filesystem
Enabled:       true
Directory:     /home/<user>/.cache/verdictan/llm-provider-cache
Entry Count:   0
Total Size:    0.0 MB / 500 MB
Hit/Miss:      0 / 0 (0.0% hit rate)
Evictions:     0
Warmed:        true

The default directory is $XDG_CACHE_HOME/verdictan/llm-provider-cache, or ~/.cache/verdictan/llm-provider-cache when XDG_CACHE_HOME is not set. On macOS, it is ~/Library/Caches/verdictan/llm-provider-cache. Make sure that Directory is the directory of the gateway.

Entry Count and Total Size come from a scan of the directory when the command starts. Hit/Miss and Evictions come from the .cache-stats.json file that the gateway writes in the directory. The size limit is VERDICTAN_LLM_CACHE_MAX_BYTES. The default is 500 MB. When VERDICTAN_LLM_CACHE_WARMUP_ENABLED is false, the command does not scan, and the entry count is 0.

For other backends, the command prints (filesystem stats not available for this backend) after the backend and Enabled lines.

verdictan cache list

verdictan cache list prints the entries of the filesystem backend, with the largest or most recent first.

Usage

verdictan cache list [--top <n>] [--by-size]

Options

Flag	Description
--top <n>	Number of entries to print. The default is 20.
--by-size	Sort by size, largest first. Without this flag, the command sorts by access time.

Examples

verdictan cache list --top 10 --by-size

Each row has the key, the size, and the time since the last access. The command shortens a key that is longer than 48 characters. When there are no entries, it prints Cache is empty.

The command reads the entries from disk when it starts, so the access time of each entry is the start time of the command. Sort by size to get a useful order.

verdictan cache inspect

verdictan cache inspect prints the metadata of one entry. It does not print the response body.

Usage

verdictan cache inspect <key>

Options

Argument	Description
<key>	The complete cache key.

Examples

verdictan cache inspect '<cache-key>'

The output has the key, the HTTP status, the body size, the storage time, the key version, the number of headers, and the content type. When no entry has the key, the command prints No cache entry found for key: <key> and exits with code 0.

A key that list shortened ends with …. That key does not match. Use the complete key.

verdictan cache clear

verdictan cache clear deletes all entries of the selected backend.

Usage

verdictan cache clear [--yes]

Options

Flag	Description
--yes	Do not ask for confirmation.

Examples

verdictan cache stats
verdictan cache clear
verdictan cache stats

Without --yes, the command asks This will delete all cached entries. Continue? [y/N]. An answer other than y or yes prints Aborted., and the command exits with code 0. After the deletion, the command prints Cache cleared.

Before you clear a shared backend, examine the URL, bucket, prefix, collection, and credentials in your environment. The command deletes the entries for all gateways that use the same storage. For qdrant, it deletes the collection.

To stop the use of old entries without a deletion, change the cache buster on the gateway:

export VERDICTAN_LLM_CACHE_BUSTER="replace-with-new-value"

The gateway adds the value to its cache keys, so earlier entries do not match. The earlier entries stay in storage.

Limits and failure behavior

• For redis, valkey, s3, gcs, and qdrant, clear does not report delete failures. The command logs a warning and prints Cache cleared. Examine the backend to confirm the result.
• With VERDICTAN_LLM_CACHE_ENABLED=false or the memory backend, clear deletes nothing from the gateway and still prints Cache cleared.
• When your shell variables are different from the gateway variables, the commands examine a different cache.

To confirm that the gateway uses the cache, send the same non-streaming Chat Completions or Responses request two times. The gateway response has the header x-cache-status: hit or x-cache-status: miss. Low latency alone does not show a hit.

Troubleshooting

Symptom	Cause	Fix
VERDICTAN_LLM_CACHE_BACKEND='auto' requires an explicit backend in production; ...	VERDICTAN_LLM_CACHE_BACKEND is not set.	Set it to the gateway backend.
invalid VERDICTAN_LLM_CACHE_BACKEND (expected ...)	The value is not a backend name.	Correct the value. file and disk also select filesystem.
VERDICTAN_LLM_CACHE_BACKEND=redis or valkey requires VERDICTAN_LLM_CACHE_REDIS_URL	A backend variable is missing. The s3, gcs, and qdrant backends give similar errors.	Set the variables that the error names.
invalid VERDICTAN_LLM_CACHE_TTL_SECS (expected positive integer seconds)	The TTL is not an integer.	Set a number of seconds.
stats shows a different backend or directory than the gateway	The shell environment is different from the gateway environment.	Copy the cache variables from the gateway service.
A repeated request gets x-cache-status: miss	The request is different, the entry expired, or the cache buster changed.	Compare the requests, VERDICTAN_LLM_CACHE_TTL_SECS, and VERDICTAN_LLM_CACHE_BUSTER.

Next steps

• Runtime Configuration: turn on the cache in the policy config and set the backend variables.
• verdictan gateway: examine the environment of an installed gateway service.
• verdictan gateway run: start a gateway with the cache variables.