Skip to main content

verdictan models

The CLI does not include two optional local models. Use verdictan models to download, verify, and remove them on the gateway host.

ModelPinned versionFilesUsed by
embeddingall-minilm-l6-v2-int8-ort-1.30.0all-MiniLM-L6-v2 (int8 ONNX) and the onnxruntime-web WebAssembly runtime, approximately 38 MBembedding-detector with backend: local
ocrtesseract-core-6.1.2-eng-fast-4.1.0tesseract.js-core and the English language data, approximately 7 MBdocument-analyzer for text in images

Without the models, the gateway operates as before. embedding-detector with backend: local compares shared words, and document-analyzer does not read text in images.

The gateway never downloads a model. Only verdictan models install adds the files.

Before you start​

  • Run the commands on the gateway host, as the user that runs the gateway.
  • The commands use the data directory $VERDICTAN_DATA_DIR. When this variable is not set, they use ~/.verdictan/data. If the gateway service sets VERDICTAN_DATA_DIR, set the same value in your shell.
  • For a download, the host needs HTTPS access to cdn.jsdelivr.net and huggingface.co. For a host without network access, use --from <DIR>.

The files go into <data directory>/models/<model>/<version>.

verdictan models list​

verdictan models list prints each model, its pinned version, its state, its size, and the control that uses it. For each file, it also prints the size, the license, the SHA-256 digest, and the download URL.

Usage​

verdictan models list [--json]

Options​

FlagDescription
--jsonPrint a JSON array. Each entry has id, version, feature, state, directory, bytes, and artifacts. Each artifact has name, url, sha256, size, and license.

list finds the state from the file names and sizes only. To verify the digests, use status.

verdictan models status​

verdictan models status calculates the SHA-256 digest of each installed file and compares it with the pinned digest.

Usage​

verdictan models status [--json]

Options​

FlagDescription
--jsonPrint a JSON array. Each entry has id, version, state, directory, bytes, and reason.

The state is one of these values:

StateMeaning
installedAll pinned files are present and agree with their digests. reason is null.
not_installedThe pinned version is not installed. reason also tells you when a different version is installed.
invalidA file is missing, has the wrong size, or does not agree with its digest. reason names the file.

Examples​

On a host without the models, the output is:

embedding all-minilm-l6-v2-int8-ort-1.30.0: not installed (not installed). Install it with: verdictan models install embedding
  Without it, embedding-detector backend local uses lexical similarity.
ocr tesseract-core-6.1.2-eng-fast-4.1.0: not installed (not installed). Install it with: verdictan models install ocr
  Without it, document-analyzer does not read text in images.

verdictan models install​

verdictan models install gets the pinned files of one model, verifies each file, and puts the files into the data directory.

Usage​

verdictan models install <MODEL> [--from <DIR>]

Options​

Argument or flagDescription
<MODEL>embedding or ocr.
--from <DIR>Read the pinned files from this directory. The command does not use the network.

Examples​

verdictan models install embedding
verdictan models install ocr

The command prints one Downloading <file> (<size>) from <url> line on stderr for each file. At the end, it prints Installed <model> <version> in <directory>. If the pinned version is already installed and valid, it prints <model> <version> is already installed in <directory> and makes no change.

The command checks the size and the SHA-256 digest of each file before it installs the model. If one file does not agree, the command stops and installs nothing. The command writes the files to a temporary directory first. It moves them into position only after all files are verified. An install of a new pinned version removes the earlier versions of the same model.

Install without network access​

  1. On a host with network access, run verdictan models list to get the URL of each file.
  2. Download each file into one directory. Keep the file names from the URLs.
  3. Copy the directory to the gateway host.
  4. Install the model from the directory:
verdictan models install ocr --from ./verdictan-models/ocr

The command prints Reading <file> (<size>) from <path> for each file. It applies the same size and digest checks as a download.

verdictan models remove​

verdictan models remove deletes all installed versions of one model.

Usage​

verdictan models remove <MODEL>

Options​

ArgumentDescription
<MODEL>embedding or ocr.

Examples​

verdictan models remove embedding

The command prints Removed the embedding model. A running gateway keeps a loaded model until it restarts. If the model is not installed, it prints The embedding model is not installed.

How the gateway uses the models​

  • The gateway loads a model when a control first needs it. Before it loads the files, it verifies their sizes and digests again.
  • When a model is not installed, the gateway logs this one time and uses the fallback. It looks for the model again after 60 seconds, so an install takes effect without a restart.
  • When an installed model does not load, the gateway logs the error and uses the fallback until it restarts.
  • When the local embedding model fails on one input, embedding-detector uses lexical similarity for that input.
  • OCR has a time limit of 10 seconds for each image. To change it, set VERDICTAN_OCR_TIMEOUT_MS on the gateway to a value from 1 to 600000. When OCR fails or exceeds the limit, the image stays unread.

Readiness report​

GET /readyz reports the models in optional_models. The models do not change the readiness status.

{
  "optional_models": {
    "embedding": {
      "state": "installed",
      "version": "all-minilm-l6-v2-int8-ort-1.30.0",
      "runtime": "loaded",
      "feature": "embedding-detector"
    },
    "ocr": {
      "state": "not_installed",
      "version": "tesseract-core-6.1.2-eng-fast-4.1.0",
      "runtime": "unavailable",
      "feature": "document-analyzer",
      "reason": "not installed",
      "fallback": "images stay unread"
    }
  }
}
FieldMeaning
stateinstalled, not_installed, or invalid, from the file names and sizes.
runtimeloaded after the gateway loads the model. unavailable after a load attempt failed or found no model. idle before the first use.
reason, fallbackPresent only when the model is not usable. fallback gives the behavior that the gateway uses.

Troubleshooting​

SymptomCauseFix
unknown model <name>; use one of: embedding, ocrThe model name is not correct.Use embedding or ocr.
model <model>: <file> does not match its pinned SHA-256 digest <digest>A downloaded or copied file is different from the pinned file.Download the file again from the URL that verdictan models list prints.
model <model>: <file> has <n> bytes, expected <m>The file is incomplete or is a different version.Download the file again.
model <model>: download <url>: ...The host cannot get the file.Make sure that the host can reach the URL, or use --from <DIR>.
/readyz shows not_installed after an installThe gateway uses a different data directory.Install the model as the gateway user with the VERDICTAN_DATA_DIR of the gateway.
/readyz shows runtime: unavailable for an installed modelThe model did not load. The gateway log has the error.Run verdictan models status, install the model again if necessary, and restart the gateway.

Next steps​