Embedding Detector

embedding-detector compares each sentence of a request with reference texts, such as a description of your trade secrets, and flags a sentence that is similar enough. It has built-in references for identifiers, card data, health data, and injection, and you can add your own.

The default local backend compares shared words. It does not recognize synonyms or meaning. An external embedding endpoint gives a different similarity measure, but it receives the request text.

Configuration

policy-config.yaml

pack:
  name: embedding-detector-example
  version: 1.0.0
  enabled: true
policies:
  chain:
    - embedding-detector
policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.75
    action: block
    categories:
      - label: trade_secret
        reference_text: >-
          proprietary manufacturing process internal formula secret recipe
      - label: competitive_intel
        reference_text: >-
          competitor pricing strategy acquisition target market positioning

Verify the local comparison

Test with the first reference sentence and a greeting:

Add to policy-config.yaml

testing:
  suites:
    - name: embedding-detector
      cases:
        - name: blocks-reference-sentence
          input: {messages: [{role: user, content: "proprietary manufacturing process internal formula secret recipe"}]}
          expected: {verdict: block, reason_code: embedding.other}
        - name: allows-greeting
          input: {messages: [{role: user, content: "Hello, how are you?"}]}
          expected: {verdict: allow, reason_code: ok}

Run verdictan policy test --json. Both cases pass: the reference sentence shares every word and is blocked, and the greeting shares none.

Now test a paraphrase from your own application. Word overlap can miss it, so a low score does not prove that content is safe.

Fields

Field	Type	Default	Description
backend	local or external	local	local scores word overlap in the gateway. external gets embeddings from endpoint.
endpoint	string	http://localhost:8080/embed	Embeddings endpoint, for external only
model	string	all-MiniLM-L6-v2	Model name in the embeddings request, for external only
api_key	string		Inline bearer token for the endpoint. A value with spaces or control characters is ignored.
similarity_threshold	number	0.70	Cosine similarity at which a sentence counts as a detection, from 0 to 1
timeout_ms	integer	20	Timeout of one embeddings request, from 1 to 60000
action	redact or block	redact	Verdict on a detection
categories	list of objects	[]	Your own categories, added to the built-in ones
categories[].label	string		Category name. Required.
categories[].reference_text	string		Text to compare with. Required.

Built-in categories

These always apply. categories adds to them. Each line gives the label, the reason code, and the reference text.

• social_security_number, embedding.pii: social security number SSN taxpayer identification
• credit_card, embedding.pan: credit card number payment card PAN Visa Mastercard
• medical_record, embedding.phi: medical record number MRN patient identifier health record
• biometric_data, embedding.phi: fingerprint biometric scan retinal iris facial recognition
• financial_account, embedding.pii: bank account routing number IBAN swift wire transfer
• prompt_injection, embedding.other: ignore previous instructions disregard system prompt override
• export_controlled, embedding.other: ITAR EAR export controlled technical data defense article

Your own categories use embedding.other.

How it works

1. The gateway joins the text of all request messages.
2. The control splits it into sentences at ., !, ?, and line breaks, and skips sentences of three characters or fewer.
3. It scores each sentence against each category with cosine similarity.
4. Local backend: lowercase, split into alphanumeric words, and compare word frequency vectors.
5. External backend: one POST with {"input": "<text>", "model": "<model>"} for the sentence and one for each reference. It reads data[0].embedding from each response.
6. A score at or above similarity_threshold is a detection. The first detection sets the reason code.
7. No detection: allow with embedding.clean. A detection: block or redact, following action.

The details contain detection_count, first_kind, and first_confidence: high at 0.90 or more, medium at 0.80 or more, otherwise low.

Phase and verdicts

Item	Value
Stages	pre-request (default), pre-response
Request families	All
Verdicts	allow, redact, block
Reason codes	embedding.clean, embedding.pii, embedding.pan, embedding.phi, embedding.other

Examples

Local backend, blocking detections:

Inside policy-config.yaml

policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.8
    action: block

External endpoint without a credential:

Inside policy-config.yaml

policy:
  embedding-detector:
    backend: external
    endpoint: https://embeddings.example.com/v1/embeddings
    model: replace-with-embedding-model
    timeout_ms: 200
    similarity_threshold: 0.8
    action: block

Limits and failure behavior

• Local scoring counts shared words only. A sentence must share most of its words with a short reference to reach 0.70.
• If an external request fails, times out, or returns a non-2xx status, the control falls back to local scoring. The decision event does not say which backend scored.
• The default timeout_ms: 20 is too short for most network endpoints. Raise it for external.
• The external backend sends one request per sentence plus one per category (seven built-in plus yours). Long prompts cause many requests.
• The request text goes to the external endpoint. Use only an endpoint that your data policy allows.
• api_key is an inline string. secret_key_ref and environment references are not supported for it.
• Default builds include the external backend. A build with --no-default-features accepts external but always scores locally.
• A block returns HTTP 400 with error code content_policy_violation. redact starts the shared PII redactor and does not remove the matching sentence. Use block to stop the text.

Next steps

• DLP Filter: exact terms and regular expressions.
• Prompt Injection Detection: the dedicated injection control.
• External Moderation: classification by a moderation provider.