Citation Verifier

citation-verifier compares a model answer with the source text that your application sends in the request, and can block an answer that is not grounded in it. It also finds citations in the answer, such as quotes, DOIs, and URLs, and can look some of them up.

It is not a fact checker. It measures word overlap: an incorrect claim can overlap well, and a correct paraphrase can score low.

It runs on Chat Completions and Responses with stream: false.

Configuration

This configuration checks the answer against the supplied context only, without external lookups. Apply it to a test gateway with the config-first workflow.

policy-config.yaml

pack:
  name: citation-verifier-example
  version: 1.0.0
  enabled: true
policies:
  chain:
    - citation-verifier
policy:
  citation-verifier:
    require_sources: true
    require_source_match: true
    min_confidence: 0.8
    min_groundedness: 0.8
    extract_patterns:
      - quote
    rag_context:
      verify_against_context: true
      min_context_overlap: 0.7
    output_action:
      unverified_action: block
    response:
      include_verification_report: true

Supply the text to verify against

Put the source text in verdictan.context_documents of the request. This body gives the same source to the model and to the verifier:

request.json

{
  "model": "gpt-4o-mini",
  "stream": false,
  "messages": [
    {"role": "system", "content": "Return only this exact quoted sentence: \"The museum opens at nine.\""},
    {"role": "user", "content": "When does the museum open?"}
  ],
  "verdictan": {
    "context_documents": [{"content": "The museum opens at nine."}]
  }
}

Send it with the URL and application token from the quickstart:

curl --fail-with-body "$VERDICTAN_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $VERDICTAN_CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @request.json

The nested result shows citation.verified. Now change only the context document to an unrelated sentence and send it again. The result is citation.unverified when the overlap falls below the threshold.

Model wording can vary between calls. For a repeatable test, use a provider fixture that always returns the quoted sentence.

Fields

Field	Type	Default	Description
require_sources	boolean	true	The answer must contain source evidence
require_source_match	boolean	true	With require_sources, a citation alone is not enough: a quote must match the context, the overlap must reach min_context_overlap, or an external lookup must succeed.
min_confidence	number	0.8	Minimum share of claims whose overlap reaches min_groundedness
min_groundedness	number	0.8	Minimum word overlap of one claim. Also the fallback for rag_context.min_context_overlap.
extract_patterns	string[]	case_law, academic, url, quote, statistic, regulatory	Citation types to find. academic and url also turn on external lookups. Lint accepts only the first five.
rag_context.verify_against_context	boolean	true	Compares the answer with the context documents of the request
rag_context.min_context_overlap	number	0.7	Minimum word overlap of the whole answer with the context
output_action.unverified_action	string	flag	flag, redact, or block. Only block changes the verdict. redact does not change the answer.
response.include_verification_report	boolean	true	With false, the details contain only verified and groundedness.

Lint also accepts external_resolver and output_action.hallucination_action. The runtime ignores them.

How it works

1. The control reads the answer from the provider response, and the verdictan.context_documents[].content strings from the request.
2. With verify_against_context, it calculates the groundedness: the share of unique answer words that also appear in the context.
3. It splits the answer into claims at ., !, ?, and line breaks, and scores each claim against the context. A claim passes at min_groundedness or above. The claim confidence is the share of claims that pass.
4. It extracts DOIs, PubMed IDs, URLs, case law citations, regulatory references, and quotes, as extract_patterns selects.
5. With academic, it looks up each DOI at api.crossref.org and each PubMed ID at eutils.ncbi.nlm.nih.gov. With url, it sends a HEAD and then a GET to each URL. Each call has a 5-second timeout.
6. It checks the format of case law citations. It does not query a legal database.
7. It decides whether the answer is verified:
   • An empty answer passes.
   • Without context checks, or without context documents, the answer passes when require_sources is false, when source evidence exists, or when require_source_match is false.
   • With context documents, the overlap must reach min_context_overlap, the claim confidence must reach min_confidence, and the source rule must pass.
8. A verified answer is allowed with citation.verified.
9. An unverified answer is blocked with citation.unverified when unverified_action is block. Otherwise, it is allowed with the same reason code.

The details contain verified, groundedness, claim_confidence, claims, quotes, citations, and external_lookup. A verified answer adds a verification_report. An unverified answer adds unverified_claims and unverified_claim_count.

Phase and verdicts

Item	Value
Phase	output
Enforced path	Chat Completions and Responses with stream: false
Verdicts	allow, block
Reason codes	citation.verified, citation.unverified

A block returns HTTP 400 with error code content_policy_violation and a verdictan.decision object with verdict BLOCK.

Limits and failure behavior

• With stream: true, the gateway buffers the response but does not run this control.
• Groundedness counts shared words. A paraphrase of the context scores lower than an answer that reuses its words.
• With the default extract_patterns, the gateway calls Crossref, PubMed, and every URL in the answer. Remove academic and url if the gateway must not call external services. A failed or slow lookup marks that citation as not verified. It does not block the request by itself.
• The runtime default includes regulatory, which lint rejects. Leave extract_patterns out to keep the runtime default.
• flag and redact behave the same: the answer goes out unchanged with an allow verdict.
• There is no external resolver at runtime. Case law citations get a format check only.
• With the defaults and no context documents, an answer without a citation or verified source fails. Set require_sources: false on routes that do not use citations.

Starter packs that include this control are in the policy templates hub.

Next steps

• Quality Scorer: faithfulness and relevancy thresholds.
• Quality Assertion Configuration: add assertions next to grounding checks.