Skip to content

The retrieval simulation

Answer engines do not read pages; they retrieve passages. Vercatus builds the same shape of pipeline over your own content so you can see which questions it fails on.

Each page’s main content is split into heading-aware passages of a few hundred tokens. Passages are indexed twice: lexically (a term-frequency index with inverse document frequency, Unicode-aware, character bigrams for CJK) and densely (a local multilingual embedding model), and results are fused by reciprocal rank. Both indexes live in the database, so a run of ten thousand pages retrieves as cheaply as one of ten. This is the hybrid shape production retrieval stacks use, and it fails the same way they do: passages that depend on context, restate a subject with a pronoun, or bury the answer in the fourth paragraph do not retrieve.

The questions the site should win. With analysis agents on, it is generated from the site’s own profile and each page’s answerable questions, deliberately including comparison, alternative, troubleshooting and sub-question intents that sites rarely write headings for. Without agents, it is derived from headings and titles — enough to run the simulation, weaker as a test.

Each intent is retrieved. With agents on, a model answers from the retrieved passages only and judges: was the question fully answered, was every statement grounded in the passages, what was missing, and did two pages supply the same support (cannibalisation). Without agents, the verdict comes from retrieval scores alone and is coarser.

  • Coverage by intent class — where the site systematically cannot answer (commercial and comparison questions are the usual gaps).
  • Gaps — for each partial or unanswered intent, what a complete answer needed. This is the content roadmap, derived from retrieval rather than from a keyword tool.
  • Cannibalisation — intents where several of the site’s own pages compete with near-equal passages.
  • Never retrieved — content pages that no intent reaches, which usually means their passages are not written in a way retrieval can match to a question.
  • Cross-language leakage — a query in one language answered by another language’s page although the site publishes in the query’s language.

It is a model of retrieval, not a measurement of any live engine. Real engines add ranking signals, freshness and source-credibility weighting that this simulation does not. Treat verdicts as predictions to be confirmed against real-engine citation tracking, not as observations.