Allow-list the crawler
Every Vercatus request carries an RFC 9421 HTTP Message Signature with the web-bot-auth tag and a
Signature-Agent header naming the key directory. That is the part worth trusting: user-agent strings
can be sent by anyone.
Cloudflare
Section titled “Cloudflare”Add a WAF custom rule with action Skip (bot management, managed challenge) for requests where the
Signature-Agent header equals "https://vercatus.com". If your zone uses Verified Bots, the signed
requests validate against the published directory without a rule.
Akamai, Fastly, self-managed
Section titled “Akamai, Fastly, self-managed”Verify the Signature-Input / Signature headers against the Ed25519 keys published at
https://vercatus.com/.well-known/http-message-signatures-directory, and exempt verified requests from
challenges. If signature verification is not yet available on your platform, match the user agent
Vercatus/1.0 and the source addresses published in the same directory document.
What the identity probes look like
Section titled “What the identity probes look like”To measure how your site treats other crawlers, Vercatus also sends a small number of requests carrying the published user agents of Googlebot, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Google-Extended. These are always signed as Vercatus, so they are distinguishable from the real crawlers and from spoofing. Allow-listing by signature covers them; allow-listing by user agent does not, and that is intended — the probes exist precisely to see what an unallow-listed identity receives.
At most two requests per second per host, with automatic backoff on 429 and 5xx. Lower on request.
See also the public crawler page, written for security teams.