Content signals policy

Cloudflare's Content Signals Policy (September 2025, released under CC0) adds a machine-readable Content-Signal line to robots.txt carrying three preferences: search (building a search index and returning links), ai-input (putting content into AI models, e.g. retrieval-augmented generation) and ai-train (training or fine-tuning models). Cloudflare serves Content-Signal: search=yes, ai-train=no, and no ai-input signal, in the managed robots.txt of the over 3.8 million domains that had turned that feature on. Signals are preferences, not enforcement; Cloudflare advises pairing them with WAF rules and Bot Management. The IETF AIPREF working group is chartered to standardise a vocabulary for AI-usage preferences and how to attach it through robots.txt and HTTP headers.