Markdown and tables
Many AI fetchers read a page as extracted text, not as a rendered screen. Anthropic's web fetch tool retrieves a URL's full text content, accepts only text, HTML and PDF, and does not support websites dynamically rendered with JavaScript. Trafilatura, an extraction library its documentation says is used by Hugging Face, IBM, NVIDIA, the Allen Institute for AI and others, drops recurring boilerplate, keeps paragraphs, titles, lists, quotes and code (tables optionally) and outputs text or Markdown. Content that exists only in images, canvas or client-rendered widgets therefore does not reach such pipelines, while HTML headings, lists and tables carry their structure through.
Sources
- 1
- 2
CGrade C: reputable industry data or observation. A grade describes the provenance of the sources above; it is not a probability that a change will work. The effect basis says how the size of the effect is known: measured by a study, taken from an industry report, only documented without a size, or not quantified.
Put this evidence to work
Related entries
More in Retrieval, or search the whole base.
All Retrieval entries