ResearchAll sizesGlobal
4.5%
of desktop websites name OpenAI's GPTBot in robots.txt, up from 2.9% in 2024
Sample: HTTP Archive crawl of millions of home pages, desktop and mobile
HTTP Archive, Web Almanac (SEO chapter), 2025 crawl
ResearchUpdated 23 Oct 2026
39.6%of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast)
39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast) (2025 crawl, HTTP Archive, Web Almanac (SEO chapter))
ResearchChecked at source, 23 Oct 2026All sizesGlobalAI search and zero-click · AI crawlers
In plain words
What this number really says
39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast) (2025 crawl).
39.6%
the latest figure (2025 crawl)
Research
type of source
Why you can trust it. HTTP Archive, Web Almanac (SEO chapter): research, research firm or analyst compilation.
Key facts
- Period 2025 crawl
- Region Global
- Company size All sizes
- Next data Yearly
Source: HTTP Archive, Web Almanac (SEO chapter), 2025 crawl
Method and use
How it was measured, and how to use it
How it was measured
- Publisher
- HTTP Archive, Web Almanac (SEO chapter)
- Source type
- Research: Research firm or analyst compilation
- What it measures
- 39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast)
- Data period
- 2025 crawl (ends 31 Jul 2025)
- Published by the source
- 15 Jan 2026
- In the source's words
39.6% of llms.txt files are related to All in One SEO
- Checked at source
- 23 Oct 2026: Read on almanac.httparchive.org, Web Almanac 2025 SEO chapter (published 15 Jan 2026, updated 9 Jun 2026). 2025 crawl. Detected from default comments the plugins leave in the files.
- Reuse
- Web Almanac content is open (licence not re-checked this session). Cite the Web Almanac chapter and year.
- Next expected update
- Yearly (HTTP Archive, yearly)
- Original page
- almanac.httparchive.org: Web Almanac (SEO chapter)
- Data files
- CSV · JSON (stable links)
Full change log
No changes recorded yet. First checked at the source on 23 Oct 2026.
Cite this number
APA
HTTP Archive. (2026). 39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast). Via digitaltreed, https://digitaltreed.com/statistics/ai-search/llms-txt-files-from-aioseo
MLA
"39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast)." HTTP Archive, Web Almanac (SEO chapter), 2025 crawl. digitaltreed, https://digitaltreed.com/statistics/ai-search/llms-txt-files-from-aioseo.
HTML link
<a href="https://digitaltreed.com/statistics/ai-search/llms-txt-files-from-aioseo">39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast)</a> (HTTP Archive, 2025 crawl)
Embed the chart
This stat has no chart to embed yet. Link to this page instead: https://digitaltreed.com/statistics/ai-search/llms-txt-files-from-aioseo
FAQ
Questions about this number
What is the latest figure for AI crawlers (Global)?
39.6% of llms.txt files found on the web were made by the All in One SEO plugin (3.6% by Yoast), for 2025 crawl, according to HTTP Archive, Web Almanac (SEO chapter). We checked it at the source on 23 Oct 2026.
Put this number to work
We help small and mid-size firms turn data like this into useful content, answers for customers and visibility in AI search.
Every number on this page was checked at its original source. Free to cite with a link to this page.