Google says llms.txt does nothing for Search — here's what it's actually for

Search Central made it official in June 2026

Google's AI optimisation guide, updated June 15 2026, now states directly: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." The update was added, in Google's words, "to address questions from the community" — a diplomatic way of saying the hype had outrun the facts. John Mueller had flagged the same thing on Reddit months earlier, comparing llms.txt to the keywords meta tag: a file that describes what a site claims to be about, when Google would rather just read the site. Gary Illyes confirmed in July 2025 that Google does not support it and has no plans to.

Widely deployed, rarely read, zero measurable effect

SE Ranking analysed roughly 300,000 domains and found ~10% had adopted the file — but when they ran both a statistical model and an XGBoost classifier to check whether the file correlated with AI citations, the result was flat: no effect, and removing it actually improved the model's accuracy. Adoption was nearly identical across traffic tiers (low-traffic sites: 9.88%, mid: 10.54%, high: 8.27%) — meaning it is not something high-performers are doing and others are copying. Ahrefs found a similar picture from the read side: of 137,210 domains they tracked (a technical-skew sample, so an upper bound on true adoption), 97% saw zero requests in May 2026. Of the 3% that were fetched, only 19.5% of those requests came from named AI tools — GPTBot first, Claude Code second. The Chrome Lighthouse audit accounted for roughly 1 in every 1,000 fetches.

Limy.AI's 90-day server-log study put the sheer scale in perspective: out of more than 500 million AI bot events, roughly 408 targeted llms.txt. Otterly.AI's parallel test found just 84 of 62,100 AI bot visits — 0.1% — hit the file at all. These are not rounding errors; they are the actual signal from the actual logs.

Coding agents and agentic browsers — not Search crawlers

The companies shipping llms.txt are not doing it to move rankings. They are doing it because a specific and growing class of tools reads it by convention: Cursor, Windsurf, Claude Code, GitHub Copilot, Cline and Aider all look for /llms.txt and /llms-full.txt as a navigation hint when a developer asks the agent to "use the Stripe docs" or "query the Vercel API." LangChain's open-source mcpdoc MCP server exposes llms.txt as a first-class data source. For these agents, the file is what a sitemap is for Googlebot: a structured index of where the useful content lives, so the agent does not have to crawl and guess. Chrome's Lighthouse 13.3 — released May 7 2026 — moved llms.txt auditing out of experimental status and into the default "Agentic Browsing" category, pointing in the opposite direction from the Search team. It is a genuine split inside Google: Search says skip it, Chrome says audit it.

The honest nuances

A few things worth holding alongside the headline numbers:

The Chrome/Search split is real. Lighthouse 13.3 audits llms.txt under "Agentic Browsing" while Search Central says to skip it. Neither team is wrong on their own terms — they are optimising for different crawlers and use cases. If agentic browsing grows as a traffic source, the Lighthouse team may end up on the right side of history.

Mueller's gameability critique has merit. A file that tells AI what a site claims to be about — without any verification — is structurally similar to the keywords meta tag. No major AI vendor (OpenAI, Anthropic, Google, Meta, Mistral) has committed to it as a citation signal in production. Until one does, treating it as a GEO lever is speculation.

One early case was directional, not proof. A single self-reported proof-of-concept (dev5310, February 2026) found that after submitting llms.txt to Search Console, Google AI Mode cited it as a top source within 24 hours — while tracked keyword rankings did not move. This is directional and interesting, but it is n=1, self-reported, and not a reproducible study. We note it for completeness, not as evidence of a broad effect.

The cost of shipping it is low. Half a day to draft, no ongoing maintenance, no risk of penalties. That makes it a cheap asymmetric bet: if agent traffic grows and the file matters, you are ready; if it stays noise, you have lost almost nothing. The framing to use is agent-readiness infrastructure — not a ranking or citation lever.

"Comparable to the keywords meta tag"

John Mueller's Reddit comment is the clearest framing from inside Google: "AFAIK none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it). To me, it's comparable to the keywords meta tag — this is what a site-owner claims their site is about… why not just check the site directly?" The analogy is exact: the keywords meta tag was ignored because site owners abused it; llms.txt carries the same structural risk. Gary Illyes, separately, confirmed in July 2025 that Google does not support it and is not planning to. The June 2026 docs are not a reversal of a prior position — they are the position, made official.

Build it for agents, not for Google

llms.txt is not a search-ranking tool. The data from Ahrefs, SE Ranking, Limy.AI and Otterly.AI all agree: it has no measurable effect on AI citations in the products most people think of as "AI search." Google has now put that in writing. What it is is a navigation convention for the growing layer of coding agents and agentic browsers that crawl your docs when a developer asks them a question. Stripe and Vercel ship it because their primary audience increasingly includes developers working inside Cursor and Claude Code. If yours does too, it is worth the half-day to write one. If your audience is consumers finding you through Google or Perplexity, invest that time in something the models actually use: high-quality, well-structured content that earns citations on its own.

Generate your llms.txt with Baseline