Generate the three files that define how AI systems see your site — a content map, crawler permissions, and per-page directives.
llms.txt at https://yourdomain.com/llms.txt. Switch to Full for llms-full.txt — the same structure with complete page text for richer AI context. Spec →
/robots.txt. In Custom mode, each blocked bot gets its own path — use /blog to block only that section, / to block everything.
noai — tells compliant training crawlers to exclude your text from datasets.noimageai — tells image-scraping AI crawlers not to use your visuals for model training.<link rel="llms"> tag pointing to your llms.txt so AI assistants can discover it automatically.<head> on every page you want to control. Meta tags are hints — robots.txt is the enforcement layer.
A structured markdown index of your site's pages — helps AI assistants understand what you cover and cite you accurately. Also generates llms-full.txt with complete page text.
Per-crawler directives for 14 major AI bots. Allow all, block all, or pick exactly who gets in — with per-bot path control in Custom mode.
HTML <head> snippets using noai and noimageai to opt pages out of AI training datasets.
Ask me about your projects, reports, brand mentions, backlinks, or anything on the platform.