How to Set Up llms.txt and robots.txt for AI Crawlers on WordPress (2026 Guide)

The guide shows five steps to make WordPress content reachable by AI crawlers: edit robots.txt, disable Cloudflare Bot Fight Mode, create llms.txt, add FAQPage schema, and add Article author schema.

It explains how to allow GPTBot, PerplexityBot, and ClaudeBot in robots.txt and how to edit that file via WPCode, Yoast, or Rank Math.

It instructs disabling Cloudflare Bot Fight Mode (or allowing definite automated bots) and verifying crawler requests in server logs.

It recommends creating a llms.txt in the site root describing key pages and adding JSON-LD FAQPage and Article schema, then testing with Google's Rich Results Test.