perplexitybot

Eight SEO and GEO agent skills for WordPress: AI crawler log forensics, page citability scoring, brand visibility in AI answers, robots.txt and llms.txt governance, migration redirects, SEO plugin driving, and agency portfolio reporting. Every analysis ends in a self-contained HTML report.

The best AEO/GEO skill for Claude Code. Audit, fix, and monitor your website's visibility across ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews. 100+ optimization techniques, 33 evidence collectors, 4-vector GEO Score. Built on Princeton KDD 2024 paper. Free, MIT.

See which AI crawlers can read your site — GPTBot, ClaudeBot, PerplexityBot & 20 more. Curated, operator-sourced bot list + a zero-dep CLI and GitHub Action to audit robots.txt, test reachability, read access logs, and gate it all in CI.

Validates whether GPTBot, ClaudeBot, PerplexityBot and other LLM crawlers are allowed by a robots.txt file.

Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples

Agent-native multi-bot web crawler simulator. See your site through the eyes of Googlebot, GPTBot, ClaudeBot, and PerplexityBot.

Visual robots.txt manager for the AI era. Block or allow GPTBot, ClaudeBot, PerplexityBot, and 20+ AI crawlers with one click.

Four free MCP tools to check whether AI engines can find, read and recognise a website. No account, no API key, no LLM calls.

Check whether AI crawlers can access a URL (robots.txt) plus page legibility signals. Zero dependency CLI and library.

21 AI crawler user-agents we detect, and which 14 of them actually reached one real site over 74 days. First-party detection list plus first-party access logs. Not observed is written "not observed", never 0.

User-agent simulator for major LLM crawlers: measures response code, content length, and JS-dependency

Record which AI crawlers read your pages. Forwards GPTBot, ClaudeBot, PerplexityBot and friends to rankveo, and nothing else.

AI Observatory — verified AI and search crawler retrieval monitoring for business websites

Free no-tracking browser utility for drafting and auditing AI crawler robots.txt rules.

Machine-readable dataset of AI crawler user-agents — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, and more. Includes robots.txt recipes and weekly verification.

robots.txt for AI crawlers: copy-paste templates that allow AI search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) and let you block AI training (GPTBot, ClaudeBot, Google-Extended), a verified AI crawler list, an llms.txt template and access tests. For generative engine optimization (GEO) and AI SEO.

Audit robots.txt for sitemap hygiene, crawl mistakes, and explicit AI crawler policy.

Open readers behind CrawlCheck, the verification layer for the agentic web: robots.txt policy per crawler and agent, user-agent group shadowing detection, and request verification against operators' published IP feeds.