Good for Bots

Website owners increasingly need to control how AI crawlers access their content. Good for Bots addresses this by offering a free scanner that measures a site's technical readiness for AI agents, then delivering a public directory of results that lets both owners and potential AI applications understand what content is accessible.

The scanner operates transparently, checking the files that crawlers read first: robots.txt rules, llms.txt declarations, sitemaps, and raw HTML. It evaluates structured data, Markdown rendering, and overall crawler access, then assigns a score out of 100. The product doesn't promise anything vague—the founder explicitly notes that the scan measures technical infrastructure, not rankings or mentions in any AI assistant. This honesty is rare and credible.

What distinguishes Good for Bots is its integration with modern development workflows. Every fix recommendation includes a prompt that developers can feed directly to Claude Code, Cursor, or Codex. Rather than asking website owners to manually implement scoring rubrics, the tool supplies ready-to-use prompts that coding agents can interpret and apply. This bridges the gap between measurement and remediation, turning a diagnostic tool into something actionable.

The free tier is generous. Anyone can scan a site, see the detailed findings, view existing reports in the directory, and access the interactive score breakdown showing what changed between scans. The free plan also grants a dofollow backlink once the live badge appears on the scanned website—a meaningful incentive for participation without forcing payment.

The Pro tier ($20 per month for featured directory listings, with the monitoring product separate) adds weekly rescans and email notifications when scores drop or bot access changes. The Pro reports maintain permanent scan history and include prompts for every fix, not just the highest-impact ones. The product clearly separates concerns: the scanning and scoring are identical whether free or Pro, but the monitoring and convenience features require the paid plan.

The directory itself fills a gap in the current landscape. As more websites think strategically about AI crawler access, having a categorized registry of how sites actually perform—searchable by platform and detected standards—provides transparency that individual scans alone cannot. It creates a public artifact of technical readiness and turns compliance or optimization into a visible, comparable metric.