seo-ai-crawlers

Audit AI crawler access and Google AI-feature eligibility for a page — check the snippet controls that actually gate AI Overviews and AI Mode (noindex, nosnippet, max-snippet, data-nosnippet), resolve per-bot effective access from robots.txt for the audited path (training vs retrieval vs user-fetch tokens), read Content-Signal posture, diff what different user-agents are served, confirm non-JS crawlers can read the…