Technical AEO comes down to one question: can AI systems find, fetch, read and understand your most important pages? This checklist walks through every layer, from crawler access and rendering through indexing, structured data, architecture, duplicates and performance, with the most common failures first. Work through it for your priority pages, then expand.
Before you start: list your priority pages
Audit these first:
- Homepage and about page (your entity home)
- Pricing and plans
- Product and feature pages
- Use-case and solution pages
- Comparison and alternatives pages
- Top 10 to 20 guides and blog posts
- Key documentation and help articles
1. Crawler access
- robots.txt allows AI search crawlers: OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, Bingbot, Applebot
- robots.txt allows user-triggered fetchers: ChatGPT-User, Claude-User, Perplexity-User
- Training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended) are set according to a deliberate policy. See should you block AI training crawlers.
- Each subdomain (docs, blog, help) has its own correct robots.txt
- CDN and firewall bot rules don't block or challenge verified AI crawlers
- Rate limits don't return 429s to AI crawlers on normal crawl volumes
- Curl with AI user agents returns 200 and full content for priority pages
Reference: AI crawlers and robots.txt guide.
2. Rendering
- Priority page content is present in the raw HTML (view source, curl)
- Prices, plan details and key facts aren't loaded only by JavaScript
- FAQ and accordion content is in the initial HTML
- Title, meta description, canonical and JSON-LD are in the server HTML
- Deep URLs return full HTML, not an empty app shell
Reference: JavaScript rendering and AI crawlers.
3. Indexing
- Site verified in Google Search Console and Bing Webmaster Tools
- XML sitemaps submitted to both, containing only canonical, indexable URLs
- Sitemap referenced in robots.txt
- Priority pages indexed in Google and Bing (URL inspection)
- No unintended
noindexornosnippeton priority pages - IndexNow configured for Bing and participating engines
Reference: Bing Webmaster Tools and IndexNow.
4. Structured data
- Organization schema on the entity home, with stable @id and sameAs links
- Product, Offer or SoftwareApplication schema on product pages
- Article or BlogPosting schema with author and accurate dates on content
- BreadcrumbList on templated pages
- LocalBusiness schema on location pages (if relevant)
- All markup matches visible content and validates without errors
Reference: schema markup for AEO.
5. Site architecture
- Priority pages are within about three clicks of the homepage
- Navigation uses real links in server-rendered HTML
- Priority pages have descriptive internal links pointing to them
- No orphan pages among priority content
- Descriptive, stable URLs
Reference: site architecture for AI retrieval.
6. Duplicates and conflicts
- One canonical host, protocol and trailing-slash convention, with redirects
- Canonical tags on parameterized and filtered URLs
- Retired pages (old pricing, discontinued products) redirected
- No two live pages with conflicting prices, limits or claims
- Staging environments not indexable
Reference: canonicals and duplicate content.
7. Performance and reliability
- Fast server response for priority pages from major regions
- No intermittent 5xx errors or timeouts in logs
- Main content readable without dismissing interstitials
- Core Web Vitals in good ranges for key templates
Reference: page speed and AI fetchers.
8. Accessibility and agent-readiness
- Logical headings, labeled forms and buttons, descriptive link text
- Alt text on meaningful images and charts
- Key flows (demo request, sign-up, checkout) work with keyboard alone
Reference: AI agents and agentic browsing.
9. Content accessibility
- Key findings and guides aren't fully gated
- Docs needed by prospects are public
Reference: gated content and AI visibility.
10. Monitoring
- Server or CDN logs reviewed monthly for AI crawler activity and errors
- AI referral traffic tracked as a channel in analytics
- Prompt set tracked across AI assistants
- Alerts for robots.txt changes and spikes in bot errors
References: AI crawler log analysis, AI visibility metrics.
Prioritizing fixes
| Priority | Issue type | Why |
|---|---|---|
| 1 | Crawlers blocked; content not in HTML | Makes pages invisible to AI |
| 2 | Priority pages not indexed in Google or Bing | Removes them from retrieval |
| 3 | Conflicting duplicates | Causes inaccurate answers |
| 4 | Missing Organization and Product schema | Weakens entity understanding |
| 5 | Architecture and internal links | Affects discovery and freshness |
| 6 | Performance and accessibility | Affects reliability and agents |
When to re-audit
- After any redesign or migration
- After changing CDN, firewall or bot protection settings
- After major CMS or framework upgrades
- At least twice a year
The bottom line
Run this checklist on your priority pages, fix access and rendering first, then indexing, structured data, architecture, duplicates and performance. A technically sound site doesn't guarantee AI citations, but a technically broken one almost guarantees you won't get them.
