Technical AEO

    JavaScript Rendering and AI Crawlers

    Most AI crawlers don't run JavaScript, so client-rendered content can be invisible to them. How to test your site and fix it with SSR or prerendering.

    Saarthak Srivastav

    Saarthak Srivastav

    · 4 min read

    Key takeaways

    • Googlebot renders JavaScript, but most AI crawlers fetch only the raw HTML, so client-rendered content can be invisible to them.
    • Single-page apps that ship an empty root element are the most common cause of AI invisibility on modern sites.
    • Test by fetching pages with curl or with JavaScript disabled and checking whether the important text is present.
    • Fix with server-side rendering, static generation or prerendering of the pages that matter.
    • Also check metadata and structured data: if they're injected by JavaScript, crawlers may miss them too.

    Most AI crawlers read only the HTML your server sends; they don't run JavaScript. If your site builds its content in the browser, as many single-page apps do, those crawlers may see an empty page. Googlebot is the main exception because it renders JavaScript, which is why a site can rank in Google and still be invisible to ChatGPT, Claude and Perplexity. This guide explains how to test your site and the options for fixing it.

    Why this matters more for AI than for Google

    Crawler Renders JavaScript?
    Googlebot Yes, with a modern rendering engine
    Bingbot Partially
    OAI-SearchBot, GPTBot, ChatGPT-User Generally read raw HTML
    Claude-SearchBot, ClaudeBot, Claude-User Generally read raw HTML
    PerplexityBot, Perplexity-User Generally read raw HTML

    Rendering is expensive at web scale, so most AI crawlers skip it. Behavior can change, but the safe assumption is: if it isn't in the initial HTML, AI may not see it.

    What goes missing

    On a client-rendered site, the raw HTML often contains little more than:

    <body>
      <div id="root"></div>
      <script src="/assets/app.js"></script>
    </body>
    

    Everything a person sees, from headings and product descriptions to prices, FAQs and article text, is added later by JavaScript. Common casualties:

    • Entire pages on single-page apps (React, Vue, Angular without server rendering)
    • Prices and plan details loaded from an API
    • Reviews and ratings loaded by a widget
    • Tabs and accordions that fetch content on click
    • Help center articles rendered by an embedded widget
    • Meta tags, canonical tags and structured data injected by scripts

    How to test

    1. View source, not the inspector

    In your browser, use "View page source" (not the developer tools Elements panel, which shows the rendered page). Search for a sentence from the page. If it's missing, it's rendered client-side.

    2. Fetch with curl

    curl -s https://www.example.com/pricing | grep -i "per month"
    

    Try it with an AI crawler user agent too:

    curl -s -A "Mozilla/5.0 (compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" https://www.example.com/pricing | grep -i "per month"
    

    3. Disable JavaScript

    Turn off JavaScript in your browser and load the page. What you see is roughly what a non-rendering crawler gets.

    4. Ask an assistant

    Ask ChatGPT, Claude or Perplexity to summarize a specific URL. If the summary is empty or generic, rendering may be the problem.

    How to fix it

    Option 1: Server-side rendering (SSR)

    The server generates full HTML for each request, and JavaScript then makes it interactive. Frameworks such as Next.js, Nuxt, Remix and SvelteKit support SSR. Best for dynamic pages whose content changes per request.

    Option 2: Static site generation (SSG)

    HTML is generated at build time for each page. Excellent for marketing pages, blogs and documentation. Fast and simple to host.

    Option 3: Prerendering

    For existing single-page apps, prerender the important routes to static HTML at build time, or serve prerendered snapshots to crawlers. This is often the quickest fix when a full migration isn't practical. Make sure crawlers and users get the same content.

    Option 4: Hybrid

    Render the content-critical parts on the server, and keep highly interactive parts (dashboards, configurators) client-side. Most sites only need marketing, product, pricing, blog and docs pages to be server-rendered.

    Don't forget the head

    Even with server-rendered body content, check that these are in the initial HTML:

    • <title> and meta description
    • Canonical tag
    • Open Graph tags
    • Structured data (JSON-LD). See schema markup for AEO.
    • hreflang tags for international sites

    If a script injects them after load, many crawlers won't see them.

    Prioritize the pages that matter

    You don't need to server-render everything. Start with:

    1. Pricing and plans
    2. Product and feature pages
    3. Comparison and use-case pages
    4. Blog posts and guides
    5. Documentation and help articles
    6. About page and other entity pages

    These are the pages AI assistants retrieve for buying and how-to questions. See the technical AEO audit checklist.

    Common pitfalls

    • Lazy-loaded text that only appears on scroll
    • Content behind client-side routing with no server response for deep URLs (they return the same empty shell)
    • Hydration errors that break the page after load
    • Different content for crawlers and users, which can be treated as cloaking
    • Blocking JavaScript and CSS files in robots.txt, which hurts Googlebot's rendering

    The bottom line

    If your important content only exists after JavaScript runs, most AI crawlers can't read it. Test with view-source and curl, then fix the pages that matter with server-side rendering, static generation or prerendering, including the metadata and structured data in the head. It's often the single biggest technical AEO win on modern websites.

    For the rest of the crawler picture, see our AI crawlers and robots.txt guide.

    FAQ

    Frequently asked questions

    Can ChatGPT read JavaScript websites?

    OpenAI's crawlers, like most AI crawlers, primarily read the HTML the server returns. Content that only appears after JavaScript runs in a browser may not be seen. Serve important content in the initial HTML to be safe.

    Does Googlebot render JavaScript?

    Yes. Googlebot renders pages with a modern browser engine, though rendering can be delayed and can fail on slow or error-prone pages. AI crawlers from other companies generally don't render, which is why server-rendered HTML matters more for AI search.

    What is prerendering?

    Prerendering generates static HTML for pages ahead of time, at build time or on demand, so crawlers receive full content without running JavaScript. It's a common fix for single-page applications.

    How do I check what an AI crawler sees?

    Fetch the page with curl, optionally using an AI crawler's user agent string, and look for your key content in the response. You can also view the page source in a browser (not the inspector, which shows the rendered page).

    • JavaScript
    • rendering
    • SSR
    • prerendering
    • technical SEO
    • AI crawlers
    Saarthak Srivastav

    Written by

    Saarthak Srivastav

    AEO & GEO, AEO-Rank

    Saarthak Srivastav writes and reviews articles on the AEO-Rank blog, drawing on what the AEO-Rank team sees tracking AI answers across ChatGPT, Gemini, Claude, Perplexity and Grok.