AEO Fundamentals

    How AI Assistants Generate Answers: Training Data vs Live Retrieval

    Where ChatGPT, Claude, Gemini and Perplexity get their information: model training, live web search, and how the two combine into one answer.

    Saarthak Srivastav

    Saarthak Srivastav

    · 4 min read

    Key takeaways

    • AI assistants answer from two sources: patterns learned during training, and documents retrieved at the moment of the question.
    • Training knowledge is broad but frozen at a cutoff date and only changes when a new model ships.
    • Live retrieval is current and cites sources, but only works if your pages can be found, fetched and understood.
    • Whether an assistant searches depends on the question. Products, prices, comparisons and recent events usually trigger search.
    • AEO works on both: consistent brand facts across the web for training, and crawlable, answer-first pages for retrieval.

    AI assistants generate answers from two sources: what the model learned during training, and what it retrieves from the web at the moment you ask. Training gives broad, general knowledge that's frozen at a cutoff date. Retrieval adds current, specific information with citations. Most answers about products and brands blend both. Understanding how they combine tells you what you can influence, and how fast.

    Source 1: what the model learned in training

    A large language model (LLM) is trained by reading an enormous amount of text: public web pages, books, code, forums, reference material and licensed data. During training it learns statistical patterns: which words, facts and ideas tend to go together.

    The result is a model that can:

    • Explain concepts ("what is a CRM?")
    • Summarize common knowledge ("what are the main project management methods?")
    • Recall associations ("Brand X is known for its free tier")

    Limits of training knowledge

    • It has a cutoff. The model knows nothing from after its training data was collected. A brand that launched, rebranded or changed pricing since then may be unknown or misdescribed.
    • It's compressed. The model doesn't store web pages; it stores patterns. Niche facts mentioned rarely may be fuzzy or wrong.
    • It reflects consensus. If most sources describe you one way, that's what the model learns, even if it's outdated.
    • It changes slowly. It updates only when a new model version is trained and released, typically months apart.

    What shapes training knowledge about your brand

    How often and how consistently your brand appears across the web before the cutoff: your own site, news coverage, reviews, forum discussions, directories, documentation and partner pages. Many companies let some or all AI training crawlers access their content; see our AI crawlers and robots.txt guide for how to decide.

    Source 2: what the assistant retrieves live

    To answer questions about current or specific topics, assistants use retrieval: they search an index, read relevant pages and write an answer grounded in them. This is often called retrieval-augmented generation (RAG).

    The typical flow:

    1. Decide whether to search. The assistant judges whether the question needs fresh or specific information.
    2. Write search queries. It turns your question into one or more queries, often adding more specific variants.
    3. Retrieve candidates. It pulls results from a search index: its own, a partner's like Bing or Google, or both.
    4. Read passages. It extracts the most relevant chunks from the top results.
    5. Generate and cite. It writes the answer from those passages and attaches source links.

    Strengths and limits of retrieval

    • Current. It can use a page published yesterday.
    • Citable. Users can see and click the sources.
    • Dependent on access. If crawlers can't fetch your page, or your content only appears after JavaScript runs, you're invisible here.
    • Passage-driven. The best-matching chunk wins, not the best overall page.

    It depends on the platform and the question:

    Question type Likely to trigger search?
    "What is [concept]?" Often not, unless it's new or niche
    "How do I [task]?" Sometimes
    "Best [product] for [use case]" Usually
    "[Brand A] vs [Brand B]" Usually
    "[Brand] pricing / reviews" Almost always
    Recent news, events, local info Almost always

    Perplexity searches for almost every question. ChatGPT, Claude and Gemini decide case by case, and users can also ask them to search. Google's AI Overviews and AI Mode are always grounded in Google Search. See how each assistant finds sources for the details.

    How the two sources combine

    Even when an assistant searches, the model's training still shapes the answer:

    • Query writing. The model's existing understanding determines what it searches for. If it doesn't associate your brand with a category, it may not search for you by name.
    • Interpretation. It reads retrieved pages through what it already "knows," which affects how it summarizes them.
    • Gap-filling. Where retrieved sources are thin, the model fills in from training, which is where outdated or wrong descriptions often come from.

    This is why a brand can rank well in search and still be under-mentioned in AI answers, or be described with an old positioning even when the current website is accurate.

    What this means for AEO

    Work on both sources, with different expectations for speed:

    For retrieval (fast feedback)

    • Allow AI search crawlers and user fetchers.
    • Serve key content in server-rendered HTML.
    • Publish pages that answer specific buyer questions directly. See how to write content AI assistants cite.
    • Make sure the third-party pages assistants retrieve, such as reviews, roundups and directories, describe you accurately.

    For training (slow, compounding)

    • Describe your brand, category and positioning consistently everywhere.
    • Earn coverage in widely read publications and communities.
    • Retire or update outdated pages that describe old products or pricing.
    • Use clear structured data to anchor your brand as an entity.

    How to tell which source an answer used

    • Citations present? The answer used retrieval. Check which pages were cited.
    • No citations, general description? Likely model knowledge. Compare the description with your current positioning to spot outdated training data.
    • Mix of both? Look for claims that don't appear in any cited source; those came from the model.

    Tracking this across many prompts, as covered in AI visibility metrics, shows whether your problem is mostly retrieval (fixable quickly) or mostly model knowledge (a longer project).

    The bottom line

    AI answers come from a frozen-but-broad memory and a live-but-selective search. You can't edit the memory directly, but you can shape what future models learn by being consistent and widely covered. You can influence retrieval this week by being crawlable and by publishing the clearest answers. Strong AEO programs do both.

    FAQ

    Frequently asked questions

    Where does ChatGPT get its information?

    From two places. The underlying model learned from a large body of text during training, which gives it general knowledge up to a cutoff date. For many questions, ChatGPT also searches the web and writes its answer from the pages it finds, citing them as sources.

    What is a training cutoff?

    It's the point in time after which a model has no knowledge from its training data. Anything that happened or changed after the cutoff is unknown to the model unless the assistant retrieves it through search or another tool.

    Do AI assistants search the web for every question?

    No. Perplexity searches for almost every question, but ChatGPT, Claude and Gemini decide case by case. Questions about current events, products, prices, comparisons and local information are the most likely to trigger a search.

    Can I update what an AI model knows about my company?

    Not directly. You can make sure accurate, consistent information about your company is widely available on the web, which future training runs may learn from. For immediate impact, focus on retrieval: crawlable, clear pages and accurate third-party listings that assistants find when they search.

    • how AI works
    • training data
    • retrieval
    • RAG
    • AI search
    Saarthak Srivastav

    Written by

    Saarthak Srivastav

    AEO & GEO, AEO-Rank

    Saarthak Srivastav writes and reviews articles on the AEO-Rank blog, drawing on what the AEO-Rank team sees tracking AI answers across ChatGPT, Gemini, Claude, Perplexity and Grok.