LLMs and data compression
Large language models effectively compress enormous amounts of training data into their model parameters. This document looks at how that data is compressed, not from a technical point of view, but from a content creation and Answer Engine Optimization (AEO) point of view, and how understanding that compression can inform an AEO content strategy.
During training, information that appears repeatedly across many sources can contribute to stronger learned associations between concepts and entities. The model does not need to retain every source or every occurrence of an assertion individually; instead, recurring patterns can become part of its compressed representation of the subject.
For AEO simulation using a small language model, Solution Smith calls these assertions "commons," borrowing the concept from the idea of information held in common. In this model, commons are assertions that are sufficiently established that the assertion itself does not require a citation to a particular source.
A common assertion can still contain mentions of entities that are important to an answer. For example:
- "Google is the largest search engine."
The assertion itself may be treated as common, while Google is a mention of an entity. The distinction becomes important when examining how information moves from source documents into an AI-generated answer. A source may contribute an assertion to the model's general knowledge without that source being retained as the citation for the assertion.
RAG-based AI answers with LLMs
Currently, Google, ChatGPT, Perplexity, Copilot, and other AI systems can supplement an LLM's native knowledge by retrieving information from external indexes or other data sources. Depending on the system, those sources can include search indexes such as Google Search, Bing, or Brave Search, as well as other proprietary or specialized indexes. The Google's Hybrid AI / Search Engine Model looks deeper into this model.
These retrieval-based AI answers are often described as Retrieval-Augmented Generation (RAG), although the underlying architectures and retrieval methods can differ between systems. Retrieved information can supplement the model's native knowledge for the particular answer without necessarily becoming part of the model's persistent native knowledge.
RAG-based answers can be generated with more or fewer citations, depending on how the system retrieves, evaluates, and presents the underlying information. It cannot be declared exactly when a RAG-based AI answer will or will not create a citation, nor can it be assumed that citation behavior will remain fixed as these systems evolve.
Government regulation, commercial interests, licensing arrangements, changes to search indexes, and other market factors could also influence which sources are available to an AI system and how those sources are used or cited.
Native AI answers versus AI with injected data
It should be noted that Solution Smith's AI salesman or brand ambassador is based on injected data. It is not exactly a Retrieval-Augmented Generation framework; rather, additional details are introduced by providing a rich XML document.
This does not appear to be the methodology used by Wiki-LLM or the Open Knowledge Panel. The purpose here is different: the "commons" attribute provides a simplified way to both compress data and model the effect of that compression within a small language model.
The simulation therefore distinguishes between information already represented by the model and information introduced through data injection. The commons attribute is a simulation mechanism used to identify assertions that the model treats as sufficiently established to become part of its generalized knowledge rather than requiring attribution to a specific source.
Consider a book or website converted into an SLM AI-chat expert. Additional information, including advertisements, can be injected into the AI-chat expert along with citations to the source material.
An AI-chat salesman can therefore be modeled as having both common knowledge and brand knowledge. The common knowledge represents information already generalized by the model, while the injected brand knowledge represents additional information supplied specifically to the AI-chat expert. The brand knowledge can retain citations, allowing the AI to associate particular claims with their sources when generating an answer.
How AI compression affects SEO and AEO
From an SEO and AEO perspective, creating information beyond the model's existing common knowledge is a content strategy for AEO.
A website automatically created using an LLM will, in many cases, contain primarily common knowledge. When such a website is first indexed, the indexing system does not power up a data center and perform a deep dive into the entire site to determine every piece of information that might be unique. For a healthy website, the page can be quickly indexed and served.
Google does not explain its proprietary processes for evaluating a page after it has been indexed. We do know that systems such as NavBoost and user signals play a role in refining search results.
The pattern of traffic observed for a site created primarily from common knowledge, sometimes called "Mount AI," is consistent with a possible methodology in which the content is evaluated as it enters the search results. This is an inference from the observed pattern, not a description of a confirmed Google process. When the page provides little or no knowledge beyond what is already commonly represented, the page may subsequently lose visibility in search.
From an AEO perspective, this suggests an important distinction: being indexed is not the same as contributing knowledge. A page can be crawled, indexed, and initially served while still providing little information that distinguishes it from the knowledge already available to the search or AI system.
Not a quality judgment
The AI is not necessarily making a quality judgment about the writing. The issue is whether the content contributes information beyond what is already represented as common knowledge.
For example:
- "Water boils at 100 degrees Celsius."
The statement can be factually correct, clearly written, and perfectly understandable. Yet, if the AI system already treats this information as common knowledge, the statement provides no new knowledge simply because it has been written well.
The distinction is quality of expression vs information gain. A well-written statement does not automatically make the information unique.
The Statement:
- "The Best-Brand coffee steamer brews coffee at an ideal temperature regardless of elevation! A true mountain brew coffee machine."
Provides unique knowledge (an entity, product, and feature) not common knowledge.
Created
by Wayne Smith – Raising the Standards
This article is part of Solution Smith's State of AI, an ongoing research series that examines how modern AI systems process information and how those behaviors affect search, Answer Engine Optimization (AEO), and traditional SEO.
AI has fundamentally changed information retrieval. In addition to disrupting many traditional SEO practices, it has introduced Answer Engine Optimization (AEO), where content is optimized not only for search rankings but also for selection, citation, and use within AI-generated responses.