LLM Wiki: Theory part01 | What Is Karpathy's LLM Wiki, and How Is It Different from RAG?

About the “LLM Wiki: Theory” series

I am currently building a Knowledge Engine, which I plan to release as an open-source project once the architecture and documentation are ready.

The system organises knowledge according to Google’s Open Knowledge Format (OKF) v0.1 Draft. It combines the persistent knowledge layer of an LLM Wiki with RAG-based retrieval, then adds a control layer, or Harness, that governs how an AI searches, reads, checks sources and assembles an answer.

People can read and maintain the knowledge pages in Obsidian, while Sigma.js provides a visual map of the relationships between them.

This short series has two parts. The first explains why the LLM Wiki idea emerged and how it differs from RAG. The second looks at the roles of OKF, Markdown, Obsidian, Sigma.js, search indexes and the Harness.

Imagine that you have a hundred documents.

They include research reports, product specifications, interviews, meeting notes, PDFs and assorted scraps of information. You load them into a document question-answering system and begin asking questions:

On what date does this contract end?
What problems do customers mention most often?
Which product decisions made over the past six months contradict one another?
Were the conclusions in Report A overturned by the later findings in Reports B and C?

All four questions involve searching documents, but they are not equally difficult.

Finding a contract date usually means locating one passage. Comparing the assumptions, definitions and conclusions across several reports means dealing with multiple sources, different time periods and conflicting evidence.

The first is mostly a retrieval problem.

The second is closer to a small research task.

Karpathy’s LLM Wiki is aimed mainly at the second sort of problem.

LLM Wiki is a knowledge architecture

In April 2026, Andrej Karpathy published a Gist called LLM Wiki. He described it as an architectural pattern for building a personal knowledge base with an LLM, rather than a finished piece of software.

It does not prescribe a particular model, vector database, graph database or agent framework. Obsidian, BM25, vector search and Git can all be useful parts of the system, but none of them is the LLM Wiki itself.

Karpathy’s question is a practical one:

Each time a model answers a question, it goes back to the source documents, finds the relevant material, makes sense of it and assembles a temporary response. Why should the useful work disappear afterwards?

In many document question-answering systems, the comparison or judgement lives only in the chat history. When a similar question appears later, the system searches, reads and organises the material all over again.

An LLM Wiki adds a maintained knowledge layer between the source material and the user.

When new material arrives, an agent may do more than build a search index. It might:

create a source summary;
update an existing concept page;
add a page for a person, company or event;
record which earlier conclusions the new source supports or disputes;
preserve comparisons and analyses worth reusing;
link knowledge pages back to the underlying sources.

The durable output is no longer just an answer. It is a set of knowledge pages that can be read, revised and queried again.

The three layers: sources, Wiki and schema

Karpathy divides the LLM Wiki into three layers.

Raw sources: preserving the evidence

This layer contains the original articles, papers, images, data files and other source material.

An LLM may read them, but it should not alter them freely. If something in the Wiki looks wrong, a person or agent needs to return to the original and determine whether the summary went too far, omitted a qualification or merged two different concepts.

Without the sources, the Wiki soon becomes a collection of second-hand claims that cannot be checked.

Wiki: preserving organised knowledge

The Wiki is a set of knowledge pages created and maintained by an LLM. It may contain:

source summaries;
people, companies and events;
concepts and definitions;
timelines;
comparisons across sources;
competing or contradictory views;
unresolved questions.

Adding a new source does not necessarily mean adding only one new summary. The agent may also update an existing page, add another piece of evidence, or mark an older conclusion as out of date.

That allows earlier research and organisation to be reused in later queries.

Schema: telling the agent how to work

The schema may take the form of an AGENTS.md, CLAUDE.md or another set of operating instructions. It tells the agent:

how files should be named;
which fields each type of page should contain;
when to create a new page;
when to update an existing one;
how to preserve sources and citations;
how to deal with contradictions;
which analyses are worth writing back to the Wiki;
how to find isolated, stale or unsupported pages.

Without this layer, the LLM is simply improvising inside a folder. That may look acceptable with a dozen pages. Once the collection grows, naming, structure and citation practices begin to drift.

The schema turns a general chat model into a constrained knowledge maintainer.

What does a typical RAG system do?

RAG stands for Retrieval-Augmented Generation.

The basic idea is simple: before a model answers a question, the system retrieves relevant material from an external collection and places it in the model’s context.

Source documents
      ↓
Split into passages
      ↓
Build a search index
      ↓
Receive a question
      ↓
Retrieve relevant passages
      ↓
Ask the LLM to answer

Modern RAG systems may combine keyword and vector search, reranking and multi-step retrieval. It would be misleading to reduce the entire field to “fetch a few chunks by vector similarity”.

The comparison here is with a common document question-answering form of RAG. Such systems usually preserve:

the original documents;
the passages produced from them;
metadata;
keyword or vector indexes.

They do not usually turn the comparisons, conclusions and conceptual work produced during each query into a maintained knowledge layer.

An LLM Wiki preserves that additional layer.

The difference is when knowledge is organised

A typical RAG system assembles most of its useful context after a question arrives.

Documents → Index

Question → Retrieve passages → Interpret and assemble → Answer

An LLM Wiki moves some of that work into ingestion and ongoing maintenance.

Documents
   ↓
Read, compare and organise
   ↓
Knowledge pages, sources and relationships
   ↓
Search and indexes

Question → Find the relevant page → Read related pages and sources → Answer

This is not a hard boundary.

Advanced RAG systems can build summaries, hierarchies and entity relationships during indexing. The more distinctive feature of an LLM Wiki is that the organised result becomes a first-class knowledge layer that people and agents can read, edit and maintain over time.

An example: WAU rose by 18%. Did the product improve?

Suppose the knowledge base contains three documents:

a product metrics specification;
a weekly operations report;
a tracking migration note.

The operations report says:

Weekly active users increased by 18% this week.

If that is the only passage retrieved, the system may conclude:

User engagement improved significantly.

The tracking migration note, however, may also say:

From 1 July, a user counts as active only after completing a core action, rather than merely signing in.

The definition changed between the two periods. The 18% increase cannot be treated as direct evidence that users became more active.

A RAG system must retrieve both the operations report and the definition change during the query.

An LLM Wiki may already contain:

a WAU metric page;
the old and new definitions of an active user;
a timeline of metric changes;
the tracking migration event;
citations pointing back to the source documents.

Once the agent reaches the WAU page, it can follow the relationships to check the definition, version and source.

The structure does not produce the correct answer by itself. It gives the agent a route through the material and makes important dependencies harder to overlook.

An LLM Wiki still needs RAG

Once knowledge has been organised into a Wiki, the agent still has to find the right pages.

For a small collection, it may read an index.md and follow links. As the collection grows, the system can still use:

full-text search;
BM25;
vector search;
reranking;
graph traversal.

Raw sources
     ↓
Knowledge compilation
     ↓
Wiki pages, metadata, citations and relationships
     ↓
Keyword, vector and graph indexes
     ↓
Agent search and reading
     ↓
Verification against the source
     ↓
Answer

Vector search is useful for finding semantically similar pages.

BM25 is often better for exact names, error codes, version strings and other precise terms.

Graph relationships add information beyond general similarity, such as:

which definition governs a metric;
which version replaced an older specification;
which sources support a conclusion;
which conditions a decision depends upon.

The LLM Wiki does not replace RAG. It gives retrieval a richer knowledge structure to search and navigate.

The main differences at a glance

Aspect	Common document-QA RAG	LLM Wiki
Main persistent assets	Documents, passages and indexes	Knowledge pages, relationships, sources and indexes
Typical retrieval unit	Passage or chunk	Page, passage, concept and related nodes
When knowledge is organised	Mostly at query time	During ingestion, maintenance and querying
Cross-source analysis	Repeated for each query	Can be preserved and updated
Human readability	Indexes are mostly internal system assets	Knowledge pages are readable artefacts
Main risks	Poor retrieval, ranking errors, missing context	Distorted summaries, stale pages, accumulated relationship errors
Maintenance cost	Retrieval pipeline and indexes	Retrieval plus pages, citations and knowledge lifecycle management

A mature agentic RAG system can also perform multi-step retrieval, retain memory and build structure.

Conversely, a Markdown Wiki without dependable sources or maintenance rules does not become trustworthy merely because it contains a few more links.

How can an LLM Wiki fail?

An LLM Wiki turns source material into shorter, easier-to-use knowledge pages. That process inevitably creates a risk of information loss.

A summary drops a qualification

The source may say:

A and B were correlated in a sample of fewer than 500 observations, without seasonal adjustment and excluding several industries.

The Wiki may preserve only:

A and B are correlated.

The sentence is shorter, but the conditions that made it meaningful have vanished.

Different concepts are merged

Two teams may both use the term “active user”. One counts sign-ins; the other counts completed transactions.

If an agent merges them into a single definition, later analysis may read smoothly while being wrong.

An old conclusion remains active

If the system only adds information and does not track replacement, expiry or validity periods, several apparently correct versions may remain in circulation at once.

A dependable LLM Wiki therefore needs more than page-to-page links. It needs a traceable path back to the original evidence.

When is an LLM Wiki worth building?

Situation	More suitable approach
Find a specific date or number in a document	RAG
Repeatedly query documents whose relationships are fairly simple	RAG or query caching
Compare several sources and preserve the result	LLM Wiki
Continue adding material to the same body of knowledge	LLM Wiki
Track definitions, versions, evidence and supersession	LLM Wiki with graph relationships and a Harness
The collection is small enough to fit directly into the model context	A long-context approach may be simpler

In practice, many systems will use a hybrid design:

Raw sources
   ├── RAG: retrieve the exact passages
   └── LLM Wiki: preserve concepts, comparisons and cross-source synthesis
                         ↓
             The agent chooses a route
                         ↓
              Verify against the source

The Wiki supplies the map. RAG digs out the original text when it is needed.

Conclusion

RAG gives an LLM access to external material while it answers a question.

An LLM Wiki adds a persistent knowledge layer, so that summaries, concepts, comparisons, contradictions and relationships do not have to be rebuilt for every query.

That creates opportunities for reuse, but it also creates another system to maintain. Retrieval quality is no longer the only concern. The system must also deal with:

distorted summaries;
duplicated or conflated concepts;
stale pages;
incorrect relationships;
claims that cannot be traced to a source;
agents with overly broad write access.

A practical question helps decide whether the additional layer is justified:

Will this body of knowledge be researched, compared and reused repeatedly?

If not, a well-built RAG system will often be enough.

If it will, repeatedly reconstructing the same understanding from the source documents may be the more expensive option.