ByteAtlas

An LLM wiki is not just a folder of documents connected to a chatbot. It is a persistent, maintained knowledge base that an LLM helps rewrite, cross-link, question, and improve over time.

The difference sounds small until the source material starts changing every day. A conventional retrieval system can fetch raw passages when a question arrives. An LLM wiki tries to do some of the expensive thinking earlier: extract the useful ideas, connect them to existing pages, record contradictions, and preserve conclusions from previous queries.

I found this pattern particularly useful around autonomous-driving scenario generation. The need was not simply “generate more scenarios.” Requirements kept accumulating: road geometry, actor intent, weather, visibility, timing, edge conditions, expected behavior, and increasingly detailed generation methods. The knowledge needed to create a good scenario was becoming more valuable than any single generated scenario.

Once those requirements and generation lessons were updated into a wiki after each round of work, later requests became noticeably easier. The LLM could recover the relevant context, reuse the accumulated constraints, and produce a useful result without rebuilding the entire mental model in every chat. The benefit was not that the model suddenly knew everything. The benefit was that the working knowledge stopped disappearing.

What is an LLM wiki?

The concept described in Andrej Karpathy’s LLM Wiki notes is a structured collection of interlinked Markdown pages maintained with an LLM. Raw sources remain available, but the wiki sits between those sources and future questions.

raw sources
     |
     v
   ingest
     |
     v
linked wiki pages <--- lint and maintenance
     |
     v
    query
     |
     +---> answer, analysis, or a new wiki page

A source may update several pages instead of becoming one isolated summary. A new document about pedestrian behavior might update the pedestrian entity page, a crossing-scenario guide, an environmental constraints page, and a list of unresolved questions. The wiki becomes a compiled representation of what has been learned.

This differs from basic RAG. In a common RAG flow, raw documents are split into chunks and retrieved at query time. The model may have to reconstruct the same multi-document conclusion repeatedly. An LLM wiki stores useful synthesis and links as durable artifacts, while still retaining source traceability.

The three core operations

Ingest: turn a source into maintained knowledge

Ingest is not “copy this file into the repository.” It is the workflow that reads a new source, extracts what matters, and integrates it into the existing structure.

• Save the original source without silently rewriting it.
• Record source metadata such as date, author, origin, and scope.
• Identify claims, concepts, procedures, constraints, examples, and open questions.
• Create a source-summary page when the source deserves one.
• Update existing topic and entity pages.
• Add links between affected pages.
• Flag conflicts instead of choosing a winner without evidence.
• Append an ingest entry to the operation log.

The important choice is granularity. One source should not automatically become one wiki page. A detailed engineering discussion can affect many pages. Conversely, a minor source may only add one citation or clarify one sentence.

I prefer supervised ingestion for high-value technical knowledge. Let the LLM propose the changes, then review what it emphasized and what it omitted. Batch ingestion is faster, but a wiki can become confidently tidy while losing the details that actually matter to practitioners.

Query: answer from the maintained wiki

Query searches the wiki, reads the relevant pages, follows useful links, and synthesizes an answer with citations. The result might be a direct answer, a comparison, a plan, a scenario specification, or another Markdown page.

A good query operation should show which pages and sources informed the answer. It should also distinguish established constraints from suggestions. That distinction is essential when the wiki mixes requirements, observations, templates, and exploratory ideas.

The compounding step is easy to miss: a valuable query result can be filed back into the wiki. If a question forces the model to connect five pages and produces a reusable decision guide, keeping that guide only in chat throws away work. Store it as a page, cite the inputs, and let future queries reuse it.

Lint: find decay before it becomes trusted context

Lint is a health check over the wiki. Markdown that parses correctly can still represent a broken knowledge base, so wiki linting must inspect both structure and meaning.

• Broken links and missing pages
• Orphan pages with no inbound links
• Duplicate pages describing the same concept
• Claims without a source
• Contradictions between pages
• Stale guidance superseded by newer material
• Important concepts mentioned repeatedly but lacking their own page
• Terminology that changes meaning across sections
• Unresolved questions that have silently become assumptions

Not every contradiction should be “fixed.” Two sources may describe different operating conditions or disagree legitimately. A useful lint pass records the conflict, identifies scope, and asks for a decision or more evidence. Automatic cleanup that erases disagreement can make the wiki less truthful.

A practical starting structure

scenario-wiki/
├── AGENTS.md
├── wiki-purpose.md
├── wiki-schema.md
├── wiki-log.md
├── sources/
│   └── 2026-10-18/
│       └── scenario-review-notes.md
└── wiki/
    ├── index.md
    ├── concepts/
    ├── requirements/
    ├── scenario-types/
    ├── generation-guides/
    ├── entities/
    ├── decisions/
    └── open-questions/

AGENTS.md or an equivalent agent instruction file explains how the LLM should operate. wiki-purpose.md defines the audience and boundary. wiki-schema.md defines page types, naming, metadata, links, citations, and the ingest, query, and lint procedures. wiki-log.md is an append-only history of operations.

The folders should reflect the questions you expect to ask. For scenario generation, useful distinctions might include scenario types, environmental requirements, actor behaviors, generation techniques, validation criteria, and decisions. Do not start with a taxonomy elaborate enough for a hundred-person organization if three pages would answer today’s questions.

Write the purpose before adding documents

A wiki with no declared purpose becomes a general dumping ground. A short purpose document can prevent that:

# Wiki purpose

This wiki captures reusable knowledge for designing and generating
simulation scenarios for autonomous-driving development.

Primary users:
- engineers requesting scenarios
- engineers implementing generation workflows
- reviewers checking requirement coverage

The wiki should preserve:
- scenario requirements and constraints
- reusable generation methods
- terminology and entity definitions
- decisions and their rationale
- known gaps and conflicting guidance

The wiki must not contain:
- credentials or secrets
- personal data
- confidential identifiers not required for the knowledge

This boundary gives the agent a reason to reject irrelevant material and a rule for redacting sensitive details. Persistent memory is useful precisely because it persists, which also makes careless ingestion more expensive.

Define the schema as operating instructions

The schema is not only a frontmatter specification. It should explain how the agent makes decisions.

# Wiki schema

Every knowledge page includes:
- title
- page type
- status: draft, reviewed, or superseded
- last updated date
- source links
- related pages

During ingest:
1. Read the source and identify affected pages.
2. Preserve source-specific claims and scope.
3. Update existing pages before creating near-duplicates.
4. Record contradictions under an explicit heading.
5. Add an entry to wiki-log.md.

During query:
1. Search index and related pages.
2. Cite wiki pages and original sources.
3. Separate requirements, observations, and proposals.
4. Offer to store reusable synthesis as a new page.

During lint:
1. Check links and required metadata.
2. Find stale, duplicate, orphaned, and conflicting pages.
3. Produce a report before making semantic changes.

The last line matters. Structural repairs can often be automatic. Semantic repairs deserve review, especially when one page may represent a formal requirement and another reflects an experimental technique.

How this helped with scenario generation

Scenario-generation requests tend to become more specific over time. An early request may ask for a cut-in scenario. Later requests add speed ranges, actor profiles, road curvature, visibility, trigger timing, acceptable variations, and expected system behavior. The instructions that produce a useful result are gradually discovered through reviews and corrections.

Before using a wiki workflow, that knowledge can remain scattered across prompts, chat history, review notes, and the memory of whoever handled the previous request. A new request looks similar, but the model receives only part of the accumulated context.

Updating the wiki after each request changed the shape of the work. A review comment about actor timing updated the relevant generation guide. A new environment constraint updated the requirement page. A useful query comparing two generation methods became a decision page. Later requests could retrieve those linked pages rather than depending on one enormous prompt.

The practical result was faster convergence on the desired scenario. The LLM had better context about how requests were normally interpreted and which details had caused problems before. Human review was still necessary, particularly for safety-sensitive requirements, but less time was spent repeating previously learned instructions.

Start small: the first-week workflow

1. Choose one narrow domain with recurring questions.
2. Write the purpose and exclusions.
3. Create a simple schema with three or four page types.
4. Ingest one representative source under supervision.
5. Ask three real questions and inspect the citations.
6. Store one useful query result back into the wiki.
7. Run lint and review the report manually.
8. Adjust the schema based on actual failure modes.

Do not begin by importing every document available. Early ingestion teaches you what the page types and metadata should be. Bulk ingestion performed before the schema stabilizes can create hundreds of consistently wrong pages.

Useful prompts for each operation

Ingest prompt

Ingest sources/2026-10-18/scenario-review-notes.md.
Follow wiki-schema.md. Before editing, list the pages you plan to
create or update and explain why. Preserve disagreements and cite the
source for every new requirement.

Query prompt

Using the wiki, draft a scenario specification for a low-visibility
pedestrian crossing. Separate established requirements from proposed
parameters. Cite the wiki pages and original sources used.

Lint prompt

Lint the wiki. Check broken links, orphan pages, missing sources,
stale requirements, terminology drift, and contradictions. Produce a
report first. Do not resolve semantic conflicts without approval.

Where an LLM wiki can go wrong

• Unsourced synthesis: polished pages gradually lose the evidence behind them.
• Silent conflict resolution: the agent chooses one claim and deletes the disagreement.
• Taxonomy inflation: every term becomes a page, making navigation worse.
• Stale authority: an old page looks trustworthy because it is well written.
• Prompt injection in sources: untrusted documents contain text that attempts to redirect the agent.
• Confidentiality leakage: sensitive raw notes become broadly searchable.
• Automation without review: high-volume ingestion produces a coherent but inaccurate knowledge base.

Treat source contents as data, not as instructions to the wiki-maintenance agent. Keep version control enabled, review diffs, and make provenance visible. A wiki that cannot explain where a requirement came from should not be treated as a requirement system.

Conclusion

An LLM wiki is valuable when knowledge changes through repeated work. Ingest integrates new material, query turns maintained knowledge into useful outputs, and lint keeps the structure from quietly decaying.

The strongest use case is not storing facts that are already easy to search. It is preserving the growing set of constraints, decisions, connections, and techniques that make the next request easier than the last. That is what made the approach useful for scenario generation: each review improved not only one output, but the context available for future outputs.

Start with a narrow purpose, a readable Markdown structure, one supervised ingest, and real queries. Let the schema evolve from the mistakes you actually observe. The original LLM Wiki notes provide the conceptual foundation, while the previously published PyTorch-to-ONNX workflow shows another example of turning an AI integration boundary into an explicit, maintainable process.