Andrej Karpathy’s post about using LLMs to build personal knowledge bases went viral in 2026 — 18 million views, multiple open-source implementations built in the weeks following. The concept: index your research materials, articles, and notes into a raw folder; let an LLM incrementally compile a wiki of interlinked markdown files; query it conversationally rather than searching documents manually.
The reason it resonated is that it named something practitioners already knew worked. The interesting question for mid-market companies is what happens when you apply the same logic at the organizational level rather than the individual level.
quadrantChart title AI Knowledge Base — Where to Start x-axis Low Query Volume --> High Query Volume y-axis Knowledge Dispersed --> Knowledge Centralized quadrant-1 High-value starting points quadrant-2 Good but lower urgency quadrant-3 Skip for now quadrant-4 Already solved Customer support history: [0.85, 0.2] Compliance policies: [0.75, 0.35] Product documentation: [0.6, 0.55] Technical architecture: [0.5, 0.3] HR policies: [0.35, 0.7] Executive email: [0.15, 0.15]
Where Most Mid-Market Companies Store Their Knowledge
Most mid-market companies have institutional knowledge in four places: email threads that the author may or may not have archived, SharePoint or Google Drive folders where naming conventions vary by who created the folder, the heads of two or three people who have been at the company long enough to know how things actually work, and a wiki that was last updated eighteen months ago and no longer reflects current reality.
This is not a technology failure. It is the natural accumulation of knowledge in organizations that grew without a deliberate knowledge management infrastructure. Documents get created when there is a reason to create them and decay when nobody maintains them. The people who know how things work are usually the most valuable and the hardest to replace. The wiki gets created with good intentions and abandoned when maintaining it stops feeling like the highest priority.
When I led the development of a complex reporting system for WellPoint, a major health insurance provider serving millions of members, the knowledge management challenge was embedded in the project structure. The rules for how claims should roll up into reports, how different product lines were categorized, how regulatory requirements varied by state — this knowledge existed in a combination of regulatory documents, institutional memory, and previous system behavior. Making it accessible to the team working on the new system required continuous effort to synthesize it into a form that developers could actually use.
The problem is not unique to healthcare. Any organization that has been operating for more than five years has accumulated institutional knowledge that is partially accessible and partially locked inside the people and documents that haven’t been organized for retrieval.
What an LLM Knowledge Base Changes
The Karpathy model — index source materials, let an LLM compile a structured, interlinked knowledge base, query it conversationally — solves a specific problem that most mid-market organizations face: knowledge that exists in dispersed form but is queried repeatedly.
The measurable version of this problem: how many hours per week does your team spend searching for answers that are technically documented somewhere, but where finding the documentation requires knowing where to look? How many customer support escalations happen because the person handling the inquiry does not have easy access to the relevant policy or historical context? How many new employees spend their first three months asking questions that someone in the organization has answered before?
For each of those questions, the answer is “more than you’d expect.” The cost is not concentrated in any single incident — it is distributed across dozens of small inefficiencies that individually are easy to dismiss but collectively represent a significant organizational drag.
An LLM knowledge base makes organizational knowledge queryable. Not searchable in the document sense — you do not find the right document, read it, and extract the relevant section. You ask a question and receive a synthesized answer drawn from across the relevant source materials. For a customer support representative asking about a specific policy, that is the difference between a thirty-second query and a five-minute search. Across a team handling hundreds of inquiries per day, that difference has a measurable cost.
Why Mid-Market Companies Are Not Doing This
The barrier is not technical complexity — the infrastructure for building LLM knowledge bases has become accessible enough that it does not require a machine learning engineering team. The barrier is organizational: someone has to decide which domain to start with, gather the source materials, maintain the quality as those materials change, and measure whether the resource is actually delivering value.
That decision and maintenance process sounds trivial and is consistently underestimated. Organizations that attempt broad organizational knowledge bases — “everything the company knows” — typically fail to maintain them. The scope is too large, the governance is too distributed, and the maintenance burden distributes to a point where nobody feels responsible for keeping any specific part of it current.
The organizations that succeed narrow the scope aggressively. One domain. One owner. One measurable outcome — reduction in query time, reduction in escalation rate, reduction in time-to-competency for new employees. Start there, prove the model, and extend only once the first domain is demonstrably working.
Where to Start
The right starting domain is the one that combines high query volume with dispersed, hard-to-find source knowledge. Customer support history is the most common starting point because the pattern is obvious: the same questions get asked repeatedly, the answers exist somewhere in the system, and finding them requires more effort than it should. Compliance documentation is the second most common starting point, particularly in regulated industries where policy questions arise frequently and getting the answer wrong has consequences.
Technical architecture documentation is underutilized as a starting point but has significant value for engineering teams. Most organizations have architecture decisions scattered across pull request comments, Confluence pages, Slack threads, and the memories of senior engineers. Making those decisions queryable — especially for new engineers joining the team — reduces the onboarding time that currently comes at the expense of your most senior people’s attention.
The tool Karpathy described, and the open-source implementations that followed it, are accessible enough that a technically capable person at a mid-market company can build a working prototype in a day. The organizational work of deciding what to index, who owns it, and how to measure whether it is working takes longer and matters more.
Start with what you have. Index it. Measure the before and after. The institutional knowledge already exists. The question is whether it is organized for retrieval or locked in email and the heads of your most irreplaceable employees.
