This weekend, Andrej Karpathy, the previous director of AI at Tesla and a founding member of OpenAI, decided he desired to read a book. But he didn’t need to read it alone. He desired to read it accompanied by a committee of artificial intelligences, each offering its own perspective, critiquing the others, and eventually synthesizing a final answer under the guidance of a "Chairman."
To make this occur, Karpathy wrote what he called a "vibe code project" — a chunk of software written quickly, largely by AI assistants, intended for fun slightly than function. He posted the result, a repository called "LLM Council," to GitHub with a stark disclaimer: "I’m not going to support it in any way… Code is ephemeral now and libraries are over."
Yet, for technical decision-makers across the enterprise landscape, looking past the casual disclaimer reveals something much more significant than a weekend toy. In a couple of hundred lines of Python and JavaScript, Karpathy has sketched a reference architecture for essentially the most critical, undefined layer of the trendy software stack: the orchestration middleware sitting between corporate applications and the volatile market of AI models.
As corporations finalize their platform investments for 2026, LLM Council offers a stripped-down have a look at the "construct vs. buy" reality of AI infrastructure. It demonstrates that while the logic of routing and aggregating AI models is surprisingly easy, the operational wrapper required to make it enterprise-ready is where the true complexity lies.
How the LLM Council works: 4 AI models debate, critique, and synthesize answers
To the casual observer, the LLM Council web application looks almost equivalent to ChatGPT. A user types a question right into a chat box. But behind the scenes, the applying triggers a complicated, three-stage workflow that mirrors how human decision-making bodies operate.
First, the system dispatches the user’s query to a panel of frontier models. In Karpathy’s default configuration, this includes OpenAI’s GPT-5.1, Google’s Gemini 3.0 Pro, Anthropic’s Claude Sonnet 4.5, and xAI’s Grok 4. These models generate their initial responses in parallel.
Within the second stage, the software performs a peer review. Each model is fed the anonymized responses of its counterparts and asked to judge them based on accuracy and insight. This step transforms the AI from a generator right into a critic, forcing a layer of quality control that’s rare in standard chatbot interactions.
Finally, a delegated "Chairman LLM" — currently configured as Google’s Gemini 3 — receives the unique query, the person responses, and the peer rankings. It synthesizes this mass of context right into a single, authoritative answer for the user.
Karpathy noted that the outcomes were often surprising. "Very often, the models are surprisingly willing to pick one other LLM's response as superior to their very own," he wrote on X (formerly Twitter). He described using the tool to read book chapters, observing that the models consistently praised GPT-5.1 as essentially the most insightful while rating Claude the bottom. Nonetheless, Karpathy’s own qualitative assessment diverged from his digital council; he found GPT-5.1 "too wordy" and preferred the "condensed and processed" output of Gemini.
FastAPI, OpenRouter, and the case for treating frontier models as swappable components
For CTOs and platform architects, the worth of LLM Council lies not in its literary criticism, but in its construction. The repository serves as a primary document showing exactly what a contemporary, minimal AI stack looks like in late 2025.
The applying is built on a "thin" architecture. The backend uses FastAPI, a contemporary Python framework, while the frontend is a typical React application built with Vite. Data storage is handled not by a fancy database, but by easy JSON files written to the local disk.
The linchpin of the complete operation is OpenRouter, an API aggregator that normalizes the differences between various model providers. By routing requests through this single broker, Karpathy avoided writing separate integration code for OpenAI, Google, and Anthropic. The applying doesn’t know or care which company provides the intelligence; it simply sends a prompt and awaits a response.
This design selection highlights a growing trend in enterprise architecture: the commoditization of the model layer. By treating frontier models as interchangeable components that might be swapped by editing a single line in a configuration file — specifically the COUNCIL_MODELS list within the backend code — the architecture protects the applying from vendor lock-in. If a brand new model from Meta or Mistral tops the leaderboards next week, it could be added to the council in seconds.
What's missing from prototype to production: Authentication, PII redaction, and compliance
While the core logic of LLM Council is elegant, it also serves as a stark illustration of the gap between a "weekend hack" and a production system. For an enterprise platform team, cloning Karpathy’s repository is merely step certainly one of a marathon.
A technical audit of the code reveals the missing "boring" infrastructure that industrial vendors sell for premium prices. The system lacks authentication; anyone with access to the net interface can query the models. There isn’t a concept of user roles, meaning a junior developer has the identical access rights because the CIO.
Moreover, the governance layer is nonexistent. In a company environment, sending data to 4 different external AI providers concurrently triggers immediate compliance concerns. There isn’t a mechanism here to redact Personally Identifiable Information (PII) before it leaves the local network, neither is there an audit log to trace who asked what.
Reliability is one other open query. The system assumes the OpenRouter API is all the time up and that the models will respond in a timely fashion. It lacks the circuit breakers, fallback strategies, and retry logic that keep business-critical applications running when a provider suffers an outage.
These absences should not flaws in Karpathy’s code — he explicitly stated he doesn’t intend to support or improve the project — but they define the worth proposition for the industrial AI infrastructure market.
Firms like LangChain, AWS Bedrock, and various AI gateway startups are essentially selling the "hardening" across the core logic that Karpathy demonstrated. They supply the safety, observability, and compliance wrappers that turn a raw orchestration script right into a viable enterprise platform.
Why Karpathy believes code is now "ephemeral" and traditional software libraries are obsolete
Perhaps essentially the most provocative aspect of the project is the philosophy under which it was built. Karpathy described the event process as "99% vibe-coded," implying he relied heavily on AI assistants to generate the code slightly than writing it line-by-line himself.
"Code is ephemeral now and libraries are over, ask your LLM to vary it in whatever way you want," he wrote within the repository’s documentation.
This statement marks a radical shift in software engineering capability. Traditionally, corporations construct internal libraries and abstractions to administer complexity, maintaining them for years. Karpathy is suggesting a future where code is treated as "promptable scaffolding" — disposable, easily rewritten by AI, and never meant to last.
For enterprise decision-makers, this poses a difficult strategic query. If internal tools might be "vibe coded" in a weekend, does it make sense to purchase expensive, rigid software suites for internal workflows? Or should platform teams empower their engineers to generate custom, disposable tools that fit their exact needs for a fraction of the fee?
When AI models judge AI: The damaging gap between machine preferences and human needs
Beyond the architecture, the LLM Council project inadvertently shines a lightweight on a particular risk in automated AI deployment: the divergence between human and machine judgment.
Karpathy’s commentary that his models preferred GPT-5.1, while he preferred Gemini, suggests that AI models could have shared biases. They may favor verbosity, specific formatting, or rhetorical confidence that doesn’t necessarily align with human business needs for brevity and accuracy.
As enterprises increasingly depend on "LLM-as-a-Judge" systems to judge the standard of their customer-facing bots, this discrepancy matters. If the automated evaluator consistently rewards "wordy and sprawled" answers while human customers want concise solutions, the metrics will show success while customer satisfaction plummets. Karpathy’s experiment suggests that relying solely on AI to grade AI is a technique fraught with hidden alignment issues.
What enterprise platform teams can learn from a weekend hack before constructing their 2026 stack
Ultimately, LLM Council acts as a Rorschach test for the AI industry. For the hobbyist, it’s a fun approach to read books. For the seller, it’s a threat, proving that the core functionality of their products might be replicated in a couple of hundred lines of code.
But for the enterprise technology leader, it’s a reference architecture. It demystifies the orchestration layer, showing that the technical challenge just isn’t in routing the prompts, but in governing the information.
As platform teams head into 2026, many will likely find themselves looking at Karpathy’s code, to not deploy it, but to know it. It proves that a multi-model strategy just isn’t technically out of reach. The query stays whether corporations will construct the governance layer themselves or pay another person to wrap the "vibe code" in enterprise-grade armor.
