How Meta built an AI agent to permanently clone domain expert knowledge

The 6-Week AI Agent That Gets Permanently Smarter

Meta's compliance team used to spend days on individual product assessments. Six weeks ago they put a new internal AI agent into production. Now those assessments take minutes. But the speed isn't the story. The story is that every time a human expert corrects the agent, that correction gets compiled into a permanent, regression-tested update to the organization's knowledge base. No model retraining. No prompt engineering. The system gets smarter, and it stays smarter.

This is not RAG. This is not fine-tuning. Meta calls it an organizational second brain, and before you get excited, you need to know exactly who this is actually for.

If your organization treats documentation as something you write once and forget, you can stop listening. This architecture will not save you. If you have teams of domain experts answering the same complex, high-stakes questions hundreds of times a year, and you're willing to curate what they know into a structured file system with dependency graphs and YAML frontmatter, then keep listening.

The pattern is domain-independent. Compliance, security review, financial risk, engineering standards. But the commitment is the same everywhere.

A Four-Layer Architecture

The system has four layers. A knowledge system. A reasoning pipeline. An evaluation framework. And a self-improvement loop that connects them.

The Knowledge System: Curation Over Retrieval

Most companies trying to build domain-specific agents dump source documents into a vector database and hope retrieval catches the right chunk at inference time. Meta's team argues that this forces the model to re-derive institutional reasoning from raw text on every single run. That's slow, inconsistent, and error-prone.

Instead, they run a long-running offline process that distills documents into curated knowledge files.

• Position files capture authoritative organizational stances.
• Taxonomy files enforce a single source of truth for domain language.
• Routing indexes map input characteristics to relevant positions deterministically, not through embedding similarity.
• Gateway files define threshold tests that prevent the agent from applying specialized knowledge where it doesn't belong.

Every file declares its dependencies and consumers in YAML frontmatter. Change one position file, and the bidirectional dependency graph tells you exactly what else might be affected. This matters because the system is explicitly designed to edit its own files.

The Reasoning Pipeline: Separating 'What' from 'How'

The reasoning layer runs on composable procedures called recipes. Knowledge files are declarative. They say what is true. Recipes are imperative. They say how to think. The separation is strict. A recipe references knowledge files but contains no domain facts. A knowledge file states a position but prescribes no procedure.

This means failures attribute cleanly. Did the agent have the right information but reach the wrong conclusion? That's a recipe problem. Did it fail because the source materials didn't contain the answer? That's a knowledge gap. This separation also means you can update organizational positions without rewriting analytical workflows, and you can fix methodology without touching domain facts.

Recipes compose into pipelines. A top-level routing recipe examines the input and delegates to sub-recipes for each phase of analysis. Early versions of the system used a single flat instruction file with semantic retrieval, dumping a massive volume of mixed-relevance context into the window on every turn. After restructuring into staged recipes, token consumption per turn dropped by roughly eighty percent. Context windows are finite. Attention degrades with volume. Progressive disclosure isn't just efficient. It improves reasoning quality.

Human Experts in Control

Human experts stay in control through two mechanisms. Checkpoints are defined moments where the agent surfaces intermediate reasoning for review before continuing. Escalations trigger when the agent hits genuine ambiguity from underspecified inputs or evidence that supports multiple defensible readings. It stops and asks. The expert's choice determines the path. These aren't just safety rails. They're training signals.

The Self-Improvement Loop

Every correction and every escalation feeds the improvement loop. That loop is the most distinctive part of the architecture. Raw expert feedback comes from conversation traces. The system extracts every substantive signal alongside the agent's full knowledge manifest. Then it applies one test. Could the agent have reached the correct conclusion from its loaded materials? If yes, the recipe is flawed. If no, there's a knowledge gap. If experts themselves disagree, it's flagged as ambiguity for human discussion.

A compiler then translates each issue into minimal file edits. Sub-agents analyze impact in parallel. But the trust mechanism is an independent adversarial review. A separate agent, running with no knowledge of the improvement rationale, sees only the proposed diffs. Its sole job is to find contradictions, broken edge cases, or undermined positions. Because it shares no context with the proposing agents, it cannot inherit their blind spots.

There's also a deterministic linter. Dangling cross-references. File size budget violations. Identifier collisions. Dependency cycles. This layer is not probabilistic. It passes or it fails.

Every proposed change then faces two-stage validation. Targeted replay runs the agent on the original scenario. A separate judge evaluates the new output against the expert feedback without knowing what was changed. If this fails, compilation retries. Then regression testing runs the domain benchmark suite. If anything regressed, compilation retries again with an updated prompt describing the failure. Only then does a human expert review a pull request with a complete audit trail. Once approved and landed, the original failing scenario and its correct answer are automatically added to the regression suite.

That's how Meta claims zero regressions across improvement cycles. Every fix permanently raises the bar.

The Counterweight: An Expensive Bet

After three development sprints spanning six weeks, domain SMEs rated the agent's output useful almost all the time. The agent handles the vast majority of analytical work, freeing experts for genuinely ambiguous cases where human judgment matters most.

Now, the counterweight. This architecture makes a very specific and expensive bet. It bets that your organization is willing to maintain over two hundred interdependent knowledge files organized in a strict taxonomy. It bets that you have the engineering resources to build what is essentially a compilation and continuous integration pipeline for institutional knowledge. It bets that your domain experts have the bandwidth to review checkpoints, handle escalations, and audit automated pull requests instead of just completing the assessments themselves.

Meta is explicit about this. They state that the number of interdependent files makes manual maintenance impossible to scale at their size, which is why they built the automated loop in the first place. But that loop requires adversarial review agents, deterministic linters, replay judges, and growing benchmark suites.

If you don't already have teams that treat knowledge curation as infrastructure, this isn't a product you install. It's a system you build from scratch, and it needs a gardener.

Core Adoption Requirements

The adoption requirements are clear.

• You need a structured knowledge system with explicit file boundaries and cross-references.
• You need a procedural layer that separates what the agent knows from how it reasons.
• You need an automated evaluation suite that compounds with each improvement cycle.
• And you need human-in-the-loop checkpoints calibrated to your actual risk tolerance.

The underlying principle is sound. Keep complexity in human-readable text files under version control, not buried in model weights. Make every improvement a diff that an expert can review in thirty seconds. But only commit to this if your organization already believes that expertise is infrastructure worth maintaining. If your experts are too busy to structure their own knowledge, this second brain will starve for nutrients fast.

TAKEYOURPILLS DOT TECH. Go ship something.