Feeding AI agents directly from traditional transactional databases or BI data lakes causes token bloat and context ambiguity. Without a structured data layer built for probabilistic reasoning, agents struggle with conflicting business definitions and rack up massive compute costs.
- Model Context Protocol (MCP): Wrap domain-owned data products in parameterized MCP tools instead of letting LLMs write their own raw SQL.
- Semantic Ontologies: Use RDF and OWL to assign explicit, unique identifiers to ambiguous business concepts to guarantee LLM precision.
- Dynamic Context: Implement dynamic tool search and format flat responses in CSV rather than JSON to drastically reduce context window token burn.
Building tiered latency layers, semantic ontologies, and dynamic tool orchestration introduces heavy infrastructure complexity that constitutes expensive premature optimization for small teams.
Script
Before you spend another sprint tweaking your agent's system prompt, check your data layer. If you're feeding your LLM from the same pipelines your BI dashboard drinks from, you're not just burning tokens. You're dumbing the agent down.
This episode is for the platform team that's been told to "add AI" to a transactional stack that's older than some of its maintainers. It's for the engineer who's watching their agent's context window fill up with verbose JSON schemas that cost more per call than the compute running the actual query.
If you're building a side project with a vector store and five PDFs, fair warning: this is heavier than you need. Bookmark it and come back when you've got fifty microservices and a finance department asking why the AI invoice tripled.
Picture this. It's Tuesday. Your internal RAG pipeline just went to production. You tested twenty queries you wrote yourself. All passed. A user asks something you didn't plan for. The LLM returns structured JSON that technically validates against your Pydantic model. Your downstream service silently corrupts a customer record.
What would have caught this? Not better prompting. The problem is that your data product called "active customer" meant one thing to your ERP and something else entirely to the agent. The model didn't hallucinate. It just never had a chance to know which definition you actually meant.
The Enterprise-Scale Data Problem
Fabiane Nardon runs data intelligence at TOTVS. That's a Brazilian tech giant you probably haven't heard of, unless you know that roughly a quarter of Brazil's GDP runs through their systems. Forty years of enterprise software. SaaS, on-premise, everything in between.
When they started building AI agents, their problem wasn't a lack of data. It was the exact opposite. They had oceans of data, and none of it was prepared for a token-hungry, latency-sensitive reasoning loop that can fire hundreds of unpredictable queries in a few minutes.
Transactional databases were built for applications with predictable access patterns. Data lakes were built for analysts running dashboards and batch reports. Neither architecture was built for a probabilistic reasoning engine that reads once, decides immediately, and can't tolerate a thirty-second warehouse delay.
A Framework for Data Retrieval
Nardon's framework for deciding where the agent reads from is straightforward. If the agent needs to write data back, or if it needs absolutely fresh data with zero delay, or if it needs to fire business rules that are locked inside the transactional system, you go straight to the source. If it can tolerate slightly stale data, needs historical processing, semantic search, or enrichment from external sources, you route it through a data platform.
But here's where most companies stop and hope for the best. They dump data into a warehouse, expose a generic get_schema tool, and let the LLM write its own SQL.
Nardon's team rejected that path. They designed their MCP tools to retrieve data by data product. Each tool has an owner, a stable interface contract, documentation, discoverability, and quality SLAs. Instead of one generic schema reader that vomits metadata into the context window, they have specific tools that know the business domain and retrieve data precisely.
The governance comes almost free because they were already running a data mesh. Domains own their data, so domains own their tools. When you have hundreds of tools floating around a company, knowing who maintains what isn't bureaucracy. It's survival.
The Semantic Ambiguity Problem
Walk into any company and ask three departments what an "active customer" is. Marketing gives you one answer. Finance gives you another. Operations gives you a third.
The problem isn't that you have multiple definitions. The problem is pretending there's only one. Nardon's team uses semantic web standards, RDF and OWL, to give each concept a unique identifier. Marketing's active customer gets one ID. Finance gets another. An ontology maps the relationships between them.
She cited a 2024 study that showed adding an ontology-based semantic layer improved LLM response precision by forty percent. The LLM doesn't need to guess which definition you meant. It follows the identifiers you explicitly gave it.
Now, here's where a staff engineer starts asking hard questions. Ontologies sound like 2001 academic Semantic Web homework. And for a lot of teams, they absolutely are overkill. If you have twelve tables and three engineers, you don't need RDF identifiers. You need better column names and a shared Slack channel.
But at enterprise scale, ambiguity isn't a bug in your schema. It's a structural guarantee. The ontology doesn't magically force the organization to agree on one definition. It simply stops the agent from pretending there's only one.
Solving for Latency
There's also the latency problem. Agents need low-latency retrieval, but traditional data platforms optimize for batch throughput and cost per terabyte, not milliseconds. TOTVS built three layers.
- A high-latency layer for bulk Spark jobs on Parquet files.
- A medium-latency layer in BigQuery for large analytical volumes.
- A low-latency layer using Postgres and DuckDB built specifically for agent queries.
They use Postgres triggers and stored procedures to fire transformation pipelines atomically the moment data lands. It isn't fashionable. It runs in milliseconds to seconds. And it's vendor-neutral enough to work across multiple clouds.
Engineering for Security and Cost
Security follows the same deterministic philosophy. Don't let the LLM generate queries. Full stop. Build parameterized tools with security logic embedded in the tool code itself. Then propagate identity through OAuth so the tool only returns rows the logged-in user is actually allowed to see. The agent authenticates against the company identity provider. The tool enforces row-level access. Not the prompt. Not the model.
Then there's cost. Token economics hit different when your local currency makes a hundred-dollar Claude bill look like a third of someone's monthly salary. Nardon's team treats token burn as an infrastructure cost to engineer away. They run what they call MCP Fabric. Instead of deploying a separate service for every MCP server, they run one Spring AI service that hosts hundreds of virtual MCP servers. Each agent gets its own MCP server containing only the tools it actually needs.
That alone cuts deployment overhead. But tool descriptions can still bloat the context window. So they built dynamic tool search. When the agent connects to its MCP server, it doesn't jam every tool description into the prompt. It sends the descriptions to a separate search service. When the agent needs to solve a problem, it calls search_tool. The service returns only the five relevant candidates using semantic search or even simple regex matching. The other ninety-five never touch the context window.
Their benchmark showed dramatic token savings as the tool count grew. They also format flat responses as CSV instead of JSON, cutting response tokens by up to half. They're testing a newer format called TOON that promises thirty to sixty percent savings, though they admit it's less battle-tested for precision since LLMs have seen far more CSV and JSON in their training data.
Is This Architecture For You?
This is where a staff engineer would push back again. This is a lot of infrastructure to own. If you have four tools and one internal agent, you don't need MCP Fabric. You don't need dynamic tool search. You probably don't need a dedicated low-latency Postgres layer. Most teams are nowhere near the scale where these optimizations matter. Building this prematurely is just expensive cosplay.
So who is this actually for? If you're running a complex enterprise system and your agents need to reason across multiple business domains with real security boundaries, you should be stealing pieces of this architecture now. Start with parameterized MCP tools. Stop letting LLMs write their own SQL. Filter your tool context per agent. Optimize your response formats for the shape of your data.
If you're a smaller team without a data mesh or domain boundaries, the full three-layer architecture with semantic ontologies is probably a distraction. You don't have the governance problem yet. Build clean data products first. The ontology can wait until you have three conflicting definitions of revenue breaking production.
The line between deterministic software and probabilistic AI is the most expensive architectural decision you'll make this year. Every piece of logic you push into a deterministic tool is a piece the model doesn't have to reason about. That means better precision, better security, and fewer tokens flying out your budget. Draw that line carefully.
Feed your agent data that was actually prepared for reasoning, not just for storage. Your finance team will notice. And more importantly, your agent will stop sounding like it skipped breakfast.
TAKEYOURPILLS.TECH. Go ship something.