
Large language models fail when asked to generate structurally rigid, legacy file formats like Excel or PowerPoint, as a single hallucinated XML tag or unchecked formula dependency corrupts the entire deliverable.
- Agent Harness: Use models like GPT-5.6 Sol strictly for reasoning, planning, and task routing rather than raw code generation.
- Deterministic Execution: Offload mathematical calculations to a secure Python environment and use specialized APIs to safely construct the proprietary file structure.
- Visual Evaluation: Implement a feedback loop where the agent visually reviews generated layouts and charts to catch rendering errors before final delivery.
This architecture forces human operators to shift from builders to auditors, introducing significant cognitive load and trust boundary risks when debugging bespoke, machine-generated logic.
Script
Picture this. A product manager asks you to build an export feature. They need a multi-tab Excel workbook with working, linked formulas. You groan. You immediately remember the brittle, miserable nightmare of using openpyxl or trying to hand-roll Office Open XML. You know that one misplaced tag corrupts the entire file.
Now, imagine asking a language model to generate that file from scratch. It's a disaster waiting to happen. Generating text is forgiving. A missing comma in a paragraph doesn't break the page. But generating a structurally rigid, proprietary, legacy file format is exactly what language models are notoriously bad at.
Text is just a string. An Excel formula linking to a hidden sheet, driving a chart on a third sheet, is a directed acyclic graph masquerading as a zip file of XML documents. One hallucinated bracket, and Microsoft Office refuses to open the file.
An Engineering Blueprint for Generating Legacy Files
An OpenAI case study just came out detailing how a company called Model ML generates native Excel models and PowerPoint decks using GPT-5.6 Sol. We are ignoring the product pitch for finance teams today. We are looking at this purely as an engineering blueprint. It's a masterclass in how to force a non-deterministic model to build fully functional, editable legacy files.
Your first instinct, when a stakeholder asks for an AI-generated presentation, is probably to write a massive prompt with a strict JSON schema. You pass that schema to a single model. You hope the output maps cleanly to a rendering engine on your backend. But that breaks down fast. Finance decks and valuation models have hundreds of interlocking dependencies. A single prompt can't hold the context, the data integration rules, the formatting constraints, and the mathematical logic without losing the thread.
The Core Takeaway: The Agent Harness
Model ML abandoned the massive single prompt. Instead, they built an agent harness. This is the core architectural takeaway. The harness explicitly separates the reasoning from the actual file construction.
GPT-5.6 Sol acts as the core planning agent. It routes tasks and keeps the original brief in context. But the agent doesn't write raw XML. It doesn't output final JSON directly into a .pptx file. The harness dynamically loads specialized toolkits based on what the agent needs at that exact second. This is how you bridge the gap.
- When the agent needs to pull data from a massive virtual data room containing hundreds of files, it loads a data integration toolkit.
- When it needs to calculate cash flows or reconcile evidence, it doesn't just guess the math. Language models are terrible at math. Instead, the agent loads a code execution environment. It writes Python scripts to run the calculations securely.
- And when it comes time to actually build the file, it hands the validated data off to specialized document editing tools.
The toolkits do the heavy lifting of interacting with the Office file APIs. It's a strict boundary. You let the language model reason, and you let deterministic code build the file.
Enforcing Quality with a Visual Evaluation Loop
They take the rigid structure enforcement a step further. For presentations, they built a visual evaluation loop. Before the agent returns a finished PowerPoint file, it reviews every single slide visually. It checks the layout, the visual hierarchy, and the chart legibility. If a chart is flattened or unreadable, the loop catches it. The model reviews the visual output of the deterministic tool.
The Reality: Failure Rates and Human Oversight
But here is where this blueprint hits the reality of production. The benchmark numbers in this case study require a very close read.
Model ML claims the system delivers review-ready finance deliverables. Their own composite evaluation shows GPT-5.6 Sol cleared the professional-readiness gate for PowerPoint in only 43.3 percent of cases. For Excel creation, the model produced fully correct models exactly 50 percent of the time.
Let those numbers sink in. In more than half of the PowerPoint generations, the output is flawed or not ready for review. Half of the Excel files have a key output wrong. How do you handle a fifty percent failure rate in a production workflow? Especially when the operational cost of generating a single Excel workbook averages 2.44 million tokens.
You handle it by changing the human's job description. This architecture requires a human operator sitting at the end of the line. The workflow explicitly relies on a finance professional checking assumptions, sources, and every formula. You are retraining analysts to shift from builders to auditors.
The High Cost of Debugging and Other Hurdles
This is a massive workflow barrier. If you have ever tried to untangle an AI-generated spreadsheet logic error, you know it often carries a much higher cognitive load than just building the formula from scratch yourself.
A human analyst can be trained on bespoke firm preferences. They can be held accountable for specific logical errors. A machine fails unpredictably. If the logic is unique to a specific deal, the time you save on the initial generation is immediately lost in the debugging phase.
Information Security Hurdles
There are also serious infosec hurdles. Processing virtual data rooms with 100,000 rows across hundreds of files in one pass introduces data privacy questions. Virtual data rooms contain strict, NDA-bound material. Pushing that volume of highly confidential data through an LLM agent requires a massive trust boundary.
The Verdict: High-End Drafting, Not Full Automation
This architecture is not end-to-end automation. It's high-end drafting. The fine print acknowledges this. The professional must check the message.
But as an engineering pattern, the agent harness is exactly how you should approach legacy formats. If your primary bottleneck is formatting thousands of standard slides or doing initial data-entry from massive data rooms into standard templates, this pattern works. It reduces analyst assembly time from an hour to five minutes.
But if you are dealing with highly bespoke models where the logic is unique to the deal, or if you lack the operational capacity to meticulously audit machine-generated Excel formulas, skip it.
Final Takeaway: Separate Reasoning from Construction
When you need to generate complex, rigid file structures, don't ask the language model to write the file. Build a harness. Give the model a code execution environment to do the math. Give it a document editing toolkit to handle the proprietary file structures. Force it to visually review its own layout.
Separate the reasoning from the construction.
This is TAKEYOURPILLS.TECH. Go ship something.