Skip to content
How I built a pipeline to turn screen recordings into bug reports in one click

How I built a pipeline to turn screen recordings into bug reports in one click

6 min read Developer Tools

A developer shares the architecture behind BugCapture, a tool that automates the tedious process of writing bug reports. By combining screen recording with AI, it instantly translates visual bugs and reproduction steps into detailed, actionable tickets for engineering teams....

Subscribe to listen
audio-thumbnail
How I built a pipeline to turn screen recordings into bug reports in one click
0:00
/0
Clinical Summary
Diagnosis

Writing text prompts to explain complex visual bugs and layout shifts to AI agents is a slow, lossy process that wastes development time on ambiguous descriptions.

Prescription
  • Record Intent: Narrate and demonstrate the visual bug directly using standard browser media APIs instead of typing.
  • Process Locally: Use a background Node server with Whisper for offline transcription and FFmpeg to extract sequential frame captures.
  • Compile Context: Package the transcript, base64-encoded screenshots, and SSH-tailed server logs into a single Markdown payload.
Side Effects

Embedding raw base64 images drastically inflates LLM token costs, while managing local servers and remote credentials introduces significant adoption friction.

Script

A single Markdown file. Inside it: twenty images encoded as raw base64 text strings, a timestamped transcript of a developer speaking, and a raw dump of remote server logs. To a human, this file is an unreadable wall of garbage text. It looks like a corrupted email attachment. But to a multimodal AI agent, it is the optimal data structure for understanding a bug.

Picture this. It is Friday afternoon. You are clicking around a remote staging server and you spot a flaky visual state. You know exactly what is wrong. The second column is twelve pixels too wide, but only when the viewport is between 768 and 900 pixels, and only after you interact with the first input field.

You know the problem. You know exactly how it behaves. But typing out that sequence of events, and explaining the precise visual glitch to Claude or Copilot, takes longer than just opening your editor and fixing it yourself. Writing text prompts to explain visual bugs is a massive waste of time.

Your words are inherently ambiguous. You describe the layout shift. The AI replies with three clarifying questions because your description missed a detail about the DOM hierarchy or the parent container's flex properties. By the time you answer, you have already lost the time you were trying to save.

Bypassing Friction with a Bug Compiler

A standalone utility called BugCapture was built to bypass this exact friction. It records your screen and your microphone. But the tool itself is not the real breakthrough. Screen recorders are a solved problem. The genius here is the format it produces.

BugCapture operates on a very specific premise. Video files are terrible inputs for Large Language Models, and text descriptions are terrible for explaining visual bugs. Instead, the tool acts as a compiler.

The Three-Channel Evidence File

It turns a short browser recording into a three-channel evidence file. When you click record, BugCapture uses the standard Web MediaRecorder API to grab your screen and audio. You click through the broken interface. You narrate what you expect to happen versus what is actually happening. You trigger the bug. Then you hit stop.

At that exact moment, a local Node server catches the file. It splits the media stream and processes it into three parallel tracks of context.

  1. The first track is intent. The tool extracts your audio and runs it through a local Whisper model to generate a text transcript of what you were thinking and doing. This is the human context.

  2. The second track is visual state. The tool pipes the video file into ffmpeg. It extracts one single frame every three seconds. It converts those frames to 85-percent quality JPEGs, and strictly caps the total output at twenty frames. This prevents the payload from becoming infinite. This is the application context.

  3. The third track is runtime state. If you enable a feature called LogLens before you record, the local server opens a parallel SSH connection to your remote staging machine. It tails the relevant backend log files while you record your screen. This creates a timestamped record of server activity that matches the exact seconds you clicked the broken interface. This is the system context.

Finally, BugCapture compiles all three tracks into one Markdown document. The transcript sits at the top. The JPEGs are converted directly into base64 text strings and embedded sequentially. The raw logs are aligned at the bottom.

You drag that single Markdown file into your AI workspace. You do not type a prompt. You do not explain the viewport width or the CSS specificity. You just pass the file.

The AI now has three independent channels of information about the exact same event. The transcript explains your intent. The screenshots show the exact visual state over time. The logs confirm what the server did in response to your clicks. An AI reading all three builds a radically more accurate mental model than it ever could from an English paragraph.

Practical Questions and Limitations

This workflow is entirely local. But it raises some immediate practical questions.

  1. Does running local Whisper transcription actually work fast enough on a standard development laptop?

    Yes, it does. The transcription happens entirely offline. BugCapture uses the Xenova transformers library to run the Whisper base-English model via ONNX. There is no API key. There is no data upload to a speech-to-text service. On a standard modern laptop, a forty-seven-second recording transcribes in about eight seconds. The local processing bottleneck is non-existent. The Markdown file is compiled and ready before you even switch focus to your LLM window.

  2. How massive is the token footprint of embedding up to twenty base64 JPEGs into a single text file?

    This is the primary friction point of this architecture. Embedding images directly as base64 strings creates a gargantuan payload. An 85-percent quality JPEG of a standard browser window can easily translate into hundreds of thousands of text characters. When you multiply that by twenty frames, you are feeding the LLM an immense wall of text. Standard multimodal APIs from OpenAI or Anthropic have highly optimized pathways for image uploads. When you upload a PNG through their interface, they resize, tile, and compress the image to calculate a predictable token cost. By dumping raw base64 strings into a Markdown file, you are bypassing those optimizations entirely. You are forcing the model to process millions of raw text characters just to reconstruct the visual state. If you run this tool ten times a day, the API costs will spike, and you will blow out your context window limits rapidly. It is a highly inefficient use of your token budget.

  3. Is it worth the friction of running a local Node server just to capture bug reports?

    BugCapture is pitched as a zero-configuration utility for basic use. But the adoption friction is undeniably high. To get this one-click workflow, you must install Node. You must clone a repository. You must execute a bash setup script. You must ensure ffmpeg is correctly configured on your specific operating system. And most importantly, you must manually spin up a local Node server with no framework, and keep it running in the background every time you want to capture a bug. You also have to grant this local script SSH credentials if you want it to tail your remote logs. Tailing remote logs requires secure access. Managing local SSH keys just to feed a screen capture tool introduces a security dynamic that many engineering teams will reject outright.

Compare this setup to a tool like Jam.dev or Loom. Those are frictionless browser extensions. They live directly in your browser. They integrate directly with Linear or Jira. They are built for team communication. They do not require a persistent local Node process. They do not ask for SSH keys. They just work.

If you are dealing with standard backend exceptions that already produce clear stack traces, BugCapture is overkill. Paste the stack trace into your prompt and move on. If you operate in a regulated environment, you cannot paste production server logs and raw UI captures into a third-party LLM anyway. That is a massive compliance violation.

The Real Takeaway: An Experiment in Context

But focusing entirely on the adoption friction misses the point of the project. BugCapture is not trying to replace your standard bug tracker. It is an experiment in context design. It proves that our default method of interacting with LLMs is flawed.

We treat these models like chat bots. We sit at our keyboards and write massive paragraphs trying to translate visual states, CSS race conditions, and viewport edge cases into English text. It is a slow, lossy translation process.

This tool demonstrates that the most efficient prompt is often no text at all. It is a synchronized dump of raw intent, visual state, and runtime state. The optimal way to explain a complex bug to an AI is to stop typing, hit record, and let a script package the evidence.

This is TAKEYOURPILLS.TECH. Go ship something.

References

/