Show, Don't Tell: AI Agents Are Now Filming Their Own Demos

“Show, don't tell” has long been the golden rule of storytelling, but it is rapidly becoming the guiding principle for Artificial Intelligence as well. As we transition from AI chatbots that simply answer questions to autonomous agents that write code and navigate web applications, a critical problem emerges: the black box. When an AI tells you it has successfully completed a complex task, how can you be sure it didn't break something along the way?

Developer Simon Willison recently tackled this exact problem with a new feature in his open-source tool, shot-scraper. The new 'video' command acts as a virtual camera crew for AI agents. Instead of just executing code invisibly in the background, an AI can now write a detailed "storyboard"—a simple text file that dictates every mouse click, keystroke, and deliberate pause. The tool then drives a web browser to perform these actions while recording a pristine, high-quality video of the entire process.

For example, if an AI is tasked with building a new feature that imports raw data into a database, it can now hand the human developer both the finished code and a crisp video demonstrating the feature working in real-time.

However, the story of how this tool was built is perhaps even more fascinating than the tool itself. Willison relied heavily on an advanced AI model to write the underlying code. In a brilliant inversion of traditional software engineering, he had the AI write the user documentation first. By reviewing the AI-generated manual, Willison could easily spot confusing design choices and refine the tool's architecture before the actual coding began.

Furthermore, this project highlights a subtle but massive shift in how software is designed. Willison ensured that the tool's built-in help menu was incredibly detailed. Why? Because the AI agent was able to read that help menu and instantly teach itself how to use the tool it had just helped create.

We are entering an era where software interfaces are being optimized not just for human eyes, but for AI comprehension. By giving AI agents the ability to operate tools and record visual proof of their actions, we are bridging the trust gap between human supervisors and their digital assistants.

Key Points

• A new tool called shot-scraper video lets AI agents record screencasts of their web interactions.
• The AI generates a storyboard script that dictates clicks and typing, which the tool turns into a video demo.
• The tool was co-developed with an AI, which wrote the documentation first to help refine the software's design.
• By embedding detailed instructions in the software, AI agents can teach themselves how to use new tools autonomously.

Why It Matters

As AI agents take on more complex, autonomous tasks, the ability to provide visual proof of their work is essential for building trust and ensuring accountability.

Sources:

• Have your agent record video demos of its work with shot-scraper video — Simon Willison's Weblog