Explore software through perspectives and runtime QA

Use perspective-based prompting and discardable QA files for non-obvious user paths; Apply the 5 Monkeys technique and use CMUX state for context-aware shortcuts; Extract implementation intent from narrated screen recordings into Gemini.

Original source pages and recordings require authorized access. The full article text here is public.

The practice is using agents as perspective-driven explorers instead of asking one generic agent to test or research everything. John shows how lenses like expert, new user, grandma, and power user can shape research and QA, how discardable QA files can push agents through non-obvious app paths, and how rough narrated screen recordings can become implementation instructions. He also explains how CMUX state can power context-aware shortcuts, why he often avoids worktree overhead, and where rigorous tests still belong: stable business logic rather than fast-changing UI experiments.

Working outcomes

  • Use perspective-based prompting and discardable QA files for non-obvious user paths
  • Apply the 5 Monkeys technique and use CMUX state for context-aware shortcuts
  • Extract implementation intent from narrated screen recordings into Gemini

Concepts and consequences

Perspective-based prompting

Instead of asking one generic agent to research or test your app, assign it a lens: expert, new user, grandma, power user, or another real audience. That lens changes what the agent notices and which assumptions it challenges.

Why it matters: It helps you escape creator bias and shape the output around the actual demographics or user types your application needs to support.

Discardable QA files

A QA file is a temporary assignment document for an agent: a numbered list of things to try, paths to walk, buttons to find, and a perspective to act from. John treats these as created in the moment, not as permanent test infrastructure.

Why it matters: They let you test the current shape of a fast-changing product without locking every changing UI decision into a brittle long-term test suite.

The 5 Monkeys technique

The goal is to have agents use the app in non-obvious ways: find buttons, click around, try shortcuts, press back after reaching a page, and look for modes or state that stay open unexpectedly.

Why it matters: This catches UX and state issues that a normal verification prompt may miss because the model tends to follow the same visible patterns the developer already expects.

CMUX state for context-aware shortcuts

John describes a CMUX inspect-style command that exposes the multiplexer state as an object, including the focused pane and that pane’s current working directory. He uses that context with a Karabiner shortcut to open tools over the terminal he is already focused on.

Why it matters: It turns the terminal layout into an automation surface: a shortcut can open an editor, run Git commands, start debugging, or trigger other tools in the project you are currently looking at.

Video-driven intent extraction

John records a rough screen interaction, narrates UI changes aloud, and drops the video into Gemini to turn it into AI instructions for implementation.

Why it matters: It lowers the cost of explaining visual changes when pointing, circling, and talking are faster than writing a polished written spec.

Reference tools and resources

Agent Browser

An experimental Vercel Labs browser-agent project John named as his go-to tool for controlling sites, clicking through pages, inspecting DOM state, and using Chrome DevTools-style information.

Use when: Reach for it when you want an agent to verify UI behavior in an actual browser. In this section, John described the tool but said he would dig into it more later.

Reference

ClickLight

A desktop overlay tool John briefly demonstrated for showing clicks, cursor actions, drawings, and keyboard shortcuts during screen shares.

Use when: Use it when recording or presenting a workflow where viewers need to see what you clicked or typed. It was a supporting presentation tool in this section, not the core QA technique.

Reference

CleanShot

A screenshot and screen recording tool John used to capture a quick narrated UI-change video.

Use when: Reach for it, or any similar screen recorder, when a visual change is easier to show and narrate than to describe in text.

search: CleanShot screen recording app

Gemini video prompting

John used Gemini for video understanding by dropping in a screen recording and asking it to convert the interaction into AI instructions.

Use when: Use it when your intended change is easier to communicate through a narrated screen recording, especially for visual layout work.

search: Gemini video understanding prompt screen recording instructions

Tailscale

A mesh networking tool John used to connect to another Mac and SSH into it from his local workflow.

Use when: Use it when you need private remote machine access from your local agent or terminal workflow without exposing services broadly.

Reference

Karabiner-Elements

A macOS keyboard customization tool John used as part of a global shortcut workflow that opened an editor/search overlay in the context of the active terminal.

Use when: Reach for it when you want app-global shortcuts that can trigger workspace-aware scripts or tools from wherever you are focused.

search: Karabiner-Elements macOS keyboard customizer

CMUX inspect

A CMUX inspection pattern John described as returning an object for the multiplexer state, including the focused pane and current working directory.

Use when: Reach for this pattern when you want shortcuts or agents to act on the exact terminal pane and project you are currently focused on.

search: CMUX inspect focused pane current working directory

Recommendations and best practices

  • Create separate research or QA prompts for expert, new-user, power-user, and inexperienced-user perspectives instead of asking one generic agent to cover everything.
  • Write QA files as disposable numbered assignments, not permanent monuments; keep them cheap enough that you are willing to delete and regenerate them.
  • Tell agents to explore non-obvious paths explicitly: find buttons, try shortcuts, press back after navigation, change modes, and look for state that stays open unexpectedly.
  • Keep rigorous tests for stable business logic and locked-down parts of the system; avoid over-testing UI areas you still expect to reshape frequently.
  • Use browser-driving tools when the task needs real clicking, DOM inspection, or performance/devtools signals rather than code inspection alone.
  • When a UI change is easier to show than write, record a short narrated screen video and ask a multimodal model to turn it into implementation instructions.
  • Watch disk usage when running many agents or worktrees on compiled projects; build artifacts such as Rust target directories can balloon fast.
  • For performance-sensitive apps, have QA agents capture memory, performance, API-call, refresh, and latency signals while they walk through the app if your tooling supports it.

Practice and acceptance criteria

Practice using agents as cheap, perspective-driven explorers so you can find UI blind spots without turning every fast-changing experiment into permanent test infrastructure.

Checklist

  1. Pick one feature or screen in your current app that has enough UI state to be interesting: navigation, modes, shortcuts, buttons, or other interactions.
  2. Create four short perspective prompts for the same feature: expert user, first-time user, inexperienced user, and power user.
  3. Write a temporary QA file with numbered tasks that explicitly ask agents to find buttons and try non-obvious paths such as shortcuts, back navigation, mode changes, and partial navigation sequences.
  4. Run separate agent or browser-verification passes against the same feature using different perspective prompts.
  5. Review the outputs for concrete non-obvious paths, stuck states, confusing UX, or repeated happy-path behavior that did not teach you anything new.
  6. For each useful finding, decide whether it belongs in a durable test because it covers stable business logic, or whether it should stay flexible because the UI is still changing.
  7. For one visual UI change, record a short narrated screen video and ask a multimodal model to turn it into implementation instructions.
  8. Inspect your terminal or multiplexer state as structured data if your setup supports it, and note which fields would let a shortcut target the correct working directory automatically.
  9. If you run many agents, worktrees, or compiled builds, watch for large generated artifacts and decide whether you need a disk-usage safeguard.

Four-Lens QA Pass

Difficulty: Easy

Choose one UI flow and create discardable QA prompts for expert, new-user, inexperienced-user, and power-user perspectives. Run an agent pass for each perspective and compare what each one notices.

Done when: You have at least one finding or observation that appears in only one perspective's output, showing that the lens changed the exploration.

Mini 5 Monkeys Run

Difficulty: Medium

Write a numbered QA assignment that tells agents to explore non-obvious behavior: find buttons, try shortcuts, use back navigation, move through partial flows, change modes, and look for state that stays open unexpectedly.

Done when: You end with notes about any surprising paths, stuck states, mode issues, or confirmation that the non-happy-path pass did not reveal a useful issue.

Video Intent to Instructions

Difficulty: Medium

Record a short screen video of a UI you want changed while narrating what should change visually, such as button styling, section order, or header placement. Feed it to a multimodal model and ask for implementation instructions, then review the useful parts before turning them into an agent prompt.

Done when: The resulting prompt contains concrete visual changes and references to the relevant UI elements without requiring another person to watch the video.

Single-Branch Agent Coordination With Safeguards

Difficulty: Hard

Prototype a single-branch multi-agent workflow for one feature: ask a manager agent to assign bounded parallel tasks, keep the agents in the same project instead of adding worktrees by default, and add a simple disk-usage check for large generated build artifacts.

Done when: You can run exploratory agent work without unnecessary worktree overhead, with clear task boundaries and a visible safeguard against runaway build artifacts.

Questions to test understanding

Why does perspective-based prompting usually produce better QA findings than asking one generic agent to test the app?

  • A. It changes what the agent notices, which assumptions it questions, and which paths it tries.
  • B. It guarantees the agent will write production-ready integration tests.
  • C. It makes the agent reuse your existing browser cookies automatically.
  • D. It prevents the agent from exploring anything outside the happy path.

Answer: A. A persona or lens pushes the agent toward different risks and user behaviors. An expert, new user, grandma, and power user will naturally stress different parts of the same interface.

What makes a discardable QA file useful in this lesson's workflow?

  • A. It becomes the final source of truth for every future regression test.
  • B. It gives an agent a temporary numbered mission that can be regenerated as the UI changes.
  • C. It replaces all business-logic tests permanently.
  • D. It stores private auth cookies so browser agents can skip login.

Answer: B. The point is cheap scaffolding: numbered tasks, a perspective, and non-obvious paths for the current version of the product. It should be easy to delete or rewrite.

Which instruction best matches the 5 Monkeys technique?

  • A. Confirm that the signup button works using only the documented happy path.
  • B. Inspect the codebase and summarize every component dependency.
  • C. Find buttons, try shortcuts, press back after navigation, change modes, and look for surprising state behavior.
  • D. Write a complete Playwright suite before looking at the UI.

Answer: C. The 5 Monkeys idea in this section is to push agents toward non-obvious app usage rather than one careful happy-path check. John’s example was a shortcut leading to a page, then pressing back and noticing a mode stayed open.

Why is exposing CMUX state as an object powerful for a Codex-style workflow?

  • A. It lets a shortcut or agent target the focused pane and current working directory without manual setup.
  • B. It automatically fixes merge conflicts across all branches.
  • C. It encrypts remote SSH traffic without any other tool.
  • D. It converts screen recordings into implementation instructions.

Answer: A. Structured pane state turns your terminal layout into an automation surface. A global shortcut can launch the right editor, Git command, debugger, or other tool in the context you are already focused on.

What is the safest way to apply the video-to-instructions workflow from the lesson?

  • A. Record a rough interaction, narrate the intended UI changes clearly, ask a multimodal model for implementation steps, then review the output before coding.
  • B. Record a video silently and let the model infer all business requirements.
  • C. Upload sensitive production data because video prompts are always private by default.
  • D. Treat the model's generated checklist as production code with no human review.

Answer: A. The video only needs to be clear enough to communicate visual intent, especially with narration and pointing. The generated instructions are a draft to inspect, not a substitute for judgment.

Reflection prompts

  • Where are you currently over-testing a fast-changing UI when a discardable QA file would give you faster learning?
  • Which user perspective is most underrepresented in your current development process, and what would that person probably try that you would not?
  • What parts of your local workflow could become automation surfaces if your focused pane and working directory were available as structured state?

Further applications

  1. Build a four-perspective prompt matrix for one feature and compare what each persona notices that the others miss.
  2. Create a throwaway QA file with numbered non-obvious path tasks and run agent passes against the same UI.
  3. Prototype a video-to-instructions loop: record 30 seconds of pointing and narration, then ask a multimodal model for junior-developer implementation steps.
  4. Add lightweight performance logging to QA runs so agents capture memory, API calls, refreshes, and latency while they explore.
  5. Experiment with a single-branch multi-agent workflow before adding worktrees, and only add worktrees when isolation clearly beats the coordination overhead.

Applied exercises

Perspective Prompt Matrix Blueprint

Objective: Construct a flexible research template config folder containing four distinct system prompts: persona-expert.md, persona-novice.md, persona-grandparent.md, and persona-poweruser.md. Write an execution bash script using a local LLM CLI tool to process an identical feature specification document across all four prompts, outputting isolated markdown results into a retrieval/ target folder.Video-to-Code Instructional Pipeline

Objective: Replicate John’s rapid layout prototyping pipeline. Use any native capture tool (e.g., CleanShot, QuickTime) to record a 30-second screen interaction of an existing local frontend interface. Explicitly dictate audio adjustments (e.g., color alterations, structural re-ordering). Process the raw output file directly through a long-context multimodal model like Gemini, and evaluate the clarity of the resulting markdown instructions.

Questions and answers

When generating a lot of research, what guiding principles do you reach for for staying organized, as well as optimized for the retrieval?

John relies on the core concept of perspective-based prompting. Instead of collecting unstructured data dumps, he assigns specific personas (expert, new user, grandparent, power user) to separate data pipelines. He also generates modular, numbered QA assignment files that instruct agents to discover non-obvious app paths, keeping the output highly targeted and organized.

do you have a goto system for agents to test and verify via browser? i have issues, with auth, etc...

John uses Agent Browser, an open-source experimental project originating from Vercel Labs. It excels at driving Chrome dev tools, analyzing performance metrics, and navigating complex layouts. Note that as pointed out in the chat session, it does not implicitly forward local browser cookies, so prompts must include explicit onboarding steps or registration rules for authentication sequences.

do you use worktrees? shortcut that copies over .env as well? and shut them down?

John used to rely on Git worktrees when models were less capable, but has mostly abandoned them. Because his recent projects at Vercel have been isolated solo assignments, he prefers working directly out of a single main branch or standard PR branch. Instead of juggling port mappings and setting up environment configurations for separate worktrees, he lets parallel agents operate within bounded directories in the same branch. This avoids a known issue where massive Rust compilation targets duplicate disk usage and exhaust disk space.

is screendrop what you were using when you were sending stuff to other mac using oracle? you could see your other mac in realtime?

No, Screendrop is a separate open-source, local alternative to Clean Shot that John is tracking. The remote display synchronization shown earlier was simply Apple’s native mac OS Screen Sharing app routed securely across a private Tailscale mesh network connection.

What are your thoughts on testing? These QA files seem like a partial replacement for rigorous testing suites.

John confirms he increasingly leans toward discardable runtime QA files over heavy test suites. Because code generation tools make changing software quick and trivial, over-indexing on rigid testing setups creates friction against iteration. He advises keeping strict test suites strictly reserved for fixed business logic that will not change, while using flexible QA agents running overnight loops to handle fluid visual interface verification.

So, it would be interesting to see how you keep track. So, is it just that you have the different conversations pinned to the left, and then you look at them to keep an overview, or how do we do it? And then, especially this research, planning, building part, and the specialized agents?

John begins to answer as the chunk cuts off, explaining that tracking specialized agents involves coordinating dedicated task-based UI interfaces and distinct workspace contexts.

The continuation uses a primary orchestrator and distinct focused panes for quality control, performance, research, and note-taking.

Operational playbooks

Recording references

Timestamped source page

Visual reference index

  • 01:41:00: Agent Browser GitHub documentation display on Chrome
  • 01:42:40: Editor view showing generated website update instructions and text/code edits
  • 01:45:50: GitHub/repository picker or file search view with ClickLight-related repository visible
  • 01:46:45: Signal Knit hoodie site preview with layout/selection overlay visible
  • 01:48:15: Browser/editor split with Signal Knit update instructions modal over the page
  • 01:54:10: Code/editor workspace with design asset sheet visible at right
  • 01:58:55: Code/editor workspace with command palette or modal over design/code context
  • 01:59:55: Code/editor workspace with asset sheet and generated implementation text visible

Source references