Detailed companion accounts of concepts, questions, demonstrations, exercises, and unresolved observations.
These are the companion’s accounts of the recorded discussion. Model capabilities, subscription rules, prices, version-specific flags, and predictions are contextual claims, not current guidance. An observed action is not proof that the attempted result succeeded. Distinct answers and uncertainties remain separate.
Concepts and distinctions
Prompting with Images
Using annotated visual context (such as screenshots with hand-drawn arrows) to precisely identify specific components or UI sections when prompting vision-capable large language models or software agents.
Screen Annotation Workflow
A design/development shortcut involving a localized area capture tool that supports instant vector overlays (arrows, rectangles), streamlining the communication of UI components to an AI model with minimal steps.
Visual Prompting / Multimodal Input
Supplying an LLM or autonomous agent with annotated screenshots containing simple drawings like arrows to establish spatial or element context instead of relying entirely on raw string selectors.
Before/After Conversation Artifacts
Retaining original and updated web element screenshots inside chat history logs to allow easy visual diffs or visual reference verification by both the user and the agent.
Image Diffing
Automated pixel or feature-based comparison between an output page layout and the intended design asset to verify code correctness.
Arrow-to-Text-to-Action Workflow
A technique using visually annotated screenshots (with arrows, shapes, and minimal text) to define tasks or movement trajectories for automated agents or video generators.
Code Review Artifact Generator
An automated AI pipeline that scans recent commits and generates a comprehensive, formatted report detailing security alerts, topology changes, and executive summaries.
Context-Aware Automation (Script Kit)
Empowering automated workflow tools to analyze screenshots, clipboard data, and screen layouts to infer user intent and build a personal memory repository.
Information Architecture / Information UX
Designing documentation layouts with bolding, italics, underlines, and explicit sections to avoid dense walls of text and allow rapid scannability.
Commit Hooks / Push Gates
Automating automated validation checks (e.g., formatting, syntax checking, rule exclusions) on a developer's machine prior to allowing a code commit or remote repository push.
GPUI Framework
A high-performance, Rust-based user interface framework used to build native desktop application experiences.
Commit Hooks
Automated scripts running before a git commit is finalized to enforce quality checks like linters and formatters.
Cargo Crate Disk Space Management
The process of cleaning up or handling large disk consumption (e.g., 50 gigabytes) from Rust compilation artifacts.
Visual Prompting / Prompting with Images
An agent workflow leveraging an annotated image (e.g., with CleanShot arrows) paired with a clear, concise instruction sentence to guide an AI agent directly to a visual layout bug or asset.
GPUI Framework
A user interface framework mentioned by the presenter in relation to layout shifts under exploration in his tools.
Questions and answer status
--- What is the arrow pointing to?
Status: answered. The pi agent processes the screenshot and correctly returns: 'The arrow points to the highlighted "Build" text.'
a participant: 'When you say commit gate, what does that translate to?'
Status: answered. John explains that a commit gate translates directly to a Git commit hook. He highlights that developers can prompt an AI agent to build a customized commit hook containing tailored project requirements, such as blocklisting specific file types (like log files), checking code metrics, running formatters, or maintaining linter baselines.
Exercises
- Try: Capture with the CleanShot hotkey, draw ONE red arrow at the element in question, drag from the annotate overlay into your agent, and type one sentence of intent. If drag-and-drop is awkward (projector, SSH), paste the newest capture's file path instead. Basis: On-screen instructional sequence text located under 'Quick Start' header on the document page at 02:15:42.500.
Observed actions and statements
These timestamped observations retain troubleshooting detail and the limits of what was visible. Commands mentioned here are observations, not vetted installation instructions; use the topic’s prepared recipes for runnable examples.
-
02:15:50 — John discusses screen capture tools across operating systems and recommends CleanShot.
-
02:16:30 — John describes critical features for screen capture tools, such as binding global keyboard shortcuts and drawing on screen.
-
02:16:45 — a participant shares a link to Flameshot: 'https://flameshot.org/'.
-
02:17:04 — a participant shares a link to Shottr: 'https://shottr.cc/'.
-
02:17:10 — John searches for 'notion' on Google and opens notion.com. He uses CleanShot X to capture an area of the homepage and draws a red arrow pointing to the 'Build' pill badge.
-
02:17:37 — John explains that CleanShot keeps the image overlay available for drag-and-drop or saves it as the most recent clipboard entry.
-
02:18:05 — John types and runs 'take dev/notion-header' and then 'notion-header pi' in the terminal to spawn his workspace agent.
-
02:18:15 — John pastes the screenshot file path via Cmd+V into the pi terminal prompt, followed by the text: '--- What is the arrow pointing to?'.
-
02:18:50 — The terminal agent completes processing and outputs: 'The arrow points to the highlighted "Build" text.'.
-
02:19:15 — John inputs a multi-line code generation request to replicate the layout in HTML/CSS into the terminal agent: '--- I want to build a header like this in HTML and CSS where there's a pill like this build inset inside of the header text. Please build that out for me.'.
-
02:19:28 — The cmux workspace environment is active under the workspace label 'notion-header', displaying automated planning commands and text prompt inputs.
-
02:19:30 — John Lindquist explains that while vision models might not match fonts or sizes exactly, they are highly capable of locating elements indicated by user markings.
-
02:19:44 — The terminal executes repository mapping scripts and lists directory setup commands to track workspace artifacts.
-
02:20:37 — The CleanShot X annotation overlay is opened, and the presenter draws multiple colorful arrows and markings to demonstrate screenshot annotation flexibility.
-
02:21:37 — John Lindquist suggests prompting an agent to take a screenshot after generating HTML to perform an automated layout comparison diff.
-
02:23:16 — John Lindquist mentions grabbing any sort of screenshot tool.
-
02:23:21 — John notes that CleanShot has been solid for him for years and reliably does what it promises.
-
02:23:54 — John calls this a workflow you can teach to anyone on the team.
-
02:24:13 — John mentions that you can add text if you want to.
-
02:24:35 — John explains using annotated images as instructions for video generators in a storyboarding approach.
-
02:25:52 — John highlights the code review it did with a very loose prompt.
-
02:26:37 — John states that an AI configuration left in some OAuth files that stayed local and did not pass push gates.
-
02:27:04 — John Lindquist describes an AI feature like 'what's going on today'.
-
02:27:10 — John Lindquist explains that it is using the Codex account to build up its own AI functionality.
-
02:27:26 — John Lindquist expresses his ambition to make Script Kit gather context directly from the screen view.
-
02:27:32 — John Lindquist suggests actions like taking a screenshot of an app or summarizing the screen text.
-
02:27:58 — John Lindquist discusses the challenge of working with Rust and the GPUI framework regarding asset bundle sizes.
-
02:28:10 — John Lindquist reviews the past week of local commits and mentions that he has been experimenting rather than pushing changes.
-
02:28:28 — John Lindquist states his dislike for the table layout and recommends a template with better information architecture or information UX.
-
02:29:13 — John Lindquist characterizes massive data tables as information overload with excessive amounts of text.
-
02:29:28 — John Lindquist defines a commit gate as a Git commit hook.
-
02:30:22 — John Lindquist outlines constraints run by commit hooks, including line limits, specific design patterns, linters, benchmarks, and code formatters.
-
02:30:46 — John explains how to prompt an agent to set up standard commit hooks with linters and formatters.
-
02:31:16 — John discusses disk space management challenges when compiling Rust projects and Cargo builds.
Unresolved: The exact application layout type running at 02:26:28.550.
The view flashes briefly on screen for less than two seconds, showing a custom workspace tree with agent identifiers alongside a source editor block.
Discriminating evidence: Verify with future or past shards covering code execution panel structure.
Unresolved: The exact hosting deployment or resolution path of the changelog site interface.
Text is rendered standard-scale on screen and can match varying development server distributions.
Discriminating evidence: Cross-reference with project configurations inside neighboring Script Kit repositories.
Demonstration detail: 02:17:10
- Navigated to notion.com in Chrome browser.
- Took a screenshot layout and drew an analytical arrow target using CleanShot X.
- Opened a Ghostty terminal window session.
- Created and changed workspace context using 'take dev/notion-header'.
- Initialized the interactive prompt environment using 'notion-header pi'.
- Pasted the screenshot asset file path and submitted a query to verify arrow targeting.
- Submitted a detailed design prompt requesting corresponding HTML and CSS structure generation.
Tools observed: Chrome Browser, CleanShot X, Ghostty Terminal, pi prompt agent.
Command or prompt observations (not a verified recipe):
take dev/notion-header
notion-header pi
Demonstration detail: 02:19:28
- Automated workspace mapping scripts run inside the terminal environment, printing configuration details and generating layout artifacts.
Tools observed: cmux.
Command or prompt observations (not a verified recipe):
pwd && print '
-- files --
' && find . -maxdepth 2 -type f | sort | head -200
mkdir -p .artifacts
Demonstration detail: 02:20:37
- Draws a red line arrow targeting webpage headers, draws a red crossing mark, creates multi-colored secondary indicator arrows, and selects 'Don't Save' to clear modifications.
Tools observed: CleanShot X, Google Chrome.
Demonstration detail: 02:23:21
- Initiated screenshot frame frozen overlay with CleanShot X.
- Drew a red circle markup around the avatar elements.
- Selected the text tool option.
- Drew a red rounded rectangle over the avatars.
- Added a red direction arrow pointing to the right.
- Closed the screenshot overlay window and clicked 'Don't Save' on the confirmation modal.
Tools observed: CleanShot X.
Demonstration detail: 02:25:21
- Navigated to amber-willow-n6aj.here.now and scrolled through prompt documentation.
- Switched tab to liminal-voga-nyfw.here.now to view automated AI code review guide outputs.
- Briefly pulled up terminal view displaying script-kit-gpui workspace.
- Returned to the automated guide page and highlighted specific sections concerning OAuth security flags.
Tools observed: Google Chrome.
Demonstration detail: 02:27:04
- John prompts an overlay interface to highlight sample user context gathering strategies.
Tools observed: Script Kit Launcher.
Demonstration detail: 02:28:10
- John scrolls through generated documentation to assess text spacing, layout readability, and layout issues.
Tools observed: Web Browser.
Demonstration detail: 02:31:35
- Presenter switches focus to Google Chrome and cycles through open documentation tabs.
- Presenter navigates to the workshop index page on ember-bodhi-wcm.here.now and scrolls through sections.
Tools observed: Google Chrome.