---
schema_version: 1
revision: "22ce5d34dbcb5a78"
id: "factory-image-prompts-discussion"
title: "Images as precise task evidence — discussion and evidence"
---

# Images as precise task evidence — discussion and evidence

Detailed companion accounts of concepts, questions, demonstrations, exercises, and unresolved observations.

These are the companion’s accounts of the recorded discussion. Model capabilities, subscription rules, prices, version-specific flags, and predictions are contextual claims, not current guidance. An observed action is not proof that the attempted result succeeded. Distinct answers and uncertainties remain separate.

## Concepts and distinctions

### Prompting with Images

Using annotated visual context (such as screenshots with hand-drawn arrows) to precisely identify specific components or UI sections when prompting vision-capable large language models or software agents.

### Screen Annotation Workflow

A design/development shortcut involving a localized area capture tool that supports instant vector overlays (arrows, rectangles), streamlining the communication of UI components to an AI model with minimal steps.

### Visual Prompting / Multimodal Input

Supplying an LLM or autonomous agent with annotated screenshots containing simple drawings like arrows to establish spatial or element context instead of relying entirely on raw string selectors.

### Before/After Conversation Artifacts

Retaining original and updated web element screenshots inside chat history logs to allow easy visual diffs or visual reference verification by both the user and the agent.

### Image Diffing

Automated pixel or feature-based comparison between an output page layout and the intended design asset to verify code correctness.

### Arrow-to-Text-to-Action Workflow

A technique using visually annotated screenshots (with arrows, shapes, and minimal text) to define tasks or movement trajectories for automated agents or video generators.

### Code Review Artifact Generator

An automated AI pipeline that scans recent commits and generates a comprehensive, formatted report detailing security alerts, topology changes, and executive summaries.

### Context-Aware Automation (Script Kit)

Empowering automated workflow tools to analyze screenshots, clipboard data, and screen layouts to infer user intent and build a personal memory repository.

### Information Architecture / Information UX

Designing documentation layouts with bolding, italics, underlines, and explicit sections to avoid dense walls of text and allow rapid scannability.

### Commit Hooks / Push Gates

Automating automated validation checks (e.g., formatting, syntax checking, rule exclusions) on a developer's machine prior to allowing a code commit or remote repository push.

### GPUI Framework

A high-performance, Rust-based user interface framework used to build native desktop application experiences.

### Commit Hooks

Automated scripts running before a git commit is finalized to enforce quality checks like linters and formatters.

### Cargo Crate Disk Space Management

The process of cleaning up or handling large disk consumption (e.g., 50 gigabytes) from Rust compilation artifacts.

### Visual Prompting / Prompting with Images

An agent workflow leveraging an annotated image (e.g., with CleanShot arrows) paired with a clear, concise instruction sentence to guide an AI agent directly to a visual layout bug or asset.

### GPUI Framework

A user interface framework mentioned by the presenter in relation to layout shifts under exploration in his tools.

## Questions and answer status

### --- What is the arrow pointing to?

**Status: answered.** The pi agent processes the screenshot and correctly returns: 'The arrow points to the highlighted "Build" text.'

### a participant: 'When you say commit gate, what does that translate to?'

**Status: answered.** John explains that a commit gate translates directly to a Git commit hook. He highlights that developers can prompt an AI agent to build a customized commit hook containing tailored project requirements, such as blocklisting specific file types (like log files), checking code metrics, running formatters, or maintaining linter baselines.

## Exercises

- **Try:** Capture with the CleanShot hotkey, draw ONE red arrow at the element in question, drag from the annotate overlay into your agent, and type one sentence of intent. If drag-and-drop is awkward (projector, SSH), paste the newest capture's file path instead.
  **Basis:** On-screen instructional sequence text located under 'Quick Start' header on the document page at 02:15:42.500.

## Observed actions and statements

These timestamped observations retain troubleshooting detail and the limits of what was visible. Commands mentioned here are observations, not vetted installation instructions; use the topic’s prepared recipes for runnable examples.

- [02:15:50](https://proud-anchor-53zn.here.now/watch.html?t=8150#recording-player) — John discusses screen capture tools across operating systems and recommends CleanShot.

- [02:16:30](https://proud-anchor-53zn.here.now/watch.html?t=8190#recording-player) — John describes critical features for screen capture tools, such as binding global keyboard shortcuts and drawing on screen.

- [02:16:45](https://proud-anchor-53zn.here.now/watch.html?t=8205#recording-player) — a participant shares a link to Flameshot: 'https://flameshot.org/'.

- [02:17:04](https://proud-anchor-53zn.here.now/watch.html?t=8224#recording-player) — a participant shares a link to Shottr: 'https://shottr.cc/'.

- [02:17:10](https://proud-anchor-53zn.here.now/watch.html?t=8230#recording-player) — John searches for 'notion' on Google and opens notion.com. He uses CleanShot X to capture an area of the homepage and draws a red arrow pointing to the 'Build' pill badge.

- [02:17:37](https://proud-anchor-53zn.here.now/watch.html?t=8257#recording-player) — John explains that CleanShot keeps the image overlay available for drag-and-drop or saves it as the most recent clipboard entry.

- [02:18:05](https://proud-anchor-53zn.here.now/watch.html?t=8285#recording-player) — John types and runs 'take dev/notion-header' and then 'notion-header pi' in the terminal to spawn his workspace agent.

- [02:18:15](https://proud-anchor-53zn.here.now/watch.html?t=8295#recording-player) — John pastes the screenshot file path via Cmd+V into the pi terminal prompt, followed by the text: '--- What is the arrow pointing to?'.

- [02:18:50](https://proud-anchor-53zn.here.now/watch.html?t=8330#recording-player) — The terminal agent completes processing and outputs: 'The arrow points to the highlighted "Build" text.'.

- [02:19:15](https://proud-anchor-53zn.here.now/watch.html?t=8355#recording-player) — John inputs a multi-line code generation request to replicate the layout in HTML/CSS into the terminal agent: '--- I want to build a header like this in HTML and CSS where there's a pill like this build inset inside of the header text. Please build that out for me.'.

- [02:19:28](https://proud-anchor-53zn.here.now/watch.html?t=8368#recording-player) — The cmux workspace environment is active under the workspace label 'notion-header', displaying automated planning commands and text prompt inputs.

- [02:19:30](https://proud-anchor-53zn.here.now/watch.html?t=8370#recording-player) — John Lindquist explains that while vision models might not match fonts or sizes exactly, they are highly capable of locating elements indicated by user markings.

- [02:19:44](https://proud-anchor-53zn.here.now/watch.html?t=8384#recording-player) — The terminal executes repository mapping scripts and lists directory setup commands to track workspace artifacts.

- [02:20:37](https://proud-anchor-53zn.here.now/watch.html?t=8437#recording-player) — The CleanShot X annotation overlay is opened, and the presenter draws multiple colorful arrows and markings to demonstrate screenshot annotation flexibility.

- [02:21:37](https://proud-anchor-53zn.here.now/watch.html?t=8497#recording-player) — John Lindquist suggests prompting an agent to take a screenshot after generating HTML to perform an automated layout comparison diff.

- [02:23:16](https://proud-anchor-53zn.here.now/watch.html?t=8596#recording-player) — John Lindquist mentions grabbing any sort of screenshot tool.

- [02:23:21](https://proud-anchor-53zn.here.now/watch.html?t=8601#recording-player) — John notes that CleanShot has been solid for him for years and reliably does what it promises.

- [02:23:54](https://proud-anchor-53zn.here.now/watch.html?t=8634#recording-player) — John calls this a workflow you can teach to anyone on the team.

- [02:24:13](https://proud-anchor-53zn.here.now/watch.html?t=8653#recording-player) — John mentions that you can add text if you want to.

- [02:24:35](https://proud-anchor-53zn.here.now/watch.html?t=8675#recording-player) — John explains using annotated images as instructions for video generators in a storyboarding approach.

- [02:25:52](https://proud-anchor-53zn.here.now/watch.html?t=8752#recording-player) — John highlights the code review it did with a very loose prompt.

- [02:26:37](https://proud-anchor-53zn.here.now/watch.html?t=8797#recording-player) — John states that an AI configuration left in some OAuth files that stayed local and did not pass push gates.

- [02:27:04](https://proud-anchor-53zn.here.now/watch.html?t=8824#recording-player) — John Lindquist describes an AI feature like 'what's going on today'.

- [02:27:10](https://proud-anchor-53zn.here.now/watch.html?t=8830#recording-player) — John Lindquist explains that it is using the Codex account to build up its own AI functionality.

- [02:27:26](https://proud-anchor-53zn.here.now/watch.html?t=8846#recording-player) — John Lindquist expresses his ambition to make Script Kit gather context directly from the screen view.

- [02:27:32](https://proud-anchor-53zn.here.now/watch.html?t=8852#recording-player) — John Lindquist suggests actions like taking a screenshot of an app or summarizing the screen text.

- [02:27:58](https://proud-anchor-53zn.here.now/watch.html?t=8878#recording-player) — John Lindquist discusses the challenge of working with Rust and the GPUI framework regarding asset bundle sizes.

- [02:28:10](https://proud-anchor-53zn.here.now/watch.html?t=8890#recording-player) — John Lindquist reviews the past week of local commits and mentions that he has been experimenting rather than pushing changes.

- [02:28:28](https://proud-anchor-53zn.here.now/watch.html?t=8908#recording-player) — John Lindquist states his dislike for the table layout and recommends a template with better information architecture or information UX.

- [02:29:13](https://proud-anchor-53zn.here.now/watch.html?t=8953#recording-player) — John Lindquist characterizes massive data tables as information overload with excessive amounts of text.

- [02:29:28](https://proud-anchor-53zn.here.now/watch.html?t=8968#recording-player) — John Lindquist defines a commit gate as a Git commit hook.

- [02:30:22](https://proud-anchor-53zn.here.now/watch.html?t=9022#recording-player) — John Lindquist outlines constraints run by commit hooks, including line limits, specific design patterns, linters, benchmarks, and code formatters.

- [02:30:46](https://proud-anchor-53zn.here.now/watch.html?t=9046#recording-player) — John explains how to prompt an agent to set up standard commit hooks with linters and formatters.

- [02:31:16](https://proud-anchor-53zn.here.now/watch.html?t=9076#recording-player) — John discusses disk space management challenges when compiling Rust projects and Cargo builds.

### Unresolved: The exact application layout type running at 02:26:28.550.

The view flashes briefly on screen for less than two seconds, showing a custom workspace tree with agent identifiers alongside a source editor block.

**Discriminating evidence:** Verify with future or past shards covering code execution panel structure.

### Unresolved: The exact hosting deployment or resolution path of the changelog site interface.

Text is rendered standard-scale on screen and can match varying development server distributions.

**Discriminating evidence:** Cross-reference with project configurations inside neighboring Script Kit repositories.

### Demonstration detail: 02:17:10

- Navigated to notion.com in Chrome browser.
- Took a screenshot layout and drew an analytical arrow target using CleanShot X.
- Opened a Ghostty terminal window session.
- Created and changed workspace context using 'take dev/notion-header'.
- Initialized the interactive prompt environment using 'notion-header pi'.
- Pasted the screenshot asset file path and submitted a query to verify arrow targeting.
- Submitted a detailed design prompt requesting corresponding HTML and CSS structure generation.

**Tools observed:** Chrome Browser, CleanShot X, Ghostty Terminal, pi prompt agent.

**Command or prompt observations (not a verified recipe):**

```text
take dev/notion-header
notion-header pi
```

### Demonstration detail: 02:19:28

- Automated workspace mapping scripts run inside the terminal environment, printing configuration details and generating layout artifacts.

**Tools observed:** cmux.

**Command or prompt observations (not a verified recipe):**

```text
pwd && print '
-- files --
' && find . -maxdepth 2 -type f | sort | head -200
mkdir -p .artifacts
```

### Demonstration detail: 02:20:37

- Draws a red line arrow targeting webpage headers, draws a red crossing mark, creates multi-colored secondary indicator arrows, and selects 'Don't Save' to clear modifications.

**Tools observed:** CleanShot X, Google Chrome.

### Demonstration detail: 02:23:21

- Initiated screenshot frame frozen overlay with CleanShot X.
- Drew a red circle markup around the avatar elements.
- Selected the text tool option.
- Drew a red rounded rectangle over the avatars.
- Added a red direction arrow pointing to the right.
- Closed the screenshot overlay window and clicked 'Don't Save' on the confirmation modal.

**Tools observed:** CleanShot X.

### Demonstration detail: 02:25:21

- Navigated to amber-willow-n6aj.here.now and scrolled through prompt documentation.
- Switched tab to liminal-voga-nyfw.here.now to view automated AI code review guide outputs.
- Briefly pulled up terminal view displaying script-kit-gpui workspace.
- Returned to the automated guide page and highlighted specific sections concerning OAuth security flags.

**Tools observed:** Google Chrome.

### Demonstration detail: 02:27:04

- John prompts an overlay interface to highlight sample user context gathering strategies.

**Tools observed:** Script Kit Launcher.

### Demonstration detail: 02:28:10

- John scrolls through generated documentation to assess text spacing, layout readability, and layout issues.

**Tools observed:** Web Browser.

### Demonstration detail: 02:31:35

- Presenter switches focus to Google Chrome and cycles through open documentation tabs.
- Presenter navigates to the workshop index page on ember-bodhi-wcm.here.now and scrolls through sections.

**Tools observed:** Google Chrome.

## Original references

- [Published evidence reference](https://proud-anchor-53zn.here.now/resource-manifest.json)
- [Recorded discussion](https://proud-anchor-53zn.here.now/watch.html)
