Agent Markers debugging workflows: practical reference

Browser-debugging reference covering the agent-markers setup prompt, perspective-based exploratory QA, bounded event capture and marker schemas, browser/global APIs, privacy and cleanup, JSON and test-artifact exports, replay capsules, app-specific…

Agent Setup Prompt

Set up the agent-markers files and configuration for this browser project, then verify the setup before investigating any specific bug.

1. Inspect the existing agent-markers package and demo files first:
   - packages/client/src/index.ts
   - packages/client/src/collector.ts
   - packages/client/src/types.ts
   - packages/client/src/capture/console.ts
   - packages/client/src/capture/fetch.ts
   - packages/client/src/capture/errors.ts
   - packages/client/src/serialize.ts
   - apps/demo/src/main.ts
   - apps/demo/src/viewer.ts
   - scripts/smoke-demo.mjs
   - docs/WORKSHOP.md

2. If the package or demo is missing, scaffold the smallest matching structure. If files already exist, preserve the current API where possible and extend it only as needed.

3. Define or confirm the core schema:
   - DebugSession: one bounded run with sessionId, appName, schemaVersion, CapturedEvent[], and DebugMoment[]
   - CapturedEvent: id, sequence, timestamp, type, and metadata for console, fetch, error, manual, DOM, screenshot, or app events
   - DebugMoment: id, label, note, optional nextAction, timestamp, and nearbyEventIds that reference CapturedEvent ids
   - export JSON: a JSON-serializable packet that can be saved, attached to a test run, or passed to another agent

4. Implement or confirm createAgentMarkers in packages/client/src/index.ts. It should accept appName, exposeGlobal, and capture options for console, fetch, and errors. It should return mark(...), export(), and stop().

5. Implement console, fetch, error, and manual capture:
   - console capture records log/warn/error metadata without creating recursive logging loops
   - fetch capture records URL, method, status, timing, and failure metadata
   - error capture records thrown errors and unhandled rejections
   - manual capture records marker labels, notes, nextAction, and timing
   - all captured events should receive stable ids and sequence numbers

6. Add privacy-safe defaults:
   - capture metadata by default
   - do not export storage values, cookies, tokens, request bodies, response bodies, private DOM text, or screenshots unless deliberately enabled and reviewed
   - redact sensitive query params, headers, stack paths, and visible private text before saving export JSON
   - include redaction notes in the exported packet

7. Implement the collector in packages/client/src/collector.ts. Store events in order, enforce a maximum event count, and calculate nearbyEventIds for each DebugMoment using sequence or timestamp proximity.

8. Implement serialize.ts so export() always returns valid JSON without circular references. Optional DOM or screenshot fields should be marked unavailable when no hook is configured.

9. Wire createAgentMarkers into the demo app in apps/demo/src/main.ts with exposeGlobal: true for dev/test and capture enabled for console, fetch, and errors. Add a manual marker entrypoint such as a debug button or keyboard shortcut that calls markers.mark({ label, note, nextAction }).

10. Wire apps/demo/src/viewer.ts so the current session can be exported, displayed, copied, or downloaded as JSON.

11. Expose the dev/test global surface when configured:
   - window.__AGENT_MARKERS__.mark(...)
   - window.__AGENT_MARKERS__.export()
   - window.__AGENT_MARKERS__.stop()

12. Add test-runner export. In a Playwright-style test or equivalent, wait until the app is ready, call window.__AGENT_MARKERS__.mark(...), export the session with page.evaluate, and attach or save agent-markers.json. Assert markers.length > 0, a non-empty note exists, nearbyEventIds is present, and console/fetch/error/manual events appear when enabled.

13. Add bounded replay packet setup for one marker. The packet should include schemaVersion, sessionId, markerId, markerCutoffSeq, nearbyEventIds, filtered console/fetch/error/manual evidence, optional DOM/screenshot availability, included and excluded boundaries, maximum event count, redaction status, and the first verification command. Do not build a full recorder.

14. Update scripts/smoke-demo.mjs so it can launch or exercise the demo, create one marker, trigger at least one console event, fetch event, and error event where practical, export JSON, parse it, and confirm the packet is non-empty.

15. Verify with the available project commands:
   - npm test
   - npm run typecheck
   - npm run build
   - npm run smoke:demo

16. Confirm setup worked:
   - createAgentMarkers imports and initializes
   - mark() creates a DebugMoment with a specific note
   - nearbyEventIds is non-empty after nearby capture activity
   - exported events include console, fetch, error, and manual types when enabled
   - export JSON parses cleanly
   - window.__AGENT_MARKERS__ is available in dev/test only
   - stop() restores patched console, fetch, and error listeners
   - test-runner export saves or attaches agent-markers.json
   - privacy defaults exclude sensitive values

Report the files changed, setup choices, verification commands and results, sample export path, privacy boundaries, unavailable hooks, and any failed checks. If setup checks fail, stop and report the blocker. Do not debug a specific product bug until the setup and verification pass.

Big Idea

Start by sending cheap perspective-based agents through weird paths. Then use agent markers to capture the exact moment that should become a bounded debug packet.

Core pattern:

5 Monkeys QA -> mark -> capture -> explain -> review/export

The 5 Monkeys pattern finds strange states a happy-path check would miss. Agent markers turn one of those strange states into evidence another agent can use.

An agent marker is a human-intent anchor inside a debugging workflow.

It answers:

  • why this moment mattered
  • what evidence was nearby
  • what should be inspected next
  • what the next human or agent should not waste time guessing

Agent markers are not full replay systems. They are bounded debug packets.

Try This Now

Create one marker for a checkout UI bug: window.__AGENT_MARKERS__.mark({ label: "checkout disabled unexpectedly", note: "Expected submit to enable after terms and shipping were selected." }) Export the session and check that the marker has nearby evidence, not just a label.

Start With 5 Monkeys

Use 5 Monkeys before marker capture when the bug is vague, stateful, or hidden behind normal user behavior.

Assign one lens per pass:

Lens What It Tries
expert power user shortcuts, fast navigation, hidden affordances
brand-new first-time user confusion, missing affordances, unclear copy
inexperienced user accidental clicks, mistaken paths, recovery failures
impatient returning user rapid clicks, skipped steps, partial flows

Make each pass explore weird paths:

  • keyboard shortcut sequences
  • back/forward navigation mid-flow
  • repeated or rapid clicks on one control
  • toggling modes back and forth
  • starting a flow, then bailing out
  • refreshing or re-entering with partial state

Write each pass to a discardable QA note such as .notes/qa-01.md. Collate the findings, keep only useful bugs, and mark the one moment that best captures the state failure.

Then capture that moment with agent markers:

weird state found -> human marks the decisive moment -> export nearby evidence

Do not replace stable business-logic tests with 5 Monkeys. Use it to discover UI and state failures that tests and logs did not already isolate.

Files To Inspect

Agent markers project references:

packages/client/src/index.ts
packages/client/src/collector.ts
packages/client/src/types.ts
packages/client/src/capture/console.ts
packages/client/src/capture/fetch.ts
packages/client/src/capture/errors.ts
packages/client/src/serialize.ts
apps/demo/src/main.ts
apps/demo/src/viewer.ts
scripts/smoke-demo.mjs
docs/WORKSHOP.md

When To Reach For This

Use markers when logs are real but the important moment still needs human context.

Use Markers For Why
checkout bug mark the exact UI state where expected and actual diverge
auth/session refresh issue capture nearby fetch/error/console evidence around the failure
billing admin UI failure attach human expectation to the visible broken state
flaky browser test export a debug packet from the test runner
design/debug review pair screenshot/DOM hooks with a human explanation
long agent debugging session leave explicit breadcrumbs for the next pass

Skip markers when:

  • one failing unit test already isolates the bug
  • a stack trace points to one obvious line
  • no one can explain why the moment matters
  • the capture would expose sensitive data without redaction
  • the task needs full deterministic replay rather than a bounded debug packet

How This Used To Work

What you used to expect:

If enough logs are captured, the agent can infer the important moment.

What the workflow gives you now:

Logs show events. A marker ties a specific moment to human intent, nearby
evidence, and the next debugging question.

What you used to expect:

A screenshot or console dump is enough context.

What the workflow gives you now:

The useful packet combines the marker note, nearby console/fetch/error/manual
events, optional DOM or screenshot capture, privacy boundaries, and an export.

The Mental Model

DebugSession
-> CapturedEvent[]
-> DebugMoment[]
-> export JSON
-> agent prompt / test artifact / work unit
Concept Meaning
DebugSession one bounded debugging run
CapturedEvent console, fetch, error, manual, DOM, screenshot, or app event
DebugMoment the human marker: label, note, timestamp, nearby evidence
nearbyEventIds event ids around the marked moment
export JSON portable debug packet
stop() cleanup that restores patched browser APIs

Basic Browser Setup

Create markers in a browser app:

import { createAgentMarkers } from "@agent-markers/client";

const markers = createAgentMarkers({
  appName: "checkout-app",
  exposeGlobal: true,
  capture: {
    console: true,
    fetch: true,
    errors: true,
  },
});

Expose a manual marker action:

document.querySelector("#mark-debug")?.addEventListener("click", () => {
  markers.mark({
    label: "checkout payment method disappeared",
    note: "Expected saved payment selector to stay visible after refresh.",
  });
});

Export:

const session = markers.export();
console.log(JSON.stringify(session, null, 2));

Clean up patched APIs:

markers.stop();

Global Test Surface

In dev/test, expose the marker surface:

createAgentMarkers({
  appName: "checkout-app",
  exposeGlobal: true,
});

Then inspect from the browser console or a test runner:

window.__AGENT_MARKERS__.mark({
  label: "checkout failure",
  note: "Expected saved cards to render before submit.",
});

const session = window.__AGENT_MARKERS__.export();
session.markers.length;

Use the global for test export, not as the only application API.

Capture Types

Capture What It Adds Watch For
console logs, warnings, errors noisy output
fetch request URL, method, status, timing request/response bodies may be sensitive
errors thrown errors and unhandled rejections stack traces may include private paths
manual human-entered labels and notes must be specific
DOM hook optional host-provided DOM snapshot avoid private text
screenshot hook optional host-provided image capture unavailable in some environments
storage metadata keys/counts, not full values by default never dump secrets by accident

Privacy rule:

Capture metadata by default. Capture values only when deliberately enabled and safe.

Marker Shape

A useful marker has:

{
  "label": "checkout saved card missing",
  "note": "Expected saved payment methods to appear after login refresh; actual UI shows empty selector.",
  "nextAction": "Inspect auth refresh state, checkout payment query, and recent fetch events."
}

Weak marker:

{
  "label": "broken"
}

Better marker:

{
  "label": "saved payment selector empty after refresh",
  "note": "At this moment the user is logged in, the account menu shows the correct user, but checkout has no saved cards.",
  "nextAction": "Compare account payment fetch response with checkout payment state."
}

Export A Debug Packet

Export from browser:

const session = window.__AGENT_MARKERS__.export();
console.log(JSON.stringify(session, null, 2));

Save in a test:

const session = await page.evaluate(() => window.__AGENT_MARKERS__.export());
await testInfo.attach("agent-markers.json", {
  body: JSON.stringify(session, null, 2),
  contentType: "application/json",
});

Or write to disk in a local script:

const session = await page.evaluate(() => window.__AGENT_MARKERS__.export());
await fs.promises.writeFile(
  "artifacts/agent-markers/checkout-session.json",
  JSON.stringify(session, null, 2)
);

Verify The Project

Common checks:

npm test
npm run typecheck
npm run build
npm run smoke:demo

What to look for:

  • marker can be created
  • marker has a human explanation
  • nearby events are linked with nearbyEventIds
  • console/fetch/error/manual events appear when enabled
  • storage values are not dumped by default
  • export JSON is valid
  • stop() restores patched browser APIs
  • test runner can access window.__AGENT_MARKERS__

Turn A Marker Export Into An Agent Prompt

Prompt shape:

# Debug Prompt From Agent Marker

## Source Artifact

- marker export:
- schema version:
- marker id:
- marker label:

## Human Explanation

- expected:
- actual:
- why this moment matters:
- next action:

## Nearby Evidence

- console:
- fetch:
- error:
- manual:
- screenshot:
- DOM:

## Privacy Boundary

- storage values exported? no
- request bodies exported? no
- response bodies exported? no
- private text removed:

## Task

Find the smallest likely bug path and propose the first verification command.
Do not invent app state that is not in the marker packet.

Bounded Replay Capsule

Do not build a full recorder. Design the smallest packet that makes the marked moment inspectable.

# Bounded Replay Capsule

- schemaVersion:
- markerId:
- sessionId:
- markerCutoffSeq:
- nearbyEventIds:
- console:
- fetch:
- errors:
- manual:
- screenshot:
- dom:
- redaction:

## Boundaries

- included:
- excluded:
- maximum event count:
- privacy rules:

## How To Use It

- agent prompt:
- human review:
- verification command:

App-Specific Marker Flow

An app may wire a shortcut such as Ctrl+M to create durable marker evidence.

Conceptual flow:

Ctrl+M
-> create marker id
-> capture screenshot if available
-> open note/explanation UI
-> save human explanation
-> write durable app logs

Expected log strings:

dev_marker
dev_marker_explanation

Inspection shape:

rg -n "dev_marker|dev_marker_explanation" <app-log-path>/latest-session.jsonl

Keep this generic:

  • one marker id shared across marker and explanation
  • screenshot optional
  • explanation required
  • bounded nearby evidence
  • clear next action

Ask The Agent This

Add markers to a browser bug investigation:

Add agent markers to the checkout browser flow.

Capture console, fetch, errors, and manual markers. Expose
`window.__AGENT_MARKERS__` in dev/test. Add a marker for the moment saved
payment methods disappear after auth refresh. Export JSON and show how to
confirm nearbyEventIds, privacy defaults, and marker explanation.

Turn export into a debug prompt:

Use this agent-marker export as the bounded debug packet.

Focus on the marked checkout failure. Summarize the human explanation, nearby
console/fetch/error/manual evidence, privacy boundaries, and the first
verification command. Do not infer app state that is not in the packet.

Write a test export:

Create a Playwright-style test path that triggers the checkout bug, calls
`window.__AGENT_MARKERS__.mark(...)`, exports the session JSON, and attaches it
as a test artifact. Include the checks that prove marker count, nearbyEventIds,
and event types are present.

What To Confirm / How To Know It Worked

Goal Confirmation
marker client installed app imports createAgentMarkers
marker created export has markers.length > 0
explanation present marker has a specific note
nearby evidence linked marker has non-empty nearbyEventIds
console captured exported events include console type
fetch captured exported events include fetch type
errors captured exported events include error type
privacy default safe storage values are absent unless deliberately enabled
global test surface works window.__AGENT_MARKERS__.export() returns session
export saved JSON artifact exists and is non-empty
cleanup handled stop() called when patching should end
project verifies npm test, npm run typecheck, npm run build, npm run smoke:demo

If It Goes Wrong

Marker Has No Note

Add expected/actual context.

What should have happened:
What happened:
Why this moment matters:
Next action:

Export Has Too Many Raw Logs

Focus on one marker id and its nearby events. Increase event limits deliberately, not by default.

nearbyEventIds Is Empty

Mark closer to the failure or adjust the nearby event window.

Sensitive Storage Values Were Captured

Redact and rebuild the export. Capture storage metadata, not values, unless safe and intentional.

Console/Fetch Remains Patched

Call:

markers.stop();

Test Runner Cannot Export

Confirm exposeGlobal: true in dev/test and wait for app readiness before reading window.__AGENT_MARKERS__.

Screenshot Or DOM Hook Is Missing

Mark it unavailable. Do not pretend a screenshot or DOM snapshot exists.

References And Resources

  • Agent marker client entry: packages/client/src/index.ts
  • Collector source: packages/client/src/collector.ts
  • Types: packages/client/src/types.ts
  • Demo app: apps/demo/src/main.ts
  • Viewer/export UI: apps/demo/src/viewer.ts
  • Smoke script: scripts/smoke-demo.mjs
  • Playwright docs: https://playwright.dev/docs/test-assertions
  • Cypress docs: https://docs.cypress.io/

Source references