Original source pages and recordings require authorized access. The full article text here is public.
Web developers already run local dev servers, browser checks, test scripts, and agent panes; this technique turns those runtime artifacts into proof instead of vibes. It matters because UI bugs, auth regressions, and hook failures are often only obvious in logs, browser state, or terminal output, not in the agent's final message.
Copy-paste workflow
When to use: When you need to KNOW an agent really verified its work, not just trust a happy completion message. This captures the exact terminal failure, feeds it back to harden the tool, and gates fix/verify behind structured logs a supervisor compares.
Copy-paste prompt
You are my coding agent. Don't trust completion messages — prove the work with logs. Run this self-improvement + log-gated verification loop on <TOOL/WORKFLOW>.
1. Capture the EXACT terminal failure state as clean text (don't summarize it from memory). In CMUX, use the terminal-text extraction to grab the real output:
cmux read-screen
Use that real text as context.
2. Use this prompt shape against the failure — state the goal, paste the failure after a colon, then ask for a root-cause fix:
"Goal: <GOAL>. Here is the exact failure: <PASTE TERMINAL TEXT>. Explain WHY it failed in this scenario, then PATCH the hook/workflow so this case is covered next time."
3. Make verification CONCRETE and deterministic. The fix must write structured logs / JSON artifacts / timestamps to disk on each run. Then a supervisor agent runs the scenarios, waits, and COMPARES expected-vs-actual fields in those log files. That comparison is the gate — only pass when the fields match.
Guardrails: keep worker agents narrow and tool-limited; keep the orchestrator context-rich. REDACT secrets from logs.
How it works
- Capture the exact terminal failure as clean text (via CMUX
cmux read-screen) rather than a memory summary, and use it as real context. - The prompt shape — goal, then the failure after a colon, then "explain why it failed and patch the workflow" — drives a true root-cause fix, not a band-aid.
- Verification is made concrete: the workflow writes structured logs / JSON artifacts / timestamps, and a supervisor agent runs scenarios and compares expected-vs-actual fields. That field comparison is the deterministic gate.
- Keep worker agents narrow and tool-limited; keep the orchestrator context-rich.
- Redact secrets from logs.
- Reference: codex-daemons, CMUX.
Reusable skill
Install as ~/.agents/skills/log-gated-verification/SKILL.md, then restart Codex.
---
name: log-gated-verification
description: Trigger when you must prove an agent actually verified work rather than trusting a completion message — write structured logs/artifacts and have a supervisor compare expected-vs-actual fields (a deterministic gate). Do NOT trust self-reported success.
---
# Log-gated verification
Prove work with logs instead of trusting completion messages. Capture the exact failure, patch the workflow to cover it, and gate fix/verify behind structured logs a supervisor compares field-by-field.
## Steps
1. Capture the EXACT terminal failure state as clean text (not a memory summary). In CMUX, extract the real terminal output:
```bash
cmux read-screen
```
Use that real text as context.
2. Use this prompt shape against the failure — goal, then the failure after a colon, then the fix request:
```
Goal: <goal>. Here is the exact failure: <paste terminal text>.
Explain WHY it failed in this scenario, then PATCH the hook/workflow so this case is covered next time.
```
3. Make verification CONCRETE and deterministic: the workflow must write structured JSON logs / timestamps / payload snapshots to disk on each run.
4. A supervisor agent runs the scenarios, waits, and COMPARES expected-vs-actual fields in those log files. That field comparison is THE GATE — pass only when the fields match.
## Guardrails
- Do NOT trust self-reported success; the log comparison is the source of truth.
- Keep worker agents narrow and tool-limited; keep the orchestrator context-rich.
- REDACT secrets from logs.
Scenarios
Auth redirect loop with browser evidence
Role: As a full-stack Next.js developer maintaining an authenticated dashboard
A local Next.js app sometimes redirects logged-in users from /dashboard back to /login after a route middleware or session-cookie change. The agent claims the fix works, but the team needs proof from the running browser and structured logs.
As a full-stack developer, I want the agent to reproduce the auth redirect failure, capture exact runtime evidence, patch the workflow, and gate success on log comparison, so that I do not merge an auth fix based on a cheerful completion message.
Agent brief
You are my coding agent. Run a self-improving log-gated fix loop for the local auth redirect bug. First inspect the current code and existing verification scripts; do not rely on memory. Start the dev server using the repo's documented command from package.json. Reproduce the flow in a browser: visit /login, sign in with the local/dev credentials or seeded test user documented in the repo, navigate to /dashboard, refresh, and confirm whether the browser stays on /dashboard. Capture exact evidence: terminal output with `cmux read-screen`, browser console/network output if available, and a JSON log artifact at `.notes/log-gated/auth-redirect/run-<timestamp>.json`. The log must include goal, routeVisited, finalUrl, redirectCount, authCookiePresent as true/false when observable, serverStatusCodes, consoleErrors, terminalFailureText when present, and pass boolean. If the first run fails, use this repair prompt shape internally: `Goal: authenticated users should remain on /dashboard after login and refresh. Here is the exact failure: <paste terminal/browser/log evidence>. Explain WHY it failed in this scenario, then PATCH the workflow so this case is covered next time.` After patching, rerun the same browser flow and write a second JSON artifact. A supervisor pass must compare expected-vs-actual fields and only pass when finalUrl is /dashboard, redirectCount is 0 after the authenticated refresh, pass is true, and no new console/server errors appear. Finish with the files changed, commands run, log artifact paths, and a terse review note.
Implementation references
cmux read-screen
cmux markdown open .notes/log-gated/auth-redirect/tracker.md --direction right
package.json scripts for dev/test commands
.notes/log-gated/auth-redirect/*.json
Browser verification
Use a CMUX browser pane or agent-browser-style workflow to take an accessibility snapshot of /login, fill the login form, click submit, wait for navigation, evaluate window.location.pathname, refresh, evaluate it again, and capture console/network errors. Browser proof is required here because the bug is about real navigation, cookies, and route behavior.
Definition of done
- At least two structured JSON run logs exist: one reproducing the original failure when possible and one post-fix verification run.
- The supervisor comparison explicitly checks finalUrl, redirectCount, status codes, consoleErrors, and pass fields instead of trusting the worker's text.
- The final browser state after login and refresh is /dashboard with no unexpected redirect to /login.
- The final response lists exact commands run, files changed, and log paths under
.notes/log-gated/auth-redirect/.
Failure modes and mitigations
- Common mistake: the agent only runs unit tests and declares success. Mitigation: require a browser flow plus JSON log fields proving finalUrl and redirectCount.
- Common mistake: logs contain tokens, cookies, or full auth headers. Mitigation: redact secrets and store only booleans, route names, status codes, and sanitized error strings.
- Common mistake: the agent tests a stale dev server. Mitigation: restart or verify the dev server command, capture terminal output with
cmux read-screen, and include the server start timestamp in the log.
Checkout UI regression behind deterministic artifacts
Role: As a frontend product engineer responsible for a checkout or upgrade flow
A pricing or checkout screen was refactored and now one browser path intermittently disables the primary CTA, misprices a plan, or shows the wrong confirmation state. The workflow needs browser validation plus structured artifacts a supervisor can compare.
As a frontend product engineer, I want an agent to verify the checkout path with browser actions and structured scenario logs, so that regressions are caught by observable artifacts instead of a subjective agent summary.
Agent brief
You are my coding agent. Build a log-gated verification loop for the checkout/pricing flow. Inspect the source files for the pricing/checkout route, the relevant components, and existing tests. Start the local app using the repo's package scripts. Create `.notes/log-gated/checkout/criteria.md` with the expected scenario fields: selectedPlan, displayedPrice, ctaEnabled, confirmationVisible, finalUrl, consoleErrors, networkFailures, and pass. Then run three browser scenarios: default plan selection, switching plans twice, and back/refresh during checkout. For each scenario, write `.notes/log-gated/checkout/<scenario-slug>.json` with timestamp, exact steps, expected fields, actual fields, terminal evidence from `cmux read-screen` when a failure occurs, and pass boolean. If any scenario fails, use the exact failure text and artifact contents to patch the code or workflow, then rerun only the failing scenario plus one happy-path scenario. Add or update a verifier script only if the repo already has a natural place for local scripts; otherwise have the supervisor agent compare the JSON files directly. The gate passes only when all expected-vs-actual fields match and the browser proof agrees with the logs.
Implementation references
cmux read-screen
cmux markdown open .notes/log-gated/checkout/tracker.md --direction right
.notes/log-gated/checkout/criteria.md
.notes/log-gated/checkout/*.json
package.json scripts for dev/test commands
Browser verification
Use Playwright, a CMUX browser pane, or an agent-browser-style accessibility snapshot/click/evaluate sequence: open the pricing route, snapshot the buttons and plan labels, click each plan option, evaluate visible price and CTA disabled state, click the CTA, then evaluate the final confirmation URL/state. This is browser-worthy because visual state, disabled controls, and navigation are the product behavior.
Definition of done
- Three scenario JSON logs exist with expected fields and actual fields.
- A supervisor comparison marks pass only when selectedPlan, displayedPrice, ctaEnabled, confirmationVisible, finalUrl, consoleErrors, and networkFailures match the criteria.
- The agent provides browser evidence for each scenario, either through Playwright output, accessibility snapshots, screenshots, console logs, or explicit evaluate results.
- Any fix is tied back to the exact failing artifact and rerun evidence.
Failure modes and mitigations
- Common mistake: the agent overfits to one happy path. Mitigation: require default selection, repeated plan switching, and back/refresh behavior.
- Common mistake: the agent writes brittle permanent UI tests for a screen still changing daily. Mitigation: keep these as
.notes/log-gated/checkout/artifacts unless the repo already has a stable test home. - Common mistake: the browser log and JSON log disagree. Mitigation: make the supervisor compare both and fail the gate until the artifact mismatch is resolved.
Codex hook or local CLI failure hardening
Role: As a developer maintaining a repo-local Codex hook, plugin, or CLI helper
A hook or local CLI command fails under a specific edge case, such as malformed input, missing config, unexpected event JSON, or a command usage error. The agent must capture the exact terminal failure, patch the hook/workflow, and prove the case is covered next time.
As a tooling-focused developer, I want failures from hooks or CLI helpers to become regression artifacts, so that each failure hardens the workflow instead of becoming another one-off terminal mystery.
Agent brief
You are my tooling agent. Run the self-improvement error loop on the failing hook/CLI workflow. First reproduce the failing command or hook scenario exactly. Capture the terminal state with `cmux read-screen` and save the raw text to `.notes/log-gated/tooling/failures/<timestamp>.txt`. Then apply this prompt shape to your own diagnosis: `Goal: <describe the hook or CLI contract>. Here is the exact failure: <paste captured terminal text>. Explain WHY it failed in this scenario, then PATCH the hook/workflow so this case is covered next time.` Patch the smallest relevant hook/workflow code. Add structured run artifacts under `.notes/log-gated/tooling/runs/` containing scenarioName, inputShape, commandRun, expectedOutcome, actualOutcome, errorText, timestamp, and pass. Run at least the original failing case and one nearby non-failing case. A supervisor pass must inspect the run artifacts and compare expectedOutcome vs actualOutcome. Do not claim success unless the artifacts show the original failure case now passes and the nearby case still passes.
Implementation references
cmux read-screen
.notes/log-gated/tooling/failures/*.txt
.notes/log-gated/tooling/runs/*.json
Browser verification
Browser proof is not the primary proof unless the hook controls browser-visible behavior. For a terminal hook or CLI helper, the right proof is exact terminal capture plus structured run logs. If the hook opens a CMUX browser dashboard or affects a web UI, add one browser snapshot/evaluate step to confirm the visible state matches the log.
Definition of done
- The raw failure text is saved exactly, not paraphrased.
- The patch is tied to a stated root cause from the captured failure.
- At least two structured run logs exist: original failing scenario and nearby control scenario.
- A supervisor comparison reads the artifacts and marks the gate pass only when expectedOutcome and actualOutcome match.
Failure modes and mitigations
- Common mistake: the agent patches the symptom without explaining the failure condition. Mitigation: require a root-cause note derived from the captured terminal text.
- Common mistake: the agent creates an unbounded self-improvement loop. Mitigation: run a fixed set of scenarios and stop after the gate comparison.
- Common mistake: the worker agent gets too much project context and wanders. Mitigation: keep the worker scoped to the hook/CLI files, logs, and the contract for this workflow.
Implementation notes
- Keep the durable artifact small and boring: raw terminal capture, scenario criteria, JSON run logs, and a supervisor comparison.
- Use
.notes/log-gated/<scenario>/for local artifacts unless the project already has a committed test-artifact convention. - For web app scenarios, require browser proof when the behavior depends on cookies, navigation, visible UI state, console errors, network calls, or disabled controls.
- For hook or CLI scenarios, terminal capture and structured logs are the primary proof; browser automation is only useful if the workflow changes browser-visible state.
- The worker agent should patch narrowly; the orchestrator or supervisor should compare artifacts and decide pass/fail.
- Do not trust the worker's final answer as the gate. The gate is expected-vs-actual comparison over files, logs, terminal output, browser output, or other inspectable artifacts.
- Redact secrets from logs. Store booleans, shapes, route names, status codes, sanitized errors, and timestamps rather than raw tokens, cookies, or credentials.
- Grounding note: content is derived from the uploaded scenario-log-gated bundle. :contentReference[oaicite:0]{index=0}
Boundaries and caveats
- Do not introduce unrelated tools that were only mentioned elsewhere, such as broad NotebookLM, n8n, ScreenPipe, or external CI platform workflows.
- Do not claim deterministic proof from an agent's natural-language completion message.
- Do not overbuild a permanent test suite for fast-moving UI experiments unless the repo already has stable acceptance tests for that flow.
- Do not create unbounded self-improvement loops; run named scenarios, compare artifacts, and stop.
- Do not log secrets, auth headers, private customer data, or raw cookies just because logs are useful.
Agent reference map
Validated lesson 08 references
Use as the ground-truth teaching source for terminal text extraction, self-improvement loops, log-based gates, natural-language boundaries, and context pruning.
Existing log-gated prompt
Use as the copy-paste source pattern for capturing exact failure text, asking for root-cause hardening, and requiring structured log artifacts.
Existing Codex skill
Use to keep scenario language aligned with the existing toolkit skill and avoid inventing untaught verification machinery.
CMUX terminal text capture
Use to grab exact terminal output as machine-readable context before diagnosis or patching.
cmux read-screen
CMUX live tracker pane
Use when the agent should keep a visible, auto-refreshing progress view next to the running workflow.
cmux markdown open .notes/log-gated/<scenario>/tracker.md --direction right
Lesson 08 validation notes
Use to stay inside the validated taught boundary: logs/files/API calls/object shapes/timestamps/session outputs, not invented broad verifier infrastructure.
Interactive lesson checklist
Use for concrete practice steps and success criteria around failure capture, log comparison gates, and worker boundary contracts.