Controlled variation workflows: practice drills

Fourteen exercises covering variation suitability, single-variable matrices, run artifacts, saved prompt seeds, repository checkpoints, candidate isolation and overrides, uniform verification, comparison, selection, convergence, failure recovery, and final…

These exercises practice turning one deliberate prompt into controlled, verified candidates.

Use a real software project when possible. Keep each exercise small. The goal is not to generate the most candidates; the goal is to compare useful options without losing control.

Exercise 1: Decide Whether Variations Are Appropriate

For each task, choose use variations or skip variations.

Task Use / Skip Why
Add saved payment methods to checkout, with several plausible UI entry points
Fix a one-character typo in a route name
Stop duplicate billing webhook processing, with multiple plausible idempotency strategies
Apply a strict framework migration with exact steps
Compare auth/session architecture before refresh-token rotation
Implement an exact requested API shape
Choose the right test boundary for a regression
Repo is dirty and there is no saved state
No verification command exists for the candidates

Write:

# Variation Decision

- task:
- decision:
- why:
- verification available:
- state safe for fan-out:

Exercise 2: Find The Variable

Pick one request and identify the single variable under test.

Request Possible Variable
Checkout saved payment methods could be integrated several ways. UI integration strategy
Billing webhooks sometimes process the same invoice twice. idempotency strategy
Auth/session code is becoming hard to reason about. architecture boundary
This regression needs better coverage. test boundary
The agent keeps producing broad implementation plans. prompt framing
The docs are technically correct but hard to use. documentation organization

Fill out:

# Variable Notes

- request:
- chosen variable:
- why this variable matters:
- invariants:
  -
  -
  -
- out of scope:
  -
  -
  -
- candidate count:
- review question:

Rule:

If you cannot finish "we are comparing candidates that differ only by ___," stop and narrow.

Exercise 3: Write A Variation Matrix

Use one of these scenarios:

checkout saved payment methods
billing webhook duplicate-processing
auth/session architecture
test strategy for a regression

Template:

# Variation Matrix

## Goal

## Invariants

- same repo state:
- same base prompt:
- same target feature or bug:
- same constraints:
- same verification command:
- same review criteria:

## Variable Under Test

- variable:
- why this matters:
- what is out of scope:

## Candidate Count

- count:
- ids:

## Candidates

| ID | Variable setting | Output location | Verification command | Notes |
| --- | --- | --- | --- | --- |
| v1 | | | | |
| v2 | | | | |
| v3 | | | | |

## Review Criteria

1.
2.
3.
4.
5.

Check it:

rg -n "Goal|Invariants|Variable Under Test|Candidate Count|Review Criteria" matrix.md

Exercise 4: Create The Run Folder

Create a run root under artifacts/variations.

RUN_ID="exercise-$(date +%Y%m%d-%H%M%S)"
ROOT="artifacts/variations/$RUN_ID"
mkdir -p "$ROOT"/{prompts,launch,logs,review,decisions}
mkdir -p "$ROOT"/candidates/{v1,v2,v3}
find "$ROOT" -maxdepth 3 -type d | sort

Fill out:

# Run Folder Notes

- run id:
- root:
- prompt folder:
- launch folder:
- candidate folders:
- logs folder:
- review folder:
- decisions folder:

Exercise 5: Save The Section 5 Seed

Use Prompt Builder to save the base prompt and launch JSON.

Checkout example:

PROMPT='Generate three controlled implementations for saved payment methods in checkout. Keep the database contract and provider API fixed. Vary only the UI integration strategy. Each option must include files to edit, tests to run, risks, and rollback notes.'

p --print-prompt "$PROMPT" \
  > "$ROOT/prompts/base.prompt.md"

p --print-launch-json "$PROMPT" \
  > "$ROOT/launch/base.launch.json"

test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
jq '.mode, .prompt' "$ROOT/launch/base.launch.json"

Billing example:

PROMPT='Generate three controlled fixes for duplicate billing webhook processing. Keep the provider contract and database schema fixed. Vary only the idempotency strategy. Each option must name files, tests, risks, and rollback notes.'

p --print-prompt "$PROMPT" \
  > "$ROOT/prompts/base.prompt.md"

p --print-launch-json "$PROMPT" \
  > "$ROOT/launch/base.launch.json"

Fill out:

# Seed Notes

- base prompt path:
- launch JSON path:
- software goal:
- invariant 1:
- invariant 2:
- variable:
- candidate count:
- verification command:
- seed ready? yes/no
- one improvement before fan-out:

Exercise 6: Checkpoint Repo State

Before fan-out, save the current repo state.

git status --short | tee "$ROOT/review/git-status-before.txt"
git diff > "$ROOT/review/dirty-before.patch"
git diff --staged > "$ROOT/review/staged-before.patch"

Fill out:

# Repo State Before Fan-Out

- status file:
- unstaged patch file:
- staged patch file:
- repo clean? yes/no
- if dirty, what is the dirty work:
- safe next move:

Choose one safe next move:

Situation Move
repo clean continue
intentional local work commit or stash
unrelated local work move it out of the candidate path
generated noise clean or move under artifacts/
unsure inspect before launching

Exercise 7: Choose Isolation

Pick the isolation level for your variation set.

Candidate Type Best Isolation Why
prompt rewrites only copied prompt/output folders
implementation candidates editing same files git worktrees
small sequential experiments branches
UI variants that can coexist feature flags
services that need ports unique process state
generated docs output directories

Write:

# Isolation Plan

- chosen isolation:
- why:
- what could collide:
- how collisions are prevented:
- where outputs go:
- where logs go:
- how to clean up:

Exercise 8: Build Candidate Prompts

Copy the base prompt, then append one candidate setting.

for id in v1 v2 v3; do
  cp "$ROOT/prompts/base.prompt.md" "$ROOT/prompts/$id.prompt.md"
done

Checkout UI integration example:

cat >> "$ROOT/prompts/v1.prompt.md" <<'EOF'
Candidate override:
- candidate id: v1
- variable: UI integration strategy
- setting: inline payment method selector in the checkout form
- do not change any other dimension
EOF

cat >> "$ROOT/prompts/v2.prompt.md" <<'EOF'
Candidate override:
- candidate id: v2
- variable: UI integration strategy
- setting: account drawer payment method selector
- do not change any other dimension
EOF

cat >> "$ROOT/prompts/v3.prompt.md" <<'EOF'
Candidate override:
- candidate id: v3
- variable: UI integration strategy
- setting: review-step payment method selector
- do not change any other dimension
EOF

Confirm:

for id in v1 v2 v3; do
  echo "== $id =="
  rg -n "Candidate override|variable:|setting:|do not change" "$ROOT/prompts/$id.prompt.md"
done

Fill out:

# Candidate Prompt Notes

| Candidate | Variable setting | Same base prompt? | Same invariants? |
| --- | --- | --- | --- |
| v1 | | | |
| v2 | | | |
| v3 | | | |

Exercise 9: Run The Same Verification

Pick one verification command for every candidate.

Examples:

VERIFY='pnpm test checkout -- --runInBand && pnpm build'
VERIFY='pnpm test billing-webhook -- --runInBand && pnpm build'
VERIFY='pnpm test auth -- --runInBand && pnpm build'
VERIFY='pnpm test -- --runInBand'

Run:

: > "$ROOT/review/verification.tsv"

for id in v1 v2 v3; do
  (
    cd "$PARENT/$id"
    bash -lc "$VERIFY"
  ) > "$ROOT/logs/$id.verify.log" 2>&1
  status=$?
  printf '%s\t%s\t%s\n' "$id" "$status" "$VERIFY" \
    >> "$ROOT/review/verification.tsv"
done

column -t -s $'\t' "$ROOT/review/verification.tsv"

Fill out:

# Verification Notes

- verification command:
- v1 status:
- v1 log:
- v2 status:
- v2 log:
- v3 status:
- v3 log:
- candidates excluded from review because they failed:
- candidates allowed into review:

Exercise 10: Build A Comparison File

Create a simple review surface.

cat > "$ROOT/review/comparison.md" <<EOF
# Variation Comparison

## Goal

## Variable Under Test

## Verification

| Candidate | Status | Log |
| --- | --- | --- |
| v1 | see \`verification.tsv\` | \`$ROOT/logs/v1.verify.log\` |
| v2 | see \`verification.tsv\` | \`$ROOT/logs/v2.verify.log\` |
| v3 | see \`verification.tsv\` | \`$ROOT/logs/v3.verify.log\` |

## Review Criteria

1.
2.
3.
4.
5.

## Notes

| Candidate | Strengths | Risks | Keep / Drop |
| --- | --- | --- | --- |
| v1 | | | |
| v2 | | | |
| v3 | | | |
EOF

Confirm:

test -s "$ROOT/review/comparison.md" && echo "comparison:OK"

Exercise 11: Pick The Decision Outcome

Choose one outcome.

Outcome Use When
Pick one candidate wins as-is
Combine one primary candidate wins, but named pieces from others should be merged
Refine one candidate wins but needs specific improvements
Expand the round was useful, but a new dimension needs another round
Reject the set is not worth continuing

Fill out:

# Decision Outcome

- outcome:
- primary candidate:
- secondary candidate ids:
- why:
- rejected candidates:
- what to keep:
- what to drop:
- required final verification command:

Exercise 12: Save The Decision And Convergence Prompt

Write the saved decision file.

cat > "$ROOT/decisions/selection.md" <<'EOF'
# Saved Decision

- outcome:
- primary candidate:
- secondary candidate ids:
- decision:
- why:
- rejected candidates:
- required follow-up:
- final verification command:

## Review Notes

- v1:
- v2:
- v3:

## Continue Prompt

Continue from this saved decision.
Do not reopen the entire search.
Preserve the selected candidate.
Apply only the named feedback.
Run final verification and save the result.
EOF

Create the convergence prompt:

cat "$ROOT/decisions/selection.md" \
  | p --stdin --print-prompt \
  "Use this saved decision to converge the variation set. Keep scope narrow and run the required verification." \
  > "$ROOT/prompts/converge.prompt.md"

test -s "$ROOT/decisions/selection.md" && echo "decision:OK"
test -s "$ROOT/prompts/converge.prompt.md" && echo "converge-prompt:OK"

Exercise 13: Failure Path Drill

For each failure, choose the recovery.

Failure Recovery
candidates changed multiple variables
verification command was different for every candidate
repo was dirty and unrelated changes leaked into diffs
v1 passed but merge into the main branch conflicts
best idea is split between v1 and v3
all candidates fail verification
review notes say “I like parts of all of them”
candidate output folders overwrote each other
UI candidates have no screenshots
Expand round changes the previous decision

Recovery options:

rewrite matrix
regenerate candidates
define verification first
checkpoint repo state
use worktrees or separate directories
pick a primary candidate
use Combine with named parts
use Reject
use Expand with a new matrix
save screenshots
rerun final verification

Fill out:

# Failure Recovery Notes

- failure:
- likely cause:
- recovery:
- command or file to inspect:
- how to know recovery worked:

Exercise 14: Final Check

Run these checks on your variation run.

test -s "$ROOT/matrix.md" && echo "matrix:OK"
test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
test -s "$ROOT/launch/base.launch.json" && echo "launch-json:OK"
find "$ROOT/prompts" -type f | sort
find "$ROOT/logs" -type f | sort
test -s "$ROOT/review/comparison.md" && echo "comparison:OK"
test -s "$ROOT/decisions/selection.md" && echo "decision:OK"

Fill out:

# Final Variation Run Check

- every candidate had the same base prompt? yes/no
- every candidate changed one variable? yes/no
- every candidate was isolated? yes/no
- every candidate used the same verification command? yes/no
- verification logs saved? yes/no
- comparison notes saved? yes/no
- decision saved? yes/no
- convergence prompt saved? yes/no
- ready to implement selected path? yes/no

Source references