These exercises practice turning one deliberate prompt into controlled, verified candidates.
Use a real software project when possible. Keep each exercise small. The goal is not to generate the most candidates; the goal is to compare useful options without losing control.
Exercise 1: Decide Whether Variations Are Appropriate
For each task, choose use variations or skip variations.
| Task | Use / Skip | Why |
|---|---|---|
| Add saved payment methods to checkout, with several plausible UI entry points | ||
| Fix a one-character typo in a route name | ||
| Stop duplicate billing webhook processing, with multiple plausible idempotency strategies | ||
| Apply a strict framework migration with exact steps | ||
| Compare auth/session architecture before refresh-token rotation | ||
| Implement an exact requested API shape | ||
| Choose the right test boundary for a regression | ||
| Repo is dirty and there is no saved state | ||
| No verification command exists for the candidates |
Write:
# Variation Decision
- task:
- decision:
- why:
- verification available:
- state safe for fan-out:
Exercise 2: Find The Variable
Pick one request and identify the single variable under test.
| Request | Possible Variable |
|---|---|
| Checkout saved payment methods could be integrated several ways. | UI integration strategy |
| Billing webhooks sometimes process the same invoice twice. | idempotency strategy |
| Auth/session code is becoming hard to reason about. | architecture boundary |
| This regression needs better coverage. | test boundary |
| The agent keeps producing broad implementation plans. | prompt framing |
| The docs are technically correct but hard to use. | documentation organization |
Fill out:
# Variable Notes
- request:
- chosen variable:
- why this variable matters:
- invariants:
-
-
-
- out of scope:
-
-
-
- candidate count:
- review question:
Rule:
If you cannot finish "we are comparing candidates that differ only by ___," stop and narrow.
Exercise 3: Write A Variation Matrix
Use one of these scenarios:
checkout saved payment methods
billing webhook duplicate-processing
auth/session architecture
test strategy for a regression
Template:
# Variation Matrix
## Goal
## Invariants
- same repo state:
- same base prompt:
- same target feature or bug:
- same constraints:
- same verification command:
- same review criteria:
## Variable Under Test
- variable:
- why this matters:
- what is out of scope:
## Candidate Count
- count:
- ids:
## Candidates
| ID | Variable setting | Output location | Verification command | Notes |
| --- | --- | --- | --- | --- |
| v1 | | | | |
| v2 | | | | |
| v3 | | | | |
## Review Criteria
1.
2.
3.
4.
5.
Check it:
rg -n "Goal|Invariants|Variable Under Test|Candidate Count|Review Criteria" matrix.md
Exercise 4: Create The Run Folder
Create a run root under artifacts/variations.
RUN_ID="exercise-$(date +%Y%m%d-%H%M%S)"
ROOT="artifacts/variations/$RUN_ID"
mkdir -p "$ROOT"/{prompts,launch,logs,review,decisions}
mkdir -p "$ROOT"/candidates/{v1,v2,v3}
find "$ROOT" -maxdepth 3 -type d | sort
Fill out:
# Run Folder Notes
- run id:
- root:
- prompt folder:
- launch folder:
- candidate folders:
- logs folder:
- review folder:
- decisions folder:
Exercise 5: Save The Section 5 Seed
Use Prompt Builder to save the base prompt and launch JSON.
Checkout example:
PROMPT='Generate three controlled implementations for saved payment methods in checkout. Keep the database contract and provider API fixed. Vary only the UI integration strategy. Each option must include files to edit, tests to run, risks, and rollback notes.'
p --print-prompt "$PROMPT" \
> "$ROOT/prompts/base.prompt.md"
p --print-launch-json "$PROMPT" \
> "$ROOT/launch/base.launch.json"
test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
jq '.mode, .prompt' "$ROOT/launch/base.launch.json"
Billing example:
PROMPT='Generate three controlled fixes for duplicate billing webhook processing. Keep the provider contract and database schema fixed. Vary only the idempotency strategy. Each option must name files, tests, risks, and rollback notes.'
p --print-prompt "$PROMPT" \
> "$ROOT/prompts/base.prompt.md"
p --print-launch-json "$PROMPT" \
> "$ROOT/launch/base.launch.json"
Fill out:
# Seed Notes
- base prompt path:
- launch JSON path:
- software goal:
- invariant 1:
- invariant 2:
- variable:
- candidate count:
- verification command:
- seed ready? yes/no
- one improvement before fan-out:
Exercise 6: Checkpoint Repo State
Before fan-out, save the current repo state.
git status --short | tee "$ROOT/review/git-status-before.txt"
git diff > "$ROOT/review/dirty-before.patch"
git diff --staged > "$ROOT/review/staged-before.patch"
Fill out:
# Repo State Before Fan-Out
- status file:
- unstaged patch file:
- staged patch file:
- repo clean? yes/no
- if dirty, what is the dirty work:
- safe next move:
Choose one safe next move:
| Situation | Move |
|---|---|
| repo clean | continue |
| intentional local work | commit or stash |
| unrelated local work | move it out of the candidate path |
| generated noise | clean or move under artifacts/ |
| unsure | inspect before launching |
Exercise 7: Choose Isolation
Pick the isolation level for your variation set.
| Candidate Type | Best Isolation | Why |
|---|---|---|
| prompt rewrites only | copied prompt/output folders | |
| implementation candidates editing same files | git worktrees | |
| small sequential experiments | branches | |
| UI variants that can coexist | feature flags | |
| services that need ports | unique process state | |
| generated docs | output directories |
Write:
# Isolation Plan
- chosen isolation:
- why:
- what could collide:
- how collisions are prevented:
- where outputs go:
- where logs go:
- how to clean up:
Exercise 8: Build Candidate Prompts
Copy the base prompt, then append one candidate setting.
for id in v1 v2 v3; do
cp "$ROOT/prompts/base.prompt.md" "$ROOT/prompts/$id.prompt.md"
done
Checkout UI integration example:
cat >> "$ROOT/prompts/v1.prompt.md" <<'EOF'
Candidate override:
- candidate id: v1
- variable: UI integration strategy
- setting: inline payment method selector in the checkout form
- do not change any other dimension
EOF
cat >> "$ROOT/prompts/v2.prompt.md" <<'EOF'
Candidate override:
- candidate id: v2
- variable: UI integration strategy
- setting: account drawer payment method selector
- do not change any other dimension
EOF
cat >> "$ROOT/prompts/v3.prompt.md" <<'EOF'
Candidate override:
- candidate id: v3
- variable: UI integration strategy
- setting: review-step payment method selector
- do not change any other dimension
EOF
Confirm:
for id in v1 v2 v3; do
echo "== $id =="
rg -n "Candidate override|variable:|setting:|do not change" "$ROOT/prompts/$id.prompt.md"
done
Fill out:
# Candidate Prompt Notes
| Candidate | Variable setting | Same base prompt? | Same invariants? |
| --- | --- | --- | --- |
| v1 | | | |
| v2 | | | |
| v3 | | | |
Exercise 9: Run The Same Verification
Pick one verification command for every candidate.
Examples:
VERIFY='pnpm test checkout -- --runInBand && pnpm build'
VERIFY='pnpm test billing-webhook -- --runInBand && pnpm build'
VERIFY='pnpm test auth -- --runInBand && pnpm build'
VERIFY='pnpm test -- --runInBand'
Run:
: > "$ROOT/review/verification.tsv"
for id in v1 v2 v3; do
(
cd "$PARENT/$id"
bash -lc "$VERIFY"
) > "$ROOT/logs/$id.verify.log" 2>&1
status=$?
printf '%s\t%s\t%s\n' "$id" "$status" "$VERIFY" \
>> "$ROOT/review/verification.tsv"
done
column -t -s $'\t' "$ROOT/review/verification.tsv"
Fill out:
# Verification Notes
- verification command:
- v1 status:
- v1 log:
- v2 status:
- v2 log:
- v3 status:
- v3 log:
- candidates excluded from review because they failed:
- candidates allowed into review:
Exercise 10: Build A Comparison File
Create a simple review surface.
cat > "$ROOT/review/comparison.md" <<EOF
# Variation Comparison
## Goal
## Variable Under Test
## Verification
| Candidate | Status | Log |
| --- | --- | --- |
| v1 | see \`verification.tsv\` | \`$ROOT/logs/v1.verify.log\` |
| v2 | see \`verification.tsv\` | \`$ROOT/logs/v2.verify.log\` |
| v3 | see \`verification.tsv\` | \`$ROOT/logs/v3.verify.log\` |
## Review Criteria
1.
2.
3.
4.
5.
## Notes
| Candidate | Strengths | Risks | Keep / Drop |
| --- | --- | --- | --- |
| v1 | | | |
| v2 | | | |
| v3 | | | |
EOF
Confirm:
test -s "$ROOT/review/comparison.md" && echo "comparison:OK"
Exercise 11: Pick The Decision Outcome
Choose one outcome.
| Outcome | Use When |
|---|---|
| Pick | one candidate wins as-is |
| Combine | one primary candidate wins, but named pieces from others should be merged |
| Refine | one candidate wins but needs specific improvements |
| Expand | the round was useful, but a new dimension needs another round |
| Reject | the set is not worth continuing |
Fill out:
# Decision Outcome
- outcome:
- primary candidate:
- secondary candidate ids:
- why:
- rejected candidates:
- what to keep:
- what to drop:
- required final verification command:
Exercise 12: Save The Decision And Convergence Prompt
Write the saved decision file.
cat > "$ROOT/decisions/selection.md" <<'EOF'
# Saved Decision
- outcome:
- primary candidate:
- secondary candidate ids:
- decision:
- why:
- rejected candidates:
- required follow-up:
- final verification command:
## Review Notes
- v1:
- v2:
- v3:
## Continue Prompt
Continue from this saved decision.
Do not reopen the entire search.
Preserve the selected candidate.
Apply only the named feedback.
Run final verification and save the result.
EOF
Create the convergence prompt:
cat "$ROOT/decisions/selection.md" \
| p --stdin --print-prompt \
"Use this saved decision to converge the variation set. Keep scope narrow and run the required verification." \
> "$ROOT/prompts/converge.prompt.md"
test -s "$ROOT/decisions/selection.md" && echo "decision:OK"
test -s "$ROOT/prompts/converge.prompt.md" && echo "converge-prompt:OK"
Exercise 13: Failure Path Drill
For each failure, choose the recovery.
| Failure | Recovery |
|---|---|
| candidates changed multiple variables | |
| verification command was different for every candidate | |
| repo was dirty and unrelated changes leaked into diffs | |
| v1 passed but merge into the main branch conflicts | |
| best idea is split between v1 and v3 | |
| all candidates fail verification | |
| review notes say “I like parts of all of them” | |
| candidate output folders overwrote each other | |
| UI candidates have no screenshots | |
| Expand round changes the previous decision |
Recovery options:
rewrite matrix
regenerate candidates
define verification first
checkpoint repo state
use worktrees or separate directories
pick a primary candidate
use Combine with named parts
use Reject
use Expand with a new matrix
save screenshots
rerun final verification
Fill out:
# Failure Recovery Notes
- failure:
- likely cause:
- recovery:
- command or file to inspect:
- how to know recovery worked:
Exercise 14: Final Check
Run these checks on your variation run.
test -s "$ROOT/matrix.md" && echo "matrix:OK"
test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
test -s "$ROOT/launch/base.launch.json" && echo "launch-json:OK"
find "$ROOT/prompts" -type f | sort
find "$ROOT/logs" -type f | sort
test -s "$ROOT/review/comparison.md" && echo "comparison:OK"
test -s "$ROOT/decisions/selection.md" && echo "decision:OK"
Fill out:
# Final Variation Run Check
- every candidate had the same base prompt? yes/no
- every candidate changed one variable? yes/no
- every candidate was isolated? yes/no
- every candidate used the same verification command? yes/no
- verification logs saved? yes/no
- comparison notes saved? yes/no
- decision saved? yes/no
- convergence prompt saved? yes/no
- ready to implement selected path? yes/no