Controlled variation workflows: team walkthrough

Controlled software variations using one variable, a shared prompt seed, isolated candidates, common verification, comparison records, and explicit Pick/Combine/Refine/Expand/Reject decisions, including billing-webhook examples and convergence failure…

Team Learning Goal

Teach controlled variation as a decision workflow.

The point is not “generate more.” The point is:

Generate comparable options, verify each option, then preserve the decision.

By the end of this section, the room should understand:

  • why cheap generation changes the workflow
  • why quality, integration, and verification are not free
  • how to choose one variable under test
  • how to write a variation matrix
  • why isolation matters before fan-out
  • why the same verification command must run for every candidate
  • how to decide Pick / Combine / Refine / Expand / Reject
  • how Section 5’s saved prompt becomes the seed
  • how Section 7 expands what can become prompt material

Walkthrough Depth

Essential Walkthrough

  1. State: Code is free does not mean quality is free.
  2. Teach the One Dimension Rule.
  3. Show a billing webhook variation matrix.
  4. Show the artifacts/variations/<run-id>/ folder shape.
  5. Show one verification command applied to all candidates.
  6. End with Pick / Combine / Refine / Expand / Reject.

Preserve:

goal -> variable -> matrix -> isolation -> fan-out -> verify -> review -> selection -> convergence -> saved decision

Expanded Walkthrough

Add:

  • dirty repo checkpoint
  • Prompt Builder seed prompt
  • worktree isolation
  • candidate prompt overrides
  • comparison file

Deep-Dive Walkthrough

Add:

  • visible cmux fan-out
  • screenshots for UI candidates
  • convergence prompt from saved decision
  • failure paths
  • exercises

Talking Points

Opening:

Code is free does not mean quality is free.

On variations:

Variation is a decision tool, not a procrastination tool.

On the One Dimension Rule:

If every candidate changes everything, comparison teaches nothing.

On verification:

Unverified candidates are just more text.

On isolation:

Parallel work needs lanes. Without lanes, candidates overwrite each other.

On Combine:

Combine still needs a primary candidate. Otherwise it becomes “take everything I liked” and scope explodes.

Bridge from Section 5:

Section 5 made the prompt deliberate. Section 6 turns that prompt into comparable options.

Bridge to Section 7:

We just learned to vary from one prompt. Next, we widen what can count as prompt material.

Live Demo Script

1. Start With The Pattern

goal -> variable -> matrix -> isolation -> fan-out -> verify -> review -> selection -> convergence -> saved decision

2. Write The Variable

We are comparing candidates that differ only by idempotency strategy.

3. Create The Run Folder

RUN_ID="billing-webhook-$(date +%Y%m%d-%H%M%S)"
ROOT="artifacts/variations/$RUN_ID"
mkdir -p "$ROOT"/{prompts,launch,logs,review,decisions}
mkdir -p "$ROOT"/candidates/{v1,v2,v3}
find "$ROOT" -maxdepth 3 -type d | sort

4. Save A Prompt Builder Seed

PROMPT='Generate three controlled fixes for duplicate billing webhook processing. Keep the provider contract and database schema fixed. Vary only the idempotency strategy. Each option must name files, tests, risks, and rollback notes.'

p --print-prompt "$PROMPT" > "$ROOT/prompts/base.prompt.md"
p --print-launch-json "$PROMPT" > "$ROOT/launch/base.launch.json"
test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"

5. Check Repo State

git status --short | tee "$ROOT/review/git-status-before.txt"
git diff > "$ROOT/review/dirty-before.patch"
git diff --staged > "$ROOT/review/staged-before.patch"

Presenter note:

Dirty state is not forbidden. Unknown dirty state is the problem.

6. Show Same Verification

VERIFY='pnpm test billing-webhook -- --runInBand && pnpm build'
printf 'v1\t<status>\t%s\n' "$VERIFY" > "$ROOT/review/verification.tsv"
printf 'v2\t<status>\t%s\n' "$VERIFY" >> "$ROOT/review/verification.tsv"
printf 'v3\t<status>\t%s\n' "$VERIFY" >> "$ROOT/review/verification.tsv"
column -t -s $'\t' "$ROOT/review/verification.tsv"

If real tests are too slow live, show the shape and say:

In a real project, this is where broken candidates get filtered before review.

7. Save The Decision

cat > "$ROOT/decisions/selection.md" <<'EOF'
# Saved Decision

- outcome: Combine
- primary candidate: v2
- secondary candidate ids: v1
- decision: Use v2 as the base and take v1's rollback guard.
- final verification command: pnpm test billing-webhook -- --runInBand && pnpm build

## Continue Prompt

Continue from this saved decision.
Preserve v2 as the primary candidate.
Apply only the named v1 rollback guard.
Run final verification.
EOF

Minimal Useful Path

If 5 minutes behind, cut:

  • visible cmux fan-out
  • feature flag isolation
  • screenshots
  • convergence prompt command

Preserve:

  • Code is free does not mean quality is free
  • One Dimension Rule
  • variation matrix
  • dirty repo checkpoint
  • same verification command
  • Pick / Combine / Refine / Expand / Reject

If 10 minutes behind, skip worktree creation and use output folders only:

mkdir -p "$ROOT"/candidates/{v1,v2,v3}

If 20 minutes behind, teach only:

One deliberate prompt becomes a matrix.
The matrix changes one dimension.
Each candidate gets isolation.
Each candidate gets the same verification.
The final decision is Pick, Combine, Refine, Expand, or Reject.

Boundaries

From Section 5

Section 5 solves composition:

What exactly am I about to send?

Section 6 solves controlled fan-out:

What do I do after the prompt is good enough to become a seed?

To Section 7

Section 6 assumes the seed is mostly text plus project context.

Section 7 expands the prompt material:

video can carry visual state, timing, workflow, cursor motion, and failure evidence

Failure Paths To Preserve

Candidates drift:

Drift means the matrix was too weak or the candidate prompt was too loose. Stop and tighten the variable.

Unverifiable options:

If you cannot verify the candidates, you are setting up a beauty contest. Define verification first.

Dirty repo:

Dirty state is not forbidden. Unknown dirty state is the problem.

Merge conflicts:

The selected candidate worked in isolation. Convergence has to protect that proof, not bulldoze it.

Strongest idea split across candidates:

That is Combine. Pick a primary candidate and name the pieces you are taking.

All candidates fail:

That is useful information. Reject the set or expand with a tighter prompt. Do not pretend one failed candidate won.

Do Not Over-Explain

Avoid spending time on:

  • every local helper
  • every possible git worktree cleanup command
  • full review UI implementation details
  • exhaustive parallel orchestration
  • model-provider comparisons
  • abstract creativity theory

Keep the section practical:

matrix
isolation
fan-out
verification
review
saved decision

Team Checklist

Before considering this section ready, confirm the materials include:

Code is free
quality is not free
integration is not free
verification is not free
controlled variations
One Dimension Rule
good variables
bad variables
when to use variations
when not to use variations
goal -> variable -> matrix -> isolation -> fan-out -> verify -> review -> selection -> convergence -> saved decision
variation matrix template
Prompt Builder seed
artifacts/variations/<run-id>/
base.prompt.md
base.launch.json
dirty repo checkpoint
worktrees
branches
directories
feature flags
same verification command
comparison notes
diff review
screenshots for UI
test results
Pick
Combine
Refine
Expand
Reject
failure paths
Section 7 bridge to video

Practice Launch Notes

Good demo scenarios:

Scenario Variable
checkout saved payment methods UI integration strategy
billing webhook duplicate-processing idempotency strategy
auth/session architecture architecture boundary
flaky regression coverage test boundary
checkout implementation implementation alternative

Avoid:

make the app better
improve architecture
try some ideas
fix everything

The cleanest live example is billing webhook idempotency because correctness, verification, rollback, and risk are easy to discuss.

Source references