Preference capture / personal RLHF (this-or-that)

Tool roles, alternatives, selection tradeoffs, and compatibility caveats.

Tool roles, alternatives, selection tradeoffs, and compatibility caveats.

Preference capture / personal RLHF (this-or-that)

John uses: the this-or-that kit (section 10; kit published at gentle-lasso-9d5n.here.now/kit/) — compare.html + a 30-line server.py + options.json: Elo pairing, arrow-key duels, both-bad/skip, per-vote notes, light/dark per-mode tracking; every vote appends to votes.jsonl, which agents read back as explicit taste rules. This repo's own theme was chosen by it (130 votes, 3 rounds).

Alternative What it is How it differs License
FastChat (Chatbot Arena) The platform behind Chatbot Arena: side-by-side battles + Elo ratings for LLM outputs The industrial version of the same idea (pairwise vote → Elo); heavyweight, model-comparison focused; check current repository activity Apache-2.0
Argilla Collaboration tool for building high-quality AI datasets, including preference/ranking records Team-scale dataset curation with a server; overkill for a solo taste record but the real thing for RLHF data Apache-2.0
Label Studio Multi-type data labeling tool with pairwise-comparison templates General annotation platform; pairwise comparison is one template among many Apache-2.0
potato Portable academic annotation tool (Python), supports pairwise/best-worst schemes Lightweight and self-hostable like the kit, config-driven rather than iframe-anything GPL-3.0
promptfoo Prompt/agent eval harness with side-by-side output comparison views Automated + human-graded evals over prompt variants — the 'evals' complement to taste voting MIT

Pick this if…

  • the kit itself — it's ~3 small files and free; there's little reason not to start with it.
  • promptfoo — your comparisons are prompt/agent outputs and you want automation around the human votes.
  • Argilla / Label Studio — you're collecting preferences from a team, not just yourself.
  • FastChat — you're literally comparing models and want battle-tested Elo infrastructure.
  • potato — you want a tiny self-hosted study-style setup with config-defined schemes.

Source references