---
schema_version: 1
revision: "22ce5d34dbcb5a78"
id: "factory-tools-preference-rlhf"
title: "Preference capture / personal RLHF (this-or-that)"
---

# Preference capture / personal RLHF (this-or-that)

Tool roles, alternatives, selection tradeoffs, and compatibility caveats.

## Preference capture / personal RLHF (this-or-that)

**John uses:** the **this-or-that kit** (section 10; kit published at [gentle-lasso-9d5n.here.now/kit/](https://gentle-lasso-9d5n.here.now/)) — `compare.html` + a 30-line `server.py` + `options.json`: Elo pairing, arrow-key duels, both-bad/skip, per-vote notes, light/dark per-mode tracking; every vote appends to `votes.jsonl`, which agents read back as explicit taste rules. This repo's own theme was chosen by it (130 votes, 3 rounds).

| Alternative | What it is | How it differs | License |
|----|----|----|----|
| [FastChat (Chatbot Arena)](https://github.com/lm-sys/FastChat) | The platform behind Chatbot Arena: side-by-side battles + Elo ratings for LLM outputs | The industrial version of the same idea (pairwise vote → Elo); heavyweight, model-comparison focused; check current repository activity | Apache-2.0 |
| [Argilla](https://github.com/argilla-io/argilla) | Collaboration tool for building high-quality AI datasets, including preference/ranking records | Team-scale dataset curation with a server; overkill for a solo taste record but the real thing for RLHF data | Apache-2.0 |
| [Label Studio](https://github.com/HumanSignal/label-studio) | Multi-type data labeling tool with pairwise-comparison templates | General annotation platform; pairwise comparison is one template among many | Apache-2.0 |
| [potato](https://github.com/davidjurgens/potato) | Portable academic annotation tool (Python), supports pairwise/best-worst schemes | Lightweight and self-hostable like the kit, config-driven rather than iframe-anything | GPL-3.0 |
| [promptfoo](https://github.com/promptfoo/promptfoo) | Prompt/agent eval harness with side-by-side output comparison views | Automated + human-graded evals over prompt variants — the 'evals' complement to taste voting | MIT |

### Pick this if…

- **the kit itself** — it's ~3 small files and free; there's little reason not to start with it.
- **promptfoo** — your comparisons are prompt/agent outputs and you want automation around the human votes.
- **Argilla / Label Studio** — you're collecting preferences from a *team*, not just yourself.
- **FastChat** — you're literally comparing models and want battle-tested Elo infrastructure.
- **potato** — you want a tiny self-hosted study-style setup with config-defined schemes.

## Original references

- [Tool comparison reference](https://dewy-sleet-n5zs.here.now/#preference-rlhf)
