{
  "schemaVersion": "1",
  "revision": "22ce5d34dbcb5a78",
  "id": "factory-tools-preference-rlhf",
  "slug": "tools-preference-rlhf",
  "title": "Preference capture / personal RLHF (this-or-that)",
  "summary": "Tool roles, alternatives, selection tradeoffs, and compatibility caveats.",
  "category": "Tools",
  "tags": [
    "tools",
    "preference-rlhf"
  ],
  "sources": [
    {
      "label": "Tool comparison reference",
      "url": "https://dewy-sleet-n5zs.here.now/#preference-rlhf"
    }
  ],
  "sourceUnits": [
    "resources/index.html#preference-rlhf"
  ],
  "collection": "factory",
  "categoryId": "tools",
  "wordCount": 324,
  "url": "/topics/factory-tools-preference-rlhf/",
  "markdownUrl": "/markdown/factory-tools-preference-rlhf.md?raw=1&v=22ce5d34dbcb5a78",
  "jsonUrl": "/api/v1/documents/factory-tools-preference-rlhf.json",
  "headings": [
    {
      "id": "preference-capture-personal-rlhf-this-or-that",
      "text": "Preference capture / personal RLHF (this-or-that)",
      "level": 2
    },
    {
      "id": "pick-this-if",
      "text": "Pick this if…",
      "level": 3
    }
  ],
  "markdown": "# Preference capture / personal RLHF (this-or-that)\n\nTool roles, alternatives, selection tradeoffs, and compatibility caveats.\n\n## Preference capture / personal RLHF (this-or-that)\n\n**John uses:** the **this-or-that kit** (section 10; kit published at [gentle-lasso-9d5n.here.now/kit/](https://gentle-lasso-9d5n.here.now/)) — `compare.html` + a 30-line `server.py` + `options.json`: Elo pairing, arrow-key duels, both-bad/skip, per-vote notes, light/dark per-mode tracking; every vote appends to `votes.jsonl`, which agents read back as explicit taste rules. This repo's own theme was chosen by it (130 votes, 3 rounds).\n\n| Alternative | What it is | How it differs | License |\n|----|----|----|----|\n| [FastChat (Chatbot Arena)](https://github.com/lm-sys/FastChat) | The platform behind Chatbot Arena: side-by-side battles + Elo ratings for LLM outputs | The industrial version of the same idea (pairwise vote → Elo); heavyweight, model-comparison focused; check current repository activity | Apache-2.0 |\n| [Argilla](https://github.com/argilla-io/argilla) | Collaboration tool for building high-quality AI datasets, including preference/ranking records | Team-scale dataset curation with a server; overkill for a solo taste record but the real thing for RLHF data | Apache-2.0 |\n| [Label Studio](https://github.com/HumanSignal/label-studio) | Multi-type data labeling tool with pairwise-comparison templates | General annotation platform; pairwise comparison is one template among many | Apache-2.0 |\n| [potato](https://github.com/davidjurgens/potato) | Portable academic annotation tool (Python), supports pairwise/best-worst schemes | Lightweight and self-hostable like the kit, config-driven rather than iframe-anything | GPL-3.0 |\n| [promptfoo](https://github.com/promptfoo/promptfoo) | Prompt/agent eval harness with side-by-side output comparison views | Automated + human-graded evals over prompt variants — the 'evals' complement to taste voting | MIT |\n\n### Pick this if…\n\n- **the kit itself** — it's ~3 small files and free; there's little reason not to start with it.\n- **promptfoo** — your comparisons are prompt/agent outputs and you want automation around the human votes.\n- **Argilla / Label Studio** — you're collecting preferences from a *team*, not just yourself.\n- **FastChat** — you're literally comparing models and want battle-tested Elo infrastructure.\n- **potato** — you want a tiny self-hosted study-style setup with config-defined schemes."
}
