Hot or Cold
Hot or Cold is a real-time multiplayer party game. One player gets a hidden number on a spectrum like Overrated to Underrated. They write a single clue. Everyone else places a dial where they think it sits. The arguing is the game.
It is also a measurement instrument. Most human preference data is either scraped without permission or collected in a lab, where people answer differently because they know they are being watched. This is a third option. A game people want to play on a Friday night, where playing produces a clean consented record of how a person judges things. Invisible at the table, rigorous in the database.
Built solo across five months and 176 commits: the game, the content, the infrastructure and the research layer. Real players arrived in July 2026. The corpus now holds 45,935 guesses from 6,997 people.
The stack
TypeScript pnpm monorepo in four packages. Fastify 5 with Colyseus 0.16 holds live rooms over WebSocket on Fly.io. React 18.3 with Vite 6 and TanStack Router on Cloudflare Pages. Neon Postgres with Drizzle, 16 tables and 10 migrations. Auth is an emailed code, no passwords. It also ships as a Discord Activity that runs inside a voice channel with nothing to install.
Every round writes a versioned JSONB payload rather than a fixed row, so the capture schema can change without orphaning history. Player identifiers are salted one-way hashes and consent is versioned, so it is always clear which terms a given row was collected under.
The modes measure different things
The round types are not variety for its own sake. Classic gives one player a hidden target and one clue, and measures whether meaning survives the trip between two people. Crowd Read asks for your own position and your prediction of the room on the same item, which is the pairing almost no dataset has. Hot Seat puts one player on the spectrum and everyone else places them, which is the only mode that captures how a group sees an individual.
Hot Seat is also the most sensitive thing in the system, so it never leaves in any external dataset. Poles are randomised every round so left and right carry no fixed meaning, with the canonical orientation stored alongside so analysis is not fooled by the flip.
The measurement result
The benchmark asks a predictor to pick which of four clues a specific person wrote, with the three decoys drawn from the same spectrum by different authors. Chance is 25 percent.
A predictor that knows the whole clue corpus but nothing about the person scores 24.3 percent, which is chance. A predictor that knows the person scores 31.3 percent. That gap is seven points across 1,051 people.
The comparison is deliberately personal against population rather than personal against chance. Scoring a personal predictor against chance quietly takes credit for everything general knowledge already earns, which is how this kind of result usually gets overstated.
Building solo with a model
No model runs inside a round. Bots play from a precomputed clue bank, not live inference. A forty second timer is the wrong place for an inference call, and a model in the request path would have made the game slower and the data noisier.
The models are in how it got built. Claude Code wrote most of the lines across five months while I designed the game, the modes, the data model and the measurement, and reviewed every change. The split held because the model is fast at producing code and has no opinion about whether a round type earns its place, whether a number is honest, or whether a field is safe to publish.
The clearest example is the measurement work. I asked for the analysis that tested my own assumption about how well people read each other, and the answer came back weaker than the pitch. Publishing the weaker number is the entire point of having built an instrument rather than a demo.
