The trouble with scores
Ask someone to rate a film out of ten and you are really asking them to hold an entire history of films in mind, place this one within it, and compress the answer into a single number. Nobody does that. What people actually report is a feeling, filtered through mood, recency, and company. The result is familiar to anyone who has kept ratings for a while: almost everything lands between 7 and 8.5, and the 8.0 you gave something years ago no longer means what an 8.0 means today.
This isn't carelessness. Decades of research on human judgment point the same way: people are unreliable at absolute judgments and remarkably consistent at relative ones. We struggle to say how heavy a stone is; we rarely err about which of two stones is heavier.
Comparison is the easier question
Pairwise ranking takes that seriously. Instead of asking for a number, it asks the question you can actually answer: of these two, which do you prefer? Each answer is a small, honest data point, unaffected by how generous you felt the day you rated something. String enough of them together and a full ranking emerges, the way chess ratings emerge from individual games rather than from anyone declaring a player an "8 out of 10."
You need fewer comparisons than you'd think
A list of 50 items has 1,225 possible pairs, but you don't need anywhere near that many answers. Ordering a list by comparison is a sorting problem, and sorting is one of the most polished tools in computing:
- Placing one new item into an already-ordered list takes about log₂(n) comparisons: seven questions to file something into a list of 100.
- Ordering a whole list from scratch takes roughly n·log₂(n): a 20-item list is sorted in well under a hundred quick duels, most of which take a second each.
In practice it feels less like a chore than like a quiz about your own taste, and it can be spread out: answer five duels today, five next week, and the order keeps sharpening.
When your answers contradict each other
Sooner or later you will prefer A to B, B to C, and then C to A. A cycle like that isn't a failure; it is information. It usually means the three are genuinely close, or that you are comparing on different dimensions each time: A beats B on craft, B beats C on ambition, C beats A on sheer fun. A good system doesn't pretend the contradiction away. It can ask again later, let your scored ratings arbitrate, or simply accept that positions four through six are a matter of mood.
Scores and duels, together
None of this makes scores worthless. Scores carry information a duel cannot: they mark the gulf between your best and your worst, they support separate dimensions (a film's story versus its visuals, a beer's taste versus its look), and they give you tiers, averages, and trends. Duels are the precision tool; scores are the map.
That is why RateTheThings uses both. Weighted dimension scores build the leaderboard and the tier list; quick pairwise duels refine the order exactly where numbers blur, among the neighbors an 8.2 and an 8.3 can't honestly separate. If you want to see how it feels, start a list and rate a few things; the duels take care of themselves.