How Ranklist ranks things

Why not just count upvotes

An upvote answers a question nobody asked. It tells you that someone liked something, not how much they liked it compared to the alternatives. On a list of fifty films, upvotes reward whatever is most famous, because fame is what gets clicked. A single passionate group can carry an entry to the top by voting for it and ignoring everything else, and the result reads as a popularity contest rather than an assessment.

Star ratings fail differently. Averaging stars lets a single five-star rating on a brand new entry outrank a four-and-a-half-star average across ten thousand ratings, which is obviously wrong and is exactly why almost nobody publishes a plain average.

Ranklist asks for something harder and more informative: a full ordering. When you rank a list you are not saying “I like this”, you are saying “I put this above that and below the other thing”. That single change makes the aggregate meaningful, because every item on your ballot is measured against every other item by the same person at the same moment.

Step 1: your ballot becomes points

On a list with N items, an item you place at position r earns N minus r plus 1 points. Top of a twenty item list is worth 20 points, second is worth 19, and the bottom position is worth 1. This is a Borda count, a method for turning ordered preferences into a single number that has been studied and argued over since the eighteenth century. Points never go below zero, which only matters when an item is deleted from a list after ballots were already cast.

• One ballot per person per list. Re-ranking replaces your previous ballot rather than adding a second one. You cannot vote twice by voting again.
• Partial ballots are allowed. Items you never placed are simply absent from your ballot. They receive no points from you and no penalty from you, and your ballot does not count toward their vote total.
• Tier placements collapse to positions. If you rank in tier mode, the top of your S tier is position 1, the next item down is position 2, and so on through the bottom of your lowest tier. Your personal view keeps the tier letters; the global count sees a strict order, so a tier ballot and a list ballot are directly comparable.

Step 2: average, not total

The obvious next move is to add every item’s points together and sort. We did that at first, and it produced a specific, ugly failure. An item added to a list in month six can never catch up, because every ballot cast in months one through five contributed zero points to it. On one long-running list, characters with three first-place votes were pinned near the bottom behind items with hundreds of middling votes, forever. The total was measuring how long an item had existed, not how good it was.

So the score starts from the average instead. For each item we compute R, its average Borda points across the ballots that included it, and v, the number of those ballots. R answers “how highly do people who ranked this place it”. v answers “how much evidence is that”.

On its own, R is the star-rating problem again: one ballot placing an item first gives it a perfect average. That is what step 3 is for.

Step 3: smoothing toward the list’s own center

The published score blends an item’s own average with the list average, weighted by how much evidence the item has:

• R is the item’s own average Borda points, and v is the number of ballots behind it.
• C is the prior: the mean of every item’s R on that same list. It is not a global constant tuned by us, it is the list’s own center of gravity, recomputed from the same ballots. For a list with no ballots at all it falls back to the exact middle position, (N + 1) / 2.
• m is the confidence threshold: ten percent of the median ballot count among the items that have been ranked at least once, clamped to at least 5 and at most 50. Read it as “how many ballots an item needs before we trust its own average more than we trust the list average”. Quiet lists sit at 5 so newcomers differentiate quickly, and heavily ranked lists push toward 50, so climbing takes real, sustained support.

The behavior that falls out of that formula is the whole point:

• At v = 0 the score is exactly C. An item nobody has ranked sits at the list average, neither rewarded nor buried.
• At v = m the score is exactly halfway between the item’s own average and the list average.
• As v grows past m the prior’s weight decays toward nothing and the score converges on R. Items with real support are governed by their own record.

This is the same shape as IMDb’s weighted rating for its Top 250 and the same idea as a Bayesian average anywhere else. The novelty here is not the formula, it is that both C and m are derived per list from that list’s own ballots rather than hardcoded, so a list with nine rankers and a list with nine thousand are each judged on their own scale.

A worked example

The numbers below come from a made up sample list, not from live site data. Five items, labeled A through E, and four ballots. Since N = 5, first place is worth 5 points and last place is worth 1.

The ballots

The arithmetic

Every item appears on all four ballots, so v = 4 for each of them. The median vote count is 4, and ten percent of 4 is 0.4, which the clamp raises to the floor of m = 5. The mean of the five item averages is C = 3.00. With v = 4 and m = 5, each item’s own record carries 4/9 of the weight and the prior carries 5/9.

Published order: A, B, C, D, E. Notice how compressed the scores are, spanning 2.22 to 3.67 rather than 1.25 to 4.50. That compression is the system saying out loud that it does not yet trust four ballots very much. The ordering is already confident; the distances are not.

Now add a newcomer

Someone adds item F, so N becomes 6 and first place is now worth 6 points. A fifth ballot arrives ranking F first, then A, B, C, D, E. The four earlier ballots never saw F at all. Recomputing everything from scratch: the new prior is C = 4.17, the median vote count is 5, so m stays at 5.

F enters at third place. That is the interesting result, so it is worth spelling out what the alternatives would have done with the same single ballot:

• Summed Borda points would put F last with 6 points against E’s 10, purely for arriving late.
• A raw average with no smoothing would put F first outright, at a perfect 6.00, on the strength of one person’s opinion.
• The smoothed score places it third: visible, credited for a strong first showing, and not yet allowed to displace items with five ballots behind them. It also carries a Rising badge until it clears the confidence threshold, which tells readers the position is provisional and tells rankers where their ballot would matter most.

If the next four ballots also place F first, its v reaches 5, the prior’s weight falls from five sixths to a half, and F climbs on merit. If they do not, it drifts back toward the middle. Either way the list is telling the truth about how much it knows.

Ties

Two items can land on the same smoothed score. The first tie-break is the average position across ballots: the item people typically placed higher wins. Items nobody has ranked have no average position, so they sort below any ranked item they tie with rather than above it.

If two items tie on both the score and the average position, they are genuinely indistinguishable given the ballots cast, and we do not invent a winner. Their relative order is arbitrary and may swap until a ballot separates them. We would rather show an honest coin flip than a fake precision.

Why the numbers move

Rankings are computed from the ballot table at the moment you load the page. There is no cached leaderboard that someone refreshes on a schedule, which means four different things can move an item between two visits:

• A new ballot includes the item, changing its R and v.
• A new ballot changes the list average C, which moves every item that is still leaning on the prior, including items that ballot never mentioned.
• The median vote count rises, pushing m up, so newcomers need more support than they did last week. Popular lists get harder to climb.
• An item is added or removed, changing N and therefore the point value of every position.

To make that movement measurable rather than just felt, a snapshot of every public list’s order is captured once a day at 06:00 UTC. Comparing two consecutive snapshots is how the site notices a genuine shift and tells the people who ranked that list about it.

Controversy and contested items

Agreement is a separate question from position, and a good ranking should be honest about it. For each item we take the standard deviation of the positions it was given across ballots. An item everyone puts third or fourth has a small standard deviation. An item that is somebody’s first pick and somebody else’s last has a large one.

A list’s controversy score is the root mean square of those per-item standard deviations, not their plain average. Root mean square weights the genuinely divisive items more heavily, so a list where three entries cause fights does not get flattened into “calm” by twenty entries everyone agrees on. Above 1.2 reads as high controversy, above 0.6 as medium, and below that as broad consensus. The Most Controversial feed is simply this score, sorted.

The most contested item on a list is the one with the highest standard deviation among items with at least three ballots. Below three ballots a standard deviation is noise, not disagreement, so those items are excluded rather than allowed to win a fight they never had. The disagreement percentage shown alongside it compares that spread against the spread you would expect if rankers had placed the item completely at random, which for a list of N items is the square root of (N squared minus 1) divided by 12. One hundred percent means an item is placed as inconsistently as a coin toss.

In the five item example above, the per-item standard deviations are 0.58, 0.96, 0.82, 0.82 and 0.50, giving a controversy score of 0.75, which is medium. The most contested item is B, whose 0.96 spread is 68 percent of the random-placement reference.

The Rising badge

An item is flagged as Rising when three things are true at once: it was added in the last 30 days, its ballot count is still below the confidence threshold m, and the list has at least three distinct rankers. The third condition exists because on a list with one or two rankers every item is under-sampled, so the badge would mean nothing. Rising is a statement about a newcomer, not about a quiet list.

Regional views

When you filter a list by country or region, the entire calculation reruns over that subset of ballots. It is not a re-sort of the global result. R, v, the prior C and the threshold m are all recomputed from the filtered ballots, which is why a regional view can legitimately disagree with the global one instead of just reshuffling it slightly.

What the math deliberately does not do

• No paid placement. There is no input to the ranking function that money can reach.
• No editorial thumb on item order. Nobody, the founder included, can nudge an item up inside a list. Changing a result requires changing ballots. Which lists get featured on the home page is a separate, human, and openly curatorial decision that has no effect on how the items inside any list are ordered.
• No anonymous house votes. Lists published by the site open with one ballot from the founder account, shown on the list itself as a stated point of view with the reasoning behind it. It is one ballot, weighted exactly like yours, and its influence shrinks with every ballot that follows. There are no accounts casting votes that are not real opinions from a real, named account.
• No time decay. A ballot cast a year ago counts exactly as much as one cast this morning. Rankings drift because opinions accumulate, not because old ones are quietly deleted.
• No reputation weighting. XP, follower count and account age have no path into the score. Every ballot counts the same.

Common questions