In brief – I propose a simple mechanism for discounting the rank of players who have not played many games compared to everyone else. I outlined a model for this in TheThrill's thread where he bitched about the ranking system.
The essential argument is that people who play few games have an advantage in the current ranking system - and a strong disadvantage to play more games once they take the top spot. Essentially, they may get lucky and win just enough games to get themselves ranked at the top, but were they to play more games, their true, lower win % would come out and they would cede the top spot.
An ideal ranking system would do the following:
1) Allow for rank lists that span several different time periods (daily, weekly, monthly, yearly, all time ...)
2) Take into account the quality of the player you defeat when adjusting your rank
3) Recognize that the more games you play, the more accurate your rank metric will be (less error) by incorporating this into the rank metric!
ELO can do #2 really well, but fails to do #3. It takes many games to arrive at an accurate ELO score for a player, therefore it is good for long term rank lists, but not short term ones where few games are played.
My proposal does not do #2 at all, but it could later be combined with ELO to tackle all three.
In brief, it assigns an “error” or penalty to your win % (our current rank metric) that is based on the number of games you have played and the number of games the x-most prolific players have played.
The properties of the error function are such that the penalty to your rank score is greater during your first few games and shrinks as you play more AND this penalty is higher if most players have played more games than you and becomes negligible if you have played more games than most.
I tweaked the error function to be a little less drastic than the one I listed in the prior thread. The only variables involved are the number of wins (wins
), number of games played (games
), and mean number of games played by the top x-number of players (meangames
). Here is the revised formula for the Adjusted Rank Score (ARS):
Increase the value of constant I set at 4 and the penalty becomes less harsh. I think somewhere between 3-5 is ideal.
Sticking with the value of 4, here's a graph that demonstrates its effects:
This graph shows you how the Adjusted Rank Score works. In this example, a player plays his first game when the average number of games played by everyone else is 100. This new player wins every other game after losing his first and plays a total of 100 games. His win % converges on 50% (blue line on graph). The green line shows his Adjusted Rank Score – the metric that would be used to rank him on the leaderboard. After his first win, his ARS is 26%, just over half of his 50% wins. This is low because this player has played very few games compared to the mean. As he plays more games, his ARS grows closer and closer to his win %. By the time he plays 100 games, the fraction of his win % that is discounted by the error function is a mere 1.8%. The red line shows the number of percentage points discounted from the win % by the error function. The purple line shows what fraction of the win % this represents. You can see from the graph that when a player has completed half as many games as everyone else, his win % is discounted 12%. A win % of 80%, for example, would be adjusted to 70.4%.
In actuality, most players complete their first few games at the start of each month. Since the mean number of games at this point is low, everyone’s ARS will be close to their win %. If they keep up with everyone else, this will be true for the rest of the month. Those players that lag behind will see a penalty applied to their rank score that grows as they fall further behind. Playing twice as many games as the mean only buys you another 2% increase in your rank score (from 50% to 51%, for example).
Here’s what would happen to the leaderboard (as it stood at 3:50 pm on this last day of February) if we applied this rank metric using the mean number of games played by the top 13 players:
TheThrill would come out on top. Rhox, Fluffballs, and tony see almost no difference between their win % and ARS since they have played many more games than the mean. I fall to fourth place. Respectable, but appropriate given that I have played few games. If I really wanted to claw my way to the top, I’d have to play a lot more. As I play more games, each prior win counts more since my penalty decreases. Instead of rank sitting, I now have incentive to play more games. If I don’t, I see my score and rank erode further as everyone else plays more.
I also have a simple way of melding short term rank lists with the more long term ELO-based ranking system, but I can talk about that later.