leaderboard

After the competition ends the private leaderboard is revealed and your standing on this private leaderboard decides your final score.

If the split between public and private is random, and your clustering is objective, then an improvement on public leaderboard should reflect in private leaderboard.

There is a public leaderboard and a private leaderboard.

If this were a Kaggle contest, and there were a leaderboard on how well people are doing on the sample data, you'd have a perfect score on the leaderboard.

The 9th place team likely had a mediocre highly-variant model that got a very lucky leaderboard split.

You're missing the fact that the leaderboard dataset is not made available to the contestants, so they cannot memorize it.

Also, the more advanced Kagglers can use leaderboard feedback to increase model accuracy: Cluster the data sets with objective measures.

If I am reading this correctly, the green numbers to the left of the teams in the private leaderboard represent that team's position relative to the public leaderboard.

What this team did was to submit a lot of predictions and only take the predictions that improved public leaderboard score.

Even with those other methods in place, the leaderboard would still favor methods that overfit.

I restarted a few times and can reliably top the leaderboard in a minute or two.

They work extremely hard to do the impossible: preserve the perceived value of achievements and leaderboard rankings in the face of an untrusted client.

Also, if you do put in a leaderboard system then you should definitely involve time, and have the timer running when the game is in "paused" state (not background, but pressing the "pause" button deliberately).

To give someone a reputation point, all you have to do is type "++ @teammate" and to check reputation points, you just have to type "++ leaderboard." I would love to hear some feedback!