[NO PLAYERS] Seeding and ranking discussion

[Bacchus]

I would like to create a separate thread to discuss how we will go about this, as there are a number of issues.

NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS
NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS NO PLAYERS





















The basic task seems simple at first: rank X players, rank X starting locations (starloc), assign them to each other appropriately.

Three problems immediately arise however:

1) Assigning a player to a starloc affects the ranking of the neighbouring starlocs, depending on that player's skill and agressiveness.

2) Variance in player levels might not match the variance in starloc's quality. Maybe the best two starlocs in the game are of very close quality, whilst the third is much worse; whereas the second-ranked player, by general perception, is significantly weaker than the first-ranked player, and quite evenly matched with the third.

3) Rank aggregation. Say there are three lurkers, and they rank players as follows:

Code:

1. Krill      1. Seven     1. Mackoti
2. Seven      2. Mackoti   2. Krill
3. Mackoti    3. Krill     3. Seven

What are we to do with this? This is actually a simple case, as we would probably assign them to the same tier, but what about this one:

Code:

1. Capricorn 1. Gemini    1. Leo
2. Taurus    2. Leo       2. Taurus
3. Gemini    3. Taurus    3. Capricorn
4. Leo       4. Capricorn 4. Gemini

One option is to award points to each place, so the fourth place gets 0 points, third place 1 point, and so on. This method is called Borda count and would give us:

Tier 1: Taurus and Leo with 5 points
Tier 2: Leo and Gemini with 4 points

I think you can see what the problem is: Taurus is not ranked as Tier 1 by anybody, moreover the majority of the lurkers rank Leo above Taurus, but the result is that Taurus is seeded equally as Leo.

There is an even worse problem if we expand the population slightly, by adding Pisces, who suck:

Code:

1. Capricorn 1. Gemini    1. Leo
2. Taurus    2. Leo       2. Taurus
3. Gemini    3. Taurus    3. Capricorn
4. Pisces    4. Pisces    4. Pisces
5. Leo       5. Capricorn 5. Gemini

Now we obtain:
Tier 1: Taurus with 8 points
Tier 2: Leo with 7
Tier 3: Capricorn and Gemini with 6
Tier 4: Pisces with 3

Adding Pisces to the mix moved Taurus up a tier, even though the relative placement of Leo and Taurus hasn't really changed and the majority still ranks Leo above Taurus. Even worse, had Pisces had a different skill level, had it come in at second place in everybody's ranking, instead of fourth, it would have had the opposite effect, making Leo the sole occupant of Tier 1.

So this is naive approach to ranking exhibits behaviors that we don't really want to see, like whether Pisces is playing the game affecting our relative ranking of Leo to Taurus. Things get worse though. There is a mathematical problem at heart here: taking a several orderings of an identical set of elements and coming up with a new ordering that best "reflects" the sum of the existing orderings. Turns out, we can prove that there is no way to do that without violating some apparently simple requirements (like that if the majority prefers option A to option B, our resultant ordering should do so as well). More on that here: http://plato.stanford.edu/entries/arrows-theorem/

So the choice of how we are going to aggregate rankings is also an important one, as we will necessarily be giving something up. In any case, this choice needs to be made in conjunction with the prior two. A bit later I will post my thoughts on how to best approach the process.

In the meantime, if others notice yet more difficulties lurkers face in our enterprise, now is the time to point them out!

[spacetyrantxenu]

I assume this should be a [NO PLAYERS] thread, since they don't need to know how the sausage gets made. Can you edit that into the first post, Bacchus?

Your explanation of the problem is interesting, as I am sure will be your proposed mathematical solutions. But shouldn't we get our tier lists first and see if we have a problem to solve before trying to solve it? If it ends up that the lists ranking the players aren't all that different from one another this exercise could be much simpler than you're setting it up to be.

Also, this is setting up to be the best thread ever. An open solicitation of player tier lists? YES PLEASE.

After we sort out the correct tiers for the people playing in this game we can branch out and then rank each other, and then everyone who has ever participated in a game at Realms Beyond. OK class, discuss!

[spacetyrantxenu]

Player Roster:
- Old Harry (with occasional chirping from Fintourist)
- Molach
- Ipecac
- Gavagai
- Dantski and V8mark
- TheWannabe
- Greywolf
- Alhazard
- ReallyEvilMuffin
- 2metraninja

[Dreylin]

Aha, I was just in the process of doing that! I guess that answers my question about whose paper I'm going to copy...

[wetbandit]

Also, this is setting up to be the best thread ever. An open solicitation of player tier lists? YES PLEASE.

Plus addition of complicated voting mechanism!

[Mardoc]

Hmm, spoilers are going to be an issue. Most of the best evidence for player skill is the games that are currently ongoing.

Also, Bacchus, do you have a good way to handle abstentions? I know some of the players much better than others; if I give a list of seven players in order and three as 'not sure' - does that work?

[AdrienIer]

Or should we all give a grade out of 10 for each player ? And then look at the average ? That way if you don't know a player's skill you just don't grade him and you don't appear in the average.

[Mardoc]

Well, someone's got to put forth a hastily scribbled opinion for everyone to disagree with. Might as well be me.

1. Old Harry (with occasional chirping from Fintourist)
Either won or been a contender in every game he's played, IIRC. He's the hard work end of the pair, with Fintourist somewhat better at grand strategy - so 'occasional' Fintourist won't hurt as much as if it were the other way around.
2. Gavagai
Solid play, but seems a little too much war not enough peace, he seems to fall off late game. Either that, or he's just unlucky. OTOH, he's very good at tactics and relentless, both of which ought to help in a weaker field.
3. Alhazard
Couldn't quite keep up with TheHumanHydra in PBEM 65, but he was pretty close.
4. 2metraninja
Definitely understands the mechanics and whatnot, just seems to fall down a bit on micromanagement.
5. Ipecac
Skimmed PBEM 66, he seems roughly on par with Alhazard or a little below. Less resilient morale-wise.
6. Molach
Not sure, could be higher (below Gavagai). Terse reporter. Feels generally middling.
7. Greywolf
FFH instincts will probably cause him some trouble. Mid-tier among the FFH crowd, but we tend to be weaker than the BtS'ers.
8. TheWannabe
PB29 spoilers

Seen him build cities without first ring resources. On the other hand, he does build cities and knows what workers are for

Don't know:
- ReallyEvilMuffin
I vaguely think he places well, but doesn't report?
- Dantski and V8mark
He hasn't played in a while, IIRC. I think he was pretty decent back then. Has meta changed in any respect that would matter?

Gun to my head, I'd put REM and Dantski at 4 and 5, between Alhazard and 2metra. But I'm even less confident in ranking them than in ranking the rest of the field .

[Mardoc]

Your explanation of the problem is interesting, as I am sure will be your proposed mathematical solutions. But shouldn't we get our tier lists first and see if we have a problem to solve before trying to solve it? If it ends up that the lists ranking the players aren't all that different from one another this exercise could be much simpler than you're setting it up to be.

Heh, the difference between a mathematical and an engineering approach. 'Can we solve the problem in general, and then leave the specific case as an exercise for the student?' vs. 'Can we get something good enough working now?'

[GermanJoey]

Or should we all give a grade out of 10 for each player ? And then look at the average ? That way if you don't know a player's skill you just don't grade him and you don't appear in the average.

I like this idea. Also, we can square the individual ratings before we sum them if we find they're not very differentiated at the end.

Maybe before we rank, we could do rundowns on each of the players, link to some notable games for each player? A few of these guys (Greywolf, Dantski, and V8mark) I don't know very well.