13th Annual Top 25 Pre-Discussion Thread

[Shepardus]

The overall results are partially determined by the number of people voting for each game regardless of weight, so it would be a little different to give a weight of 1 compared to not including a game at all.

[gimebe]

Voting systems used to compile popularity rankings are fun and interesting. Unlike shmups.

I don't see why lesser lists with low-priority picks cut out couldn't be included. It could be treated as a list of 25 with blank votes for the low-priority slots - an example under the current system:

[25] Battle Garegga
[24] Battle Garegga
[23] Battle Garegga
[22] Battle Garegga
[21] Battle Garegga
[20] Battle Garegga
[19] Battle Garegga
[18] Battle Garegga
[17] Battle Garegga
[16] Mushihimesama Futari
[1] This game is not Garegga or Futari
(repeat as necessary)

Trimming the bottom end of one's list would simply turn the less worthy games from [1] (drop in the bucket) to [0] (nothing), which doesn't seem like that big of a difference. Or you could just make a very top-heavy ranking of 25 to state enormous favorites as illustrated above. I don't know the purpose of this specific weighting/ranking system and the specific number 25, but it seems like a working system. A change in numbering in either direction doesn't sound like a big deal since the user has a lot of freedom to weigh and rank as they like under this system already.

disregard that, i suck cocks

Apparently I misunderstood how weighting works in this voting system. An actual explanation of the voting mechanics is found HERE, in the fine print. Quoted:

Out of curiosity, is the ranking established based on the weighed ratings alone, or a mix of ratings and number of nominations?

It's a mix. 2/3 of the score comes from nominations. The remaining 1/3 uses the weighted votes.

If you want more detail, I've provided some nitty gritty below. No offense taken if you'd rather skip it, though.



Final score, the one used in the final rankings, has two components: the popular score and the weighted score.

The popular score of a given game is equal to ballots_including_game / number_of_voters. So if there was a game that was on every voter's ballot, it would get a 1. Usually no game gets very near to this, though. Last year Battle Garegga appeared on 52 ballots (more than any other game), but there were 92 voters. Hence its popular score was 52/92 (roughly 0.57).

I described how a single ballot contributes to the weighted score above, so I'll skip to the next step.

Once all ballots have been processed, the highest weighted score is found and then every weighted score is divided by this (bringing all scores into the range of 0 to 1). So with last year's numbers again, Ketsui had the highest weighted score with 3.161967. Its adjusted weighted score, therefore, was 1. Battle Garegga's weighted score was 2.731759 so its adjusted weighted score was 0.86 and some change.

The final stretch is simple stuff. A game's final score is the popular score doubled + the adjusted weighted score. If a game was on all ballots and had the highest weight, it would get a perfect score of 3. Since 3 doesn't look very impressive for a superb specimen, we multiply the final numbers by 10/3 so the final display results are out of 10.

With Battle Garegga numbers again: 2 * 0.565217 + 0.863943 = 1.994377. Multiply that by 3.333333 and you get 6.647923 .

What it means is that an user's weighting on their list has lower impact than you might think at a glance. With the inclusion of a game on a list having large impact by virtue of the popularity score (how many lists a game was included on) already, the weighing of individual titles is much more limited than I thought - it is not a ranking between 1 and 0.04 in effective vote weight, but rather more between 1 to 0.4 (guesstimate; actual math not included).

My previous post was under the impression that weighing has major weight and allows low-priority votes, when in practice the lower weight limit is much higher than I thought due to the popularity score. Frankly, I think this system is terrible and misleading due to the low range of weighting actually available to the user, with large numbers that suggest otherwise used in lists. Personally I'd like to see it tweaked towards favoring weighted score over popularity score to allow users more room to weigh their rankings, such as 3/10 popularity score + 7/10 weighted score instead of the current 2/3 popularity score + 1/3 weighted split.

[chum]

seconded, I understand the case for number of mentions mattering to an extent, but it matters way too much right now.

mone of my favorites are any popular here so it feels futile to award them lots of points when the other games on my list will just absorb the points for teh final list anyway

[rjosal]

Thirded. It's as I was saying - you don't know the reason a game was left off a list - because it wasn't played or wasn't good. It's probably an ok slightly helpful metric, but it's missing the key piece of data of what games were played. I actually like nameschonvergebens idea of required a posted hi score for games on your list. This could guarantee fair weighting, I think, even if you didn't vote your favorite game because you haven't posted a score because the weight would be there to make up for it. Also it'd be nice to see the hi score lists explode with activity. Actually the reason I've never participated is because I know my list would be wrong. I know so because I haven't played gradius V, I haven't played DBACEX. But gradius II and III would be on my list and I know ACEX would be probably top 5 after playing some of the other versions. I have a Darius cab and I know the cab would put it above all other versions. I've played tons of shmups. Put serious time into most of anything that ever appeared in the top 25. Maybe this year I'll vote, but I don't want to hurt games I haven't played yet.

[trap15]

Fourthed :V

[Nifty]

but rather more between 1 to 0.4 (guesstimate; actual math not included).

This is usually about right actually... again, for a 25-to-1 vote. There's a large amount of wiggle room for those who want to take it, probably more than what upper/lower bound numerical scoring would allow for. If they don't, I think it's still fairly implicit that a [20] or [15] isn't going to be worth a whole lot less than a [25] on a linear ballot, and so on. These aren't best-to-worst lists, they're best-to-slightly-lower-than-best lists and I want to keep something that's going to reflect that. You can decide what you want to do with your vote, as can everyone else, if the majority go linear I'm not going to tell them they're "doing it wrong".

Originally, I suspect the voting was weighted that way due to the need to account for much higher game weighting and partial votes. Even now it's necessary to account for the fact that the top games often get a huge amount of score and that 0.25 is still a very generous amount of weight for a single game. I'd rather err slightly on the cautious side than break something. Having said that, I've debated changing it to, say, 50/50, but it would make such a minor difference it didn't seem worth the bother. Could trial that maybe, with a maximum weight drop to 0.2? Really that's my main concern because I seriously dislike votestacking.

Ah yes, of course we can't make rules that would not allow YOU to vote for your favourite games

You'll pardon me for not expecting everyone to have to drop a score just to be able to say that they can enjoy something in a primarily enjoyment-based poll, nevermind the logistics of needing evidence for every single matching vote.

Actually the reason I've never participated is because I know my list would be wrong. I know so because I haven't played gradius V, I haven't played DBACEX. But gradius II and III would be on my list and I know ACEX would be probably top 5 after playing some of the other versions. I have a Darius cab and I know the cab would put it above all other versions. I've played tons of shmups. Put serious time into most of anything that ever appeared in the top 25. Maybe this year I'll vote, but I don't want to hurt games I haven't played yet.

There's no shame in voting for what you like now and leaving room for change in the future. I enjoy having options to look forward to, helps keep every year a little bit fresh.


These conversations are already starting to repeat themselves so I'm going to state some definites now: The voting system, requirements and minimum number of games aren't going to change. Willing to fiddle slightly with the weighting now that there's some call for it. Very probably adding in HMs. If that's all in order, I'd like to get the voting started this weekend.

[Shepardus]

I'm not a fan of the current weighting either, but I think 50/50 would be a fair compromise.

[pokemon123]

hope that ibara will go farther up. If people are going to make a CAVE shmup the best it should be that! (ketsui is cool too).

Also hoping to see increases in raystorm. I prefer rayforce but raystorm is still a lot of fun. Kyukou tiger? will also hopefully increase. Although it's a wish i hope doj will hopefully one day be knocked out of top 25 :V (sorry skykid), think i have a personal hatred for that game because of the chaining and pretty much making me almost never want to try to score in a game again heh.