5th Annual Top 25 Shmups of All Time! - Results

[The Coop]

I didn't take part in the previous list, because I didn't understand what the hell was going on

All I saw were listed with seemingly random numbers slapped on to various games. This year, I came to grips with what was going on, and took part. However, I too was wondering why people didn't just vote 25=#1, to 1=#25. All this 1000 and 100 stuff made little sense, and seemed to be counter productive. A simplified 25 points -> 1 point system would seem to be the easiest to implement, and still give the kind of results this list is after. I mean, who here hasn't played 25 shumps yet (besides the new folks)?

Would seemingly make tallying everything a lot easier too.

[szycag]

One argument against giving one game the lion's share of your voting power is that the numbers look all weird and it intimidates potential voters who might not see how the pie works.

To be critical, how much weight would you give someone's opinion that put everything behind one game then 1 point to everything else? Doesn't sound like someone that's played a lot of shmups. Same for the people that give out a 3, 12 2's, and 10 1's. They don't feel more strongly about those games to give them more than just a middle of the road vote to each? No offense meant to anyone who voted like that. But I don't understand people who think like, hmm I really like this game I play it all the time, and these are pretty good, these are just OK... hopefully my .1% of my voting power I'm giving it will get it up on the chart (not likely)

[j^aws]

This from the rules:

02) For each shmup, allocate an integer number of points. The points you assign will help compensate for, say, games that fewer people vote for but feel much more strongly about than others they vote for.)

It's mainly intended to compensate for FEW votes: Exactly why I used it, though it would seem not applicable for popular votes... e.g. DDP et al...

[Icarus]

Next year, if we're running the same points tally system, I'm going to stick 10⁹⁹ on D-Force and see where it ends up.
Then we'll see if the system is broken or not.

[j^aws]

Next year, if we're running the same points tally system, I'm going to stick 10?? on D-Force and see where it ends up.
Then we'll see if the system is broken or not.

If you HONESTLY feel that then go ahead. I don't see why not. But the system is based on HONESTY... so somehow, I doubt you feel that strongly about that game...

[Icarus]

If you HONESTLY feel that then go ahead. I don't see why not. But the system is based on HONESTY... so somehow, I doubt you feel that strongly about that game...

The system might be based on honesty, but there is something to be said about the validity of the results when one person can throw a huge amount behind their vote and bump it up into the honorable mentions group. Honorable mentions are fine, but what if it ended up breaking into the top 25 based on that overweighted vote? Do you honestly think that would be an accurate representation of the opinion on this board?

That is why we're having this little discussion about the way points are tallied as they are. There are ways of unbalancing the vote, and I personally don't think the current vote system provides a completely accurate method of recording opinion. Personally, and I said it before, a more fair method of vote tallying would provide a more accurate display of opinion - games that get more votes will be in the top 25 as they deserve, and not because someone threw ten billion points on one vote. A more fair voting system would encourage people to actually put thought into their top 25 and how to order their list.

This is the first time I can recall the vote system being discussed in this much depth, which says something.

[Rob]

But the system is based on HONESTY...

Is it? I voted strategically,* which I think the current point system encourages. And giving 99% of a vote to a game seems the most strategic move.

*didn't work out, Mushi and ESP Ra.De. still there

[jp]

My opinion, is that having a method of voting that makes it to where we should be "playing the system", is a bit silly. I mean, I know this is a shmups message board, and messing with scores is what we do, but to me it just feels like such a scoring method really only caters to fanboys of certain games and no one else.

I mean, I COULD have walked up and thrown down 19264789364 points on Radiant Silvergun and 3983329 points on Blast Wind and then let everything else be 10 pts and down, but... is that really neccessary? Do I really love Radiant Silvergun THAT much more than every other game? Almost makes me feel like I'm not making a list for my top 25 shmups, but rather making a list so that my favorite game can be on the top 25 list and I can throw a bunch of other shmups I like out there as well. Which in my opinion rather defeats the purpose.

[j^aws]

...
The system might be based on honesty, but there is something to be said about the validity of the results when one person can throw a huge amount behind their vote and bump it up into the honorable mentions group. Honorable mentions are fine, but what if it ended up breaking into the top 25 based on that overweighted vote? Do you honestly think that would be an accurate representation of the opinion on this board?

Taking your 10^99 example:

as integer -> infinity ; weighting -> 100 %

...This is capped for 25 games. Stargate will hover where it is now without FURTHER votes.

This is where people are missing the point: Out of obscurity, this game has gotten exposure; and if OTHERS feel this game deserves a vote in subsequent years; it's a Good Thing (TM) and it will rank higher...

That is why we're having this little discussion about the way points are tallied as they are. There are ways of unbalancing the vote, and I personally don't think the current vote system provides a completely accurate method of recording opinion. Personally, and I said it before, a more fair method of vote tallying would provide a more accurate display of opinion - games that get more votes will be in the top 25 as they deserve, and not because someone threw ten billion points on one vote.

There is no perfect system. The rules mention this.

The current system allows for your scenario of listing 25, 24...1. But your above system doesn't allow for my aforementioned scenario for an obscure game. The current system is simple yet flexible.

Whatever the final list is, someone will disagree with it...

[j^aws]

But the system is based on HONESTY...

Is it? I voted strategically,* which I think the current point system encourages. And giving 99% of a vote to a game seems the most strategic move.

*didn't work out, Mushi and ESP Ra.De. still there

Well, I honestly believe Stargate deserves to be higher than where it was: Obscurity. So, yes, I also voted strategically, but at the expense of R-Type et al; but that was my call, and I was happy to do that, honestly...

[jp]

I agree with Icarus. These lists aren't showing off a general shmups.com Top 25 Shmups, rather, they're showing a shmups.com Which 25 Shmups Have the Most Rabid Following.

[Edge]

Despite from the Stargate-drama, I think it would maybe good to work a bit on the system the poll works. I stated that last year. And therefore I would like to suggest another solution and maybe start a constuctive discussion about renewing things.


So in order to not confuse people too much, we can have the same voting system and just change the way it is counted.

I think if simply every listed games get a position/points assigned independant scoreboost for appearing on a list.

...for exemple like this:
Total share of one person points 10'000 (or enter other random number)
Every time a game makes it on a list +10'000

So if someone votes like this:

[500'000] [Lame Game]
[300] [Halo 2]
[150] [Pocco Rosso]


points would be calculated like this:

[500'000] [Lame Game]
-share of his point pie: 9991
-shortlisted game bonus: 10000
=19991

[300] [Halo 2]
share of his point pie: 6
shortlisted game bonus: 10000
=10006

[150] [Pocco Rosso]
share of his point pie: 3
shortlisted game bonus: 10000
=10003

This way huge point gaps can't unbalance the scoring anymore. Because even if someone puts his whole voting power behind one single game, he can't overcome 2 voters, no matter how small of their pie they gave the game.

So the points would be more used as a tiebreaker. Alternatively we could change the points assigned to rule to a system where people only list games and positions and are allowed to put several games on the same position if they can't decide. But if the points will make a small impact, it wouldn't be really neccessary.


And offcourse Zach Keene, I think no ones in here doubts that you are and have always done a great job with this poll. The contoversy caused in this very thead shows how much this list has grown on the users how much they bother for it's outcome. So we hope you will continue the great work.

[Ceph]

Much too complicated.

I'm for the "25-1" method, which I think would work VERY nicely.

The next best thing (imo) would be giving each voter 50 points to distibute among his/her 25 games, with each game on a list requiring at least one point and the maximum number of points allowed being 10. This would also lead to somewhat balanced results and is easy to understand.

Both systems would require everyone to vote for exactly 25 games; having more than 1 or 2 invalid games on a list would simply invalidate the whole list (in order to prevent people from assigning all valid points to only 1 or 2 games).

[God]

Honorable mentions are fine, but what if it ended up breaking into the top 25 based on that overweighted vote?

I think the only ranking games that got this treatment were Raiden DX (7th place in the weighted list, 13th or 14th in the unweighted) and Gunbird 2 (16th place weighted, 23rd or 24th unweighted). Thunderforce 3 got half a pie (12th place weighted, 23rd or 24th unweighted).

[DEL]

God hath spoken

[God]

I hope you take some hours more and post a similar recount for 2004 and 2005.

http://shmups.system11.org/viewtopic.php?t=584 (3rd annual)
http://shmups.system11.org/viewtopic.php?t=7506 (4th annual)

In 2005 Gigawing got a whole pie vote and skyrocketed to 11th place. Every other year its come in at spots 18th to 20th.

All 3 years its been on the chart Raiden DX had a whole pie vote propping it up.

[Michaelm]

The system is broken because it IS possible to influence the final list by a single voter.

Every game 1 point will not have this flaw.
Giving 25 games a point between 25 and 1 is way too much for me.
I'd like to give my vote but I don't like needing to think about it for that long.

I'm really not interested to see a list that has been influenced a lot by a few voters who calculated how they had to vote to get what they wanted.

[Turrican]

The system might be based on honesty, but there is something to be said about the validity of the results when one person can throw a huge amount behind their vote and bump it up into the honorable mentions group. Honorable mentions are fine, but what if it ended up breaking into the top 25 based on that overweighted vote? Do you honestly think that would be an accurate representation of the opinion on this board?

Yes, since everyone who votes does the same, to an extent.

Personally, and I said it before, a more fair method of vote tallying would provide a more accurate display of opinion - games that get more votes will be in the top 25 as they deserve, and not because someone threw ten billion points on one vote.

You understand that every vote has the same weight as long as you vote for exactly 25 shmups, don't you? Your "25 pie" weights exactly like all the others, if you put ten billions on a single title it basically means that you're "wasting" your entire pie to promote that single game. Once you understand this, it's pretty fair because you can push a game if you really want to, but at the expense of all the others. And no matter how many points you assign to it, it will never exceed the total weight of your pie.

The next best thing (imo) would be giving each voter 50 points to distibute among his/her 25 games, with each game on a list requiring at least one point and the maximum number of points allowed being 10. This would also lead to somewhat balanced results and is easy to understand.

It gets pretty similar to the actual system don't you think? Still, I agree that a set number of points would prevent silly numerological oddities...

Both systems would require everyone to vote for exactly 25 games; having more than 1 or 2 invalid games on a list would simply invalidate the whole list (in order to prevent people from assigning all valid points to only 1 or 2 games).

The actual voting system allows for more freedom - you can vote for hundreds of games as long as you do understand that you're just scattering your "pie" (which is the same as everyone else) into fragments. Why change this? If one cannot limit himself to 25, or doesn't want to list up to 25 titles, where's the problem.

[Michaelm]

I will not be voting anymore if the rules don't change.
I see absolutely no meaning in voting if someone just waits till short before the deadline to calculate how he has to vote to get his favorite game in the list.
This ain't a top 25 list of the shmups.com forum.
This is a top 25 list of the few who calculate their games in the top 25.

And the worst thing is that those people don't feel ashamed that they come out saying they do it.
So this list is only a competition between those few abusers of the current voting system.

The point system is there to give your favorite game more points then the other games you've listed. That's the point ! Not giving weight to one particular game.
But because it has been acknowledged that people tend to misuse this I think this is a clear call to change the voting system next year.

Or maybe I will participate next year to get Last Hope in the list

[doctorx0079]

I think the conflict here is between those who basically are only voting for one game, and those who think everyone should have to vote for at least 25. J^aws essentially voted for one game instead of 25, and that game got in the honorable mentions because that game is so important to him. It is like all other games have no value to him for purposes of this poll (although he did give some games a little credit by naming them). The people who don't like this are taking a position that amounts to: he should have to choose at least 25 games worthy of his vote. But the position of the poll organizers is that you don't have to vote for that many if you don't want to, and if you only think one game in history is worthy of your vote, you can just vote for that without penalty.

[Rob]

But the position of the poll organizers is that you don't have to vote for that many if you don't want to, and if you only think one game in history is worthy of your vote, you can just vote for that without penalty.

IMHO: if a person thinks only one game is worthy of real votes (as opposed to 1/1,000,000th of a vote) then skip adding so much filler.

Example:

[1] [only favorite here]

Of course I add about 10-15 filler games.

[WarpZone]

I see absolutely no meaning in voting if someone just waits till short before the deadline to calculate how he has to vote to get his favorite game in the list.
This ain't a top 25 list of the shmups.com forum.
This is a top 25 list of the few who calculate their games in the top 25.

This doesn't require changing the system - ballots could just be PM'd rather than made public.

[jp]

But the position of the poll organizers is that you don't have to vote for that many if you don't want to, and if you only think one game in history is worthy of your vote, you can just vote for that without penalty.

IMHO: if a person thinks only one game is worthy of real votes (as opposed to 1/1,000,000th of a vote) then skip adding so much filler.

Example:

[1] [only favorite here]

Of course I add about 10-15 filler games.

I don't see why anyone who frequents this site would vote for just one game for any reason other than to affect the outcome of the list. Seriously, if you only like one game in the genre, me thinks you aren't really much of a fan of the genre to begin with and probably shouldn't have much say in a top 25 list on said genre.

[j^aws]

Stargate has pedigree. I've been playing shmups for nearly 30 years; probably before a fair few were born in this poll. The game is very HARD, very COMPLEX and very DIFFERENT. And I LOVE it. It's also my favourite game of all time. This is my opinion. Deal with it.

If anything is at fault, I'd say voters who can't READ the CURRENT rules, nevermind UNDERSTAND the rules. Especially when a SIMPLE rule as SQUARE brackets around each vote can't be followed correctly ( ~ 15% voters, not all 1st timers either and guess the culprits?)... so whether ALL the voters 'qualified' for voting would be just as bigger concern to the accuracy of this ALL-TIME poll, if not more so, than just a few single weighted shmups...

Also:

As qualified voters -> infinity; relative 'pie' to all other voters -> 0 %

...Obviously we won't get infinite voters; but the poll is a 'snapshot' and it's a long term, evolving event. The more qualified voters participate, the less influence one 'pie' gets... which seems to be forgotten, if understood at all...

For me, this poll is as much as the TOP 25 as it is knowledge sharing about all voted shmups. I usually look through the whole list of 300+ games and perhaps find a gem I wasn't aware of.

Like I mentioned earlier; the 1st annual poll was akin to a 'seed'; it's 'gene pool' was the 1st list... not just the Top 25 but ALL the voted shmups. The current rules stipulate ~ 50-65% familiarity with OTHER voters shmups... this encourages an iterative process over subsequent years to filter out shmups that aren't deemed 'worthy' by the qualified voters... and the 'worthy' games rise to the top...

... And this is a 'means' to an 'end'... and I STILL see the final list as being FINE...

[Michaelm]

If anything is at fault, I'd say voters who can't READ the CURRENT rules, nevermind UNDERSTAND the rules.

Did you know that this is the basis most lawyers operate on.
Your attitude sucks big time in my opinion.

[j^aws]

If anything is at fault, I'd say voters who can't READ the CURRENT rules, nevermind UNDERSTAND the rules.

Did you know that this is the basis most lawyers operate on.
Your attitude sucks big time in my opinion.

Everyone had an opportunity to read the rules before voting and also had an opportunity to voice any concerns BEFORE the results. Following the rules and skimming previous years voters weighting would've highlighted said weightings...

Also, stealth trolling other voters opinions and making 'ultimatums' is an attitude that we could do without...

[Michaelm]

Also, stealth trolling other voters opinions and making 'ultimatums' is an attitude that we could do without...

I'm not stealth trolling. I compared you with lawyers, no stealth there.
And where did I put an ultimatum ?!?
Do you honestly think anyone is going to be bothered if I didn't vote anymore ?!?
I just said I wouldn't vote anymore if the system doesn't change because the system allows for people like you to alter the outcome of the list.

But thank you ! I'm honored that you believe my opinion means that much !

[j^aws]

I'm not stealth trolling. I compared you with lawyers, no stealth there.

I was referring to your previous post:

... And the worst thing is that those people don't feel ashamed that they come out saying they do it...

...I don't have ANY shame whatsoever with my opinion.

Also, I've already been called a "tard" and a "whacko" and the 'worth' of my opinion 'questioned' by others...

And where did I put an ultimatum ?!?

Here:

I will not be voting anymore if the rules don't change.

--------------

Do you honestly think anyone is going to be bothered if I didn't vote anymore ?!?

Wouldn't really bother me. What's important for the current system to work are qualified voters... and trying to vote as 'honestly' as possible... These are key, IMHO.

I just said I wouldn't vote anymore if the system doesn't change because the system allows for people like you to alter the outcome of the list.

That's an "ultimatum" i.e. NOT voting IF the system doesn't CHANGE... you're making a final proposition after making several posts about your opinion on the current system.

But thank you ! I'm honored that you believe my opinion means that much !

Your opinion is 'worth' the same as any other qualified voters opinion that voted for 25 shmups: Any less, the current system adjusts the 'weighting' of your opinion... or 'pie'; which is fine for what the system is trying to 'capture', IMHO.

[Michaelm]

... And the worst thing is that those people don't feel ashamed that they come out saying they do it...

...I don't have ANY shame whatsoever with my opinion.

You should though !

And where did I put an ultimatum ?!?

Here:

I will not be voting anymore if the rules don't change.

Thanks again for giving my words so much worth !!

What's important for the current system to work are qualified voters... and trying to vote as 'honestly' as possible... These are key, IMHO.

Then why aren't you ?!?
Why do you misuse the voting system to get your one game on the list ?
Is that honesty ? You had to calculate and stuff. You call that honesty ?

I just said I wouldn't vote anymore if the system doesn't change because the system allows for people like you to alter the outcome of the list.

That's an "ultimatum" i.e. NOT voting IF the system doesn't CHANGE... you're making a final proposition after making several posts about your opinion on the current system.

And thanks again for giving my words so much worth.
I'm not that asshole Bush with veto power if you didn't know already.

If you're trying to imply that I'm not allowed to use certain words then please say so directly !
As far as I'm concerned it's people like you who make the current voting system a bad system.
You should feel ashamed that your actions affect this whole community.

[j^aws]

You should feel ashamed that your actions affect this whole community.

Right. As does every other members actions.

- "Shameful" is your opinion.

- "Misuse" is your opinion.

- "Bad" system is your opinion.

No need to "calculate" 'coz arbitrary large number -> 100 % weighting; doesn't take much thought...

The final list is fine. Deal with it.