Discussion about the 3rd Annual Top 25 Shmups *Voting*

[nullstar]

I'll post the results for both 2003 and 2002 shortly, in case anyone would like a refresher.

I've got an explanation of the scoring system floating around somewhere, too, so LMK anyone if you want to understand the mechanics of it all before casting your vote.

[Thunder Force]

Thanks nullstar! These Annual Top 25s have become one of Shmups Forum's signature events, and I look forward to another interesting set of results from this year's voting.

Hopefully the new Vote Format will be easy for people to use and for results to be generated from. It might be handy to post the explanation of the (complex) scoring system, since we've seen a lot of new people join the boards since the last time who might want to know how this works.

LATER EDIT: This thread used to be part of the main voting topic, but it's now seperated out to enable discussions to happen without flooding the main voting thread.

[Turrican]

07) No Borderliners! Stuff that's off-limits generally include: "run-and-gun" (think Contra or Mercs); arena shooters (Robotron, Smash TV, etc.); on-rails/into-the-screen shooters (Rez, Panzer Dragoon, Space Harrier, etc.); tube shooters (Tempest, Gyruss, etc.).

Nullstar, I see some serious problem here. We should get back to that "what is a shmup" debate we had.

Tube shmups are shmups by any means. the ship is on the same plane as the enemies, the difference with Defender / Fantasy Zone is none.

So what constitutes "widely and deeply experienced"? That's impossible to truly quantify, but as a general guideline: voters should be familiar with nearly all (95+%) of the games on previous years' top 25 lists. Furthermore, they should be equally familiar with nearly as many (90+%) of the shmups on previous years' honorable mention lists. And, to the extent that other games voters have listed are available, would-be voters should be familiar with 50%-65+% of those games, as well.

Don't you feel this consolidates a list of excellent titles, making it harder for people to vote more obscure games?

Also, The "25 best of all time" is getting a bit vague. Shouldn't we consider a list of "Shmups Essentials"? I'm referring to the 1up.com feature...

[Berty]

Nullstar, I see some serious problem here. We should get back to that "what is a shmup" debate we had.

Tube shmups are shmups by any means. the ship is on the same plane as the enemies, the difference with Defender / Fantasy Zone is none.

Anything where the primary mode of fire moves along the Z axis i dont think should be considered a shmup, afterall Z axis movement puts the game is the same category as Space Harrier.

[Turrican]

There's no Z axis in Tempest.

[Anarchos]

Will asteroids/space invader and the likes be recognized as shmups?

[Turrican]

Will asteroids/space invader and the likes be recognized as shmups?

That's a touchy subject. Those two are actually more off limits than Tempest for sure. I think to remember Nullstair classified them as pre-shmups, due to scrolling lack.

[nullstar]

Will asteroids/space invader and the likes be recognized as shmups?

Yes and no -- SI is a yes. Asteroids I would consider an arena shooter for these purposes. That said, if you (or anyone) wants to vote for something you think might get classified a borderliner, PM me. We've got six+ weeks to resolve everything and will do our best to make sure you get your full votes. In your case: if you want to vote for Asteroids, I need to be able to track that so you can modify your vote if we later decide to allow it...

Tube shmups are shmups by any means. the ship is on the same plane as the enemies, the difference with Defender / Fantasy Zone is none.

I agree with you on the geometry of the plane, and will point out that there are two differences from FZ/Defender, but this thread isn't the place to debate this issue. (But I don't want to blow you off, so please accept the following as explanation for the time being: There are two issues here.

First, the differences w/ FZ and Defender: (A) Tempest et al only offers one degree of movement, while the other games offer two. (Note that I'm not saying this is the reason why the game isn't categorized as a shmup -- we're allowing Space Invaders, after all.) (B) FZ and D present the game as if it were a plane (even though it wraps around) -- and based on the countless discussions held on this forum on this topic over the years, I'd say you'll find as many people who say the perspective on Tempest-like games is so out-of-the-ordinary that it changes the game as you will people who claim that only the geometry matters. I assume you disagree, and that's fine, but I'm telling you what I've concluded based on the myriad discussions we've had about this. Can I verify or prove that opinion? no. And as I said, I'm sure you'll find many to agree with you.

Second (and the reason why none of that presently matters), the admin team has decreed multiple times that Tempest et al are not shmups, and -- unless and until they change their minds -- I'm going to respect that decision. (For what it's worth, they have been debating the Tempest-like issue in recent weeks, so it is possible there will be a change of stance before voting concludes. I wouldn't delay your vote based on that, however -- we can fix it later if it becomes a reality.)

As I said, this thread is not the place to discuss this -- I know a lot of people disagree with that categorization and want to provide you a "real" answer.

[nullstar]

So what constitutes "widely and deeply experienced"? That's impossible to truly quantify, but as a general guideline: voters should be familiar with nearly all (95+%) of the games on previous years' top 25 lists. Furthermore, they should be equally familiar with nearly as many (90+%) of the shmups on previous years' honorable mention lists. And, to the extent that other games voters have listed are available, would-be voters should be familiar with 50%-65+% of those games, as well.

Don't you feel this consolidates a list of excellent titles, making it harder for people to vote more obscure games?

They're free to vote on them -- but consider that something like 50% of the games people vote for (200+ last year) only get one vote. Also, last year there were only three (!) games that received votes from more than half of the voters. If you're saying that people should be familiar with a lot more than just the bulk of what's on the lists, I agree, but I'm also not convinced the list is getting inbred. If that changes, we'll work to fix that.

Also, The "25 best of all time" is getting a bit vague. Shouldn't we consider a list of "Shmups Essentials"? I'm referring to the 1up.com feature...

To be honest, I like the fuzzy logic of letting people define what they think is deserving without being constrained by pegging the list into a particular box. If we get more specific -- like most influential, most essential, best ever -- I think it really restricts people...and also necessitates a separate poll for each, which I think would get old quickly. (Note that I am not opposed to someone else running a more specific poll. But I think the vagueries of "top" are a benefit in this case.)

[jp]

Quick question:


Could you explain the "scoring" to me? Like, giving points based on where you ranked the games makes sense, and initially I thought that was what you were doing. But seeing say, Gradius III, getting 801 points leaves me wondering whats to stop ME from giving Radiant Silvergun 99999999999999999999999999999 points.


Sorry, its early.

[benstylus]

It seems that they're giving each person a certain weight, and your points will be converted into a percentage of your vote's weight.

So for example, if you gave Gradius 10 points and Radiant Silvergun 1000 points, your vote would be .99% for gradius and 99.01% for Radiant Silvergun.

I don't think the points matter except to determine how you want to place your vote.

[Bydobasher]

The key is that everyone gets a pie, and everyone's pie is exactly the same size as long as you vote for 25 games. If you vote for less than 25 games, then your pie is smaller. However, if you vote for more than 25 games, your pie doesn't get any larger, so you will be "diluting" your vote.

Let's pretend you do vote for 25 games. If you give RS 10 million points, and the other 24 games a point apiece, then you are basically consigning your entire pie to RS. The points you have assigned to the other 24 games are almost meaningless.

On the other hand, let's say you give 10 million points to every game on the list. Doing so would be exactly the same as giving 1 point to every game on your list.

[dboeren]

Thanks for your explaination, now the pie and numbers make some sense.

The only remaining issue to me is the version splitting. How different is different enough to split the game into versions? For instance, I could argue that a lot of the 80's to early 90's arcade shmups had to be cut down to fit on the consoles of the day, and therefore they were separate versions. Not intentially so, they TRIED to keep the game the same, but it's still not the same thing.

Perhaps the moderators of the vote can recombine votes for games they they feel do not need to be split. Then we can vote for specific platforms without worrying about diluting the vote for our favorite games. We get more data that way too.

[alpha5099]

The key is that everyone gets a pie, and everyone's pie is exactly the same size as long as you vote for 25 games. If you vote for less than 25 games, then your pie is smaller. However, if you vote for more than 25 games, your pie doesn't get any larger, so you will be "diluting" your vote.

Um ... this is probably an odd question, but could I request a smaller pie? I've been thinking about this for a few weeks now, and I've got a decent idea of what I'd vote for, but I'm not sure I could get a list down to, say, 15. But I also know that I'm not nearly experienced enough, not by nullstar's criteria. I've played almost all of the games in last years Top 25 (whether or not I came to be as familiar as is required, I'm not sure) but I've only played about half of the Honorable Mentions, and probably only a slim number of the total number of games voted for.

So, could I get a pie for someone who voted for 15 games (or some other number, 15 is just what I voted for last year), but dilute it by voting for 25? I just know I'm not qualified enough to be casting the same kind of vote as some of the major players around here.

[Anarchos]

To be honest I do not like the whole idea of having a cake you have to cut yourself, if ya know what I mean!
Wouldn't it be better just basically giving your favourite shmup 25 points, second one 24 and so on?

PS: I wonder who will be brave enough to start out?

[Bydobasher]

Um ... this is probably an odd question, but could I request a smaller pie? ... I just know I'm not qualified enough to be casting the same kind of vote as some of the major players around here.

Obviously, nullstar will decide one way or the other, but it certainly sounds to me like you're entitled to the same pie as anyone else.

Wouldn't it be better just basically giving your favourite shmup 25 points, second one 24 and so on?

There's nothing stopping you from doing so if that's what you want. Even simpler though would be to divide the pie equally by giving every game 1 point.

[TWITCHDOCTOR]

Damn...just read through the "Rules", and now I'm thinking, why bother, its too complicated!!! Seriously, I think doing my taxes would be easier!!
Basically when its all said and done, your top 3 (voted) in each catagory of vertical and horizontal will read something like this...

Top 3 vertical:
DoDonPachi
Raiden
Radiant SilverGun

Top 3 horizontal:
ThunderForce III
R-Type
Gradius V

[BulletMagnet]

Quick question: are doujin shmups applicable, or is this vote for "official" releases only?

[system11]

If games which are obviously shmups (note: shmup = shoot em up) such as Tempest, Black Widow, Gyruss, Asteroids and Robotron are to be disallowed, I'm not going to bother voting.

They're the fathers of modern shooters. Next you'll ban Space Invaders for not scrolling.

[Tar-Palantir]

nullstar: Don't you think the scores should be normalised for each poster?

[OptimusPrimeX]

Damn...just read through the "Rules", and now I'm thinking, why bother, its too complicated!!! Seriously, I think doing my taxes would be easier!!

lmao....

"GOSH"..."idiot".... As i gather my top 25 this whole scoring system is really racking my brain. Maybe it would be a better idea to get down voting + scoring system down now for future reference, or is it relaly already set? i'm confuzored. So does it really matter at all how many points we put next to each game?

[Leeram]

I think the pie system makes sense. Just split your vote straight down the line by assigning 10 points to every game, so that means you think every game is the same in terms of its "toppness"

Then think about which games are the best and by how much, then re-allcoate the points.

So if you think game X is far better than all the rest in your list then give it 20 points. It then gets more of your pie than the rest so, is the better game.

It does take some thinking about, which is the whole point of this list I guess. I reckon it's going to take me days to fully decide on my complete list of votes.

Cheers

Leeram

[nullstar]

nullstar: Don't you think the scores should be normalised for each poster?

They will be normalized for each voter (assuming they vote for a full 25 -- vote for less, and your pie is smaller by (games_voted_for/25)^2....which reminds me:

About the Scoring:
The scoring mechanism was modified slightly from the one used for the first top 25. This post will explain how the whole mechanism works. For the most part, it is irrelevant to how you vote individually, but some people like to know how everything works. (If any of you read Cryptonomicon, this system should look somewhat familiar.) If it seems overly complicated, don't worry about it -- several people have explained good strategies for ranking (or not) the games on your list:

- People vote for x number of games, 25 in our case, assigning a point value to each entry on that list.

- Each person's total point value is scaled to 1; the point values they assigned are scaled accordingly. In this way, each person can proportionately split the points they assign to particular games however they like, but the sum of those votes becomes the same for each voter.

NOTE 1: In cases where a person voted for fewer than the 25 games, the overall weight of their voting sum is reduced from 1 to (x/25)^2, where x is the number of games they voted for. Thus, someone voting for 20 games only has 64% the voting power of those voting for a full 25. Someone voting for only 12 games gets a value of around 23%. This is one of the two major differences from the scoring mechanism used the first year, which relied on the linear reduction of x/25, rather than the exponential reduction used this year.

NOTE 2: People voting for more than 25 games do not have the overall weight of their vote reduced; however, their vote is still scaled to 1, so they may be themselves possibly reducing the value of the weight they assign to the individual games listed on their ballot; certainly, the average value per individual game will drop.

- Two subscores are generated from the votes above:

- Subscore 1, percentage of voters, is determined by summing the scaled weights of all the voters. (For the 2003 vote, for instance, we had 39 voters, but several did not vote for a full 25, so the scaled number of voters was ~ 36.31) This number is then divided by the number of people voting for each game on the list, resulting in a scaled percentage of how many voters voted for each game. Games that made the list this year typically received votes from 25%-50% of the scaled number of voters.

- Subscore 2, importance to voters, is determined by summing the scaled points people assigned to each game receiving votes and finding the game with the maximum value. <i>That</i> value is then scaled to 1, and the lesser values are all scaled accordingly. This was the second change from last year's scoring and was implemented to keep the subscores between 0 and 1.

- The final score is determined by multiplying 2*Subscore 1 and adding it to Subscore 2, resulting in a value from 0 to 3. This value is then scaled to a 10 point scale (by multiplying the interim value by 10/3.)

- The list of games that received votes is sorted by final score, and the top 25 determined based on that. The point value of the 25th game is used to determine the honorable mention list. Any game that has a score of at least half that of the 25th game is given honorable mention. If/as group consensus develops on top 25 lists, the number of games on the honorable mention list will shrink. If/as consensus is lost on the top 25 list, the number of games receiving honorable mention will grow.

[nullstar]

I promised this, too -- sorry if the formatting sucks; I've got to catch a plane. I'll have to worry about the 2002 vote when I return.

The Top 25 Shmups of All Time (through 2003):

01) Do Don Pachi
02) Ikaruga
03) Thunderforce III
04) Battle Garegga
05) Layer Section / Galactic Attack / Rayforce / Gunlock
06) Radiant Silvergun
07) R-Type
08) MUSHA / Musha Aleste
09) Axelay
10) ESP ra.de.
11) Don Pachi
12) Soukyugurentai
13) Gunbird 2
14) Guwange
15) Strikers 1945 II
16) Do Don Pachi: Dai Ou Jou
17) Batsugun
18) Giga Wing
19) Gradius Gaiden
20) Zero Gunner 2
21) Salamander
22) Mars Matrix
23) Sengoku Blade
24) Zanac Neo
25) Thunderforce IV / Lightening Force


Honorable Mention (alphabetical):

Armed Police Batrider
Battle Bakraid
Blazing Lazers / Gunhed
Dangun Feveron
Dimahoo / Great Mahou Daisakusen
Einhander
G. Darius
Gaiares
Gate of Thunder
Giga Wing 2
Gokujou Parodius
Gradius / Nemesis
Gradius II / Vulcan Venture
Gradius III (SNES)
Hellfire
Lords of Thunder
Parodius / Parodius-Da!
R-Type III: The Third Lightning
R-Type Delta
Raiden
Raiden 2
Raiden Fighters Jet
Shienryu / Shienryu Gekioh / Gekioh: Shooting King
Shikigami no Shiro 2
Soldier Blade
Space Megaforce / Super Aleste
Strikers 1945
Tatsujin / Truxton
Thunderforce V
Twin Cobra / Kyukyoko Tiger
U.N. Squadron / Area 88 (SNES)

Code: Select all

------------------------------------------------------------------------

Stats of Dubious Merit:

Number of Voters: 39

Number of Games Receiving Votes: 230
Games supported by at least half of voters: 3

Number of games receiving honorable mention: 31


Games by Developer:
  - Konami:     3     5     8
  - Cave:       5     1     6
  - Psikyo:     4     1     5
  - Raizing:    2     3     5
  - Compile:    2     2     4
  - Toaplan:    1     3     4
  - Irem:       1     2     3
  - Seibu Kaihatsu:   3     3
  - Takumi:     2     1     3
  - Technosoft: 2     1     3
  - Red:        -     2     2
  - Treasure:   2     -     2
  - Taito:      1     1     2
  - Alfa System:-     1     1
  - Capcom:     -     1     1
  - Hudson:     -     1     1
  - Natsume:    -     1     1
  - Square:     -     1     1


Games by Original Platform:
  - arcade:       19     19     38
  - MD / Genesis:  3      1     4
  - PS1:           1      3     4
  - SNES / SFC:    1      3     4
  - PCE:           -      2     2
  - PCE CD:        -      2     2
  - Saturn:        -      1     1

------------------------------------------------------------------------

[Zhon]

Hmm, hope I'm not throwing fuel into the fire, but my thinking is that gyruss and such are basically space invaders with a different perspective (well, and wraparound). And various shmups don't necessarily all have a direct top-down perspective, like einhander and ray-series. Then again, it doesn't matter to me, since I wasn't going to vote for any of them, and overall, leaving out those games probably won't make much of a difference anyways to the final result. Even if they were allowed I doubt they'd make any mark.

[Anarchos]

I put my vote on a asteroids-remake and Space Bomber which is Psikyos version of Space Invaders. My opinion is that they are shmups, and that's it. The screen doesn't necesarily need to scroll in order for the game to be a shmup, because then games like rRootage wouldn't be either.

[Shatterhand]

You know, for the look of votes so far... it seems we will get the same result of previous years

I.E. Dodonpachi gets again the spot of best shmup ever, and everyone else will complain about it

Dodonpachi got too many votes with lots of points so far

[Turrican]

Code: Select all

Games by Original Platform:
  - arcade:       19     19     38
  - MD / Genesis:  3      1     4
  - PS1:           1      3     4
  - SNES / SFC:    1      3     4
  - PCE:           -      2     2
  - PCE CD:        -      2     2
  - Saturn:        -      1     1

Now THIS is an interesting result. Psone at third place for contributing shmups to the list, while saturn at seventh!!! It has to do something with the "exclusives" issue we discussed on another topic. Basically a lot of potential Saturn votes are stolen by the arcade counterparts, while PS1 manages to do better thanks to the many "true exclusives" like Zanac Neo, R-Type Delta, G Gaiden, Harmful Park...

Interesting indeed.

If games which are obviously shmups (note: shmup = shoot em up) such as Tempest, Black Widow, Gyruss, Asteroids and Robotron are to be disallowed, I'm not going to bother voting.

They're the fathers of modern shooters. Next you'll ban Space Invaders for not scrolling.

By the way, I won't submit my vote as well, following bloodflowers. His post has made me realize something I hadn't considered enough.

It is not just a matter of geometry. It's also matter of History. Geometry criteria only could bow to the perspective factor, but geometry + history cannot.

And to think these games are listed on xenocide files and reviewed too...

[Shatterhand]

I think it would be a big shame if someone didn't vote because of this.

I personally threw a vote for Gyruss, and other for Super Stardust (Asteroids alike), and I wasn't the only one....

[Herr Schatten]

You know, for the look of votes so far... it seems we will get the same result of previous years

I.E. Dodonpachi gets again the spot of best shmup ever, and everyone else will complain about it

Dodonpachi got too many votes with lots of points so far

I don't know. Maybe it's just wishful thinking, but I seem to notice a light swing towards more respect for the old school in the votes cast so far. If I remember correctly, one year ago games like Apidya, Space Manbow and the 8-bit Compile games showed up only in your list, Laydock's and mine, but this year many lists seem to contain at least one or two such items. I think I've even seen additional votes for Beamrider. Not that it will probably make any difference in the end, though.


As for DDP: People don't praise it that much for no reason. I think while few would say that it's the best thing ever, it has just the right level of "in-between-ness", so that everyone can relate to it in one way or another and can get enjoyment out of it. It's accessible enough and fun to play if you're just looking for a quick blast (if you generally prefer the old school way of shooting), yet has enough depth and intricate scoring mechanisms for those whose favourites are Dai-ou-jou or Ikaruga. You see, DDP is the game everyone, likes too, even if his/her absolute favourites are other games.

DoDonPachi is the shmup equivalent to Metallica in the late eighties.
Back then, before the genre started to dissolve into hundreds of sub-genres, Metal fans were divided in two camps: There was "Poser Metal" on one side and "Thrash Metal" on the other one. Both sides showed proper disrespect for the opposing camp, but regardless of which side people were on, Metallica was the band everyone liked, too. They were melodic enough for the posers to enjoy, yet hard and technical enough for the thrashers. It's just the same with DDP.