Legitimacy of "lagless"

[nimitz]

whichever version of MAME, lag/lagless, most closely emulates the response of the PCB is by it's own definition the legitimate version.
Once we've established which MAME version's lag comes closest to that of the arcade board, then that should surely be the version used for scoring since that will be the most faithful version as well as being the one closest to the MAME Project's own mission statement.

Amen.


As for actual numbers, pulsewidth has done some preliminary testing a few months ago with a photodiode and an optimal MAME setup (a game with 0 frames with a very fast response screen) and IIRC he got around 40ms of lag, this same game on PCB has 16ms at most (displays on the next frame), this means lagless mame would have to remove two frames of lag and be used in an optimal PC setup to be as responsive as pcb.

Currently lagless mame removes only one frame, which means it still isn't as responsive as pcb even with an optimal setup.

keep in mind these numbers are preliminary and more actual testing would be needed, but these numbers give us a good idea.

[PROMETHEUS]

Both statements have been duly noted. With that being said, if "lagless" builds give players an advantage over official MAME builds then I do not believe the scores should be compared. I know this gets tricky with all the different versions out there but this seems fair to me.

Simply put:
MAME vs MAME
Shmupmame vs Shmupmame
PCB vs PCB
and so on...

This conclusion is the worst thing that could happen in my opinion, it is terrible for a competitive environment to refuse comparing scores between versions. You are basically dividing the players in 3 or more groups, making the competitive community much weaker, as interaction and comparison between as many people as possible is what drives players to keep getting better. I think we should try our best to make everyone come together. For instance I also disapprove of splitting tables between ships in a particular game. It doesn't matter that there is slight imbalance between ships, it's better to let everyone compete on the same board, with the option of a few small separate boards for each ship, still letting people know how they perform among players who use the same ship.

I also don't agree that PCBs should always be the reference. The fact that they are "official" doesn't matter. They are only one version of a game and they aren't necessarily the best version. When console ports came out people realized that a different version from the PCBs can also be considered "official" because endorsed by the game developpers, and agree to compete on it, even though they are much farther from the PCBs in terms of gameplay that MAME usually is (speed differences making the game easier/harder depending on specific parts of the game).

What we must try doing is determine, when we have a choice, in the case of MAME versions, which of the MAME versions are suitable to be included in the general competition - which should never be splitted.

Also, I think we must realize that there is a difference margin that we should allow versions to have without invalidating their use in competition. After all, even two different PCBs sometimes perform differently ! (I heard someone measured that DDP's speed varied slightly between two PCBs, for example).

[PROMETHEUS]

As for actual numbers, pulsewidth has done some preliminary testing a few months ago with a photodiode and an optimal MAME setup (a game with 0 frames with a very fast response screen) and IIRC he got around 40ms of lag, this same game on PCB has 16ms at most (displays on the next frame), this means lagless mame would have to remove two frames of lag and be used in an optimal PC setup to be as responsive as pcb.

Currently lagless mame removes only one frame, which means it still isn't as responsive as pcb even with an optimal setup.

keep in mind these numbers are preliminary and more actual testing would be needed, but these numbers give us a good idea.

Secondly this is interesting info that, if confirmed, entirely invalidates the claims that lagless would be illegitimate. I like to hear this because I don't like going back to inferior controls, using a non lagless MAME.

[gray117]

Sigh...

^^ Basically I second what nimitz says...

Depending on your game, you're going to get 15ms or more worth of 'lag' on just about anything. But this is inevitable to how any game is made and it is so tiny that you'll have to be superman to know.

On a typical console game I'd be surprised if render lag is below 20ms after factoring in wired usb response before this even got to your tv.

Why such numbers sound significant in any respect is that 99% of the time when we're talking 'lag free', what we're actually concentrating on is minimising additional lag - where, say, 30ms on top of everything may become noticeable.

As a total, 'Lag free' accurate emulation is almost a sure thing to be worse than an original, and these are issues that are so small that they can devolve down to component quality: Let alone the 'accepted' , OS, hds, usb controllers, scaling, graphics integration, memory transfers and processor/emulation efficiency.

I dunno... imho its a bit like recording the altitude you played at, since that probably has a greater effect...

[mjclark]

As for actual numbers, pulsewidth has done some preliminary testing a few months ago with a photodiode and an optimal MAME setup (a game with 0 frames with a very fast response screen) and IIRC he got around 40ms of lag, this same game on PCB has 16ms at most (displays on the next frame), this means lagless mame would have to remove two frames of lag and be used in an optimal PC setup to be as responsive as pcb.

Currently lagless mame removes only one frame, which means it still isn't as responsive as pcb even with an optimal setup.

keep in mind these numbers are preliminary and more actual testing would be needed, but these numbers give us a good idea.

...this is interesting info that, if confirmed, entirely invalidates the claims that lagless would be illegitimate.

Absolutely! From the above information, lagless is more faithful to the arcade board and therefore (in MAME terms) actually more legitimate.

[PROMETHEUS]

Absolutely! From the above information, lagless is more faithful to the arcade board and therefore (in MAME terms) actually more legitimate.

Yeah that's good news.

[Barrakketh]

Saying that "lagless mame" is cheating is like saying that a fast response screen is cheating.

With current pc hardware limitations it is impossible to make lagless mame respond faster than pcb.

Since (as I am constantly reminded) "When used in conjunction with images of the original arcade game's ROM and disk data, MAME attempts to reproduce that game as faithfully as possible on a more modern general-purpose computer ", whichever version of MAME, lag/lagless, most closely emulates the response of the PCB is by it's own definition the legitimate version.
Once we've established which MAME version's lag comes closest to that of the arcade board, then that should surely be the version used for scoring since that will be the most faithful version as well as being the one closest to the MAME Project's own mission statement.

I'm pretty sure the core MAME devs would disagree with that because you are sacrificing visual accuracy (one way of degrading the quality of emulation) for the sake of lower latency.

[PROMETHEUS]

I'm pretty sure the core MAME devs would disagree with that because you are sacrificing visual accuracy (one way of degrading the quality of emulation) for the sake of lower latency.

This is off topic, we're talking about legitimacy for competition, where the very, very small graphic glitch does not affect player's performance at all (no unfair advantage as a consequence of it).

[louisg]

As for actual numbers, pulsewidth has done some preliminary testing a few months ago with a photodiode and an optimal MAME setup (a game with 0 frames with a very fast response screen) and IIRC he got around 40ms of lag, this same game on PCB has 16ms at most (displays on the next frame), this means lagless mame would have to remove two frames of lag and be used in an optimal PC setup to be as responsive as pcb.

Currently lagless mame removes only one frame, which means it still isn't as responsive as pcb even with an optimal setup.

Thank you! Jeez, I dunno where people get this stuff. They're probably comparing the mhz of their PC to the mhz of the PCB and figuring it's less lag on the faster system

[Evilmaxwar]

What we need here is some proper science and statistics. Playing a few times on each version isn't good enough, so the kind of thing that needs to happen is:

Play a game 100 times on the PCB
Play the same game 100 times on regular MAME
Play it another 100 times on lagless MAME

Then you'd get a mean and standard deviation for your score on each version, which should tell you whether or not lagless MAME gives you a statistically significant advantage compared to the PCB. Otherwise, the argument is mostly hypothetical. Anyone willing to be a guinea pig and try it out?

To be a more valid statistical study: You would need 50 people doing this, and 50 more doing it in the reverse order, To rule out the fact they are going to score better no matter what after playing 200 credits.

[CptRansom]

I'll do it. I promise I will still suck horribly even after 200 credits.

[PROMETHEUS]

It's going to be a few frames latency due to the host at the very least. Worst of all, this varies from system to system, so some people might be at an advantage without others even knowing it.

That's something I have experienced many times. When I play DDP on some computer at a meet, I often unpleasantly notice a whole lot of additional input lag than I'm used to (up to 100 more milliseconds, maybe even more !!). Lagless MAME still only substracts 1 frame (17ms) from that crazy lag, still making the setup a real handicap.

[=SNC=]

http://www.anandtech.com/show/2803/7

[DocHauser]

To be a more valid statistical study: You would need 50 people doing this, and 50 more doing it in the reverse order, To rule out the fact they are going to score better no matter what after playing 200 credits.

True, or you could just randomise the order you play them.

Ideally, you'd do it in a way that the player doesn't know which version they're playing, so as to rule out any psychological/placebo effects, i.e. if they're playing lagless then they might think they should do better, so they end up actually playing better; whereas if they're playing regular MAME they might think 'more lag, I'm going to suck' and play poorly as a result.

[third_strike]

If I remember correctly, developers add input lag to some of there games. By taking that lag out of gameplay you are changing the way the game is intended to be played.

Rarely ever true. In this case it's due to do the electronics (the VDP) and the sprite buffer.

I realise nimitz has also mentioned this, but it seems that people need to be reminded that MAME needs to:

a) Get input from the host, which commonly uses the USB stack (laggy) or PS/2 (little to no lag since it's IRQ-controlled).
b) Simulate the guest code until vblank then render a frame (possible vsync delay here).
c) Push out the video driver's buffer (not much latency here, AdvancedMAME does away with this step afaik).
d) The driver then pushes it out to the framebuffer (lag).
e) The VDU will then receive the frame and display it (lag).

So really, unless you have an ideal host+display which doesn't have these issues (and it's possible but does not exist right now) it's very silly to say that MAME accurately emulates the latency of the PCB. It's going to be a few frames latency due to the host at the very least. Worst of all, this varies from system to system, so some people might be at an advantage without others even knowing it. The lagless patch sacrifices visual accuracy (rendering the sprite buffer without waiting a frame) for more accurate latency in comparison to the PCB on a real host. Theoretically, you can make a close-to-ideal host but not right now so it's a moot point -- i.e., MAME is always laggier than the PCB and would need adjustment to become more accurate in that manner.

So really, there's nothing wrong with scores which use the lagless patch as far as latency goes. The problem Haze et. al. have is the fact that, due to the advantage it gives players, many would gladly sacrifice visual accuracy for lower latency. I think that if someone finds the visual difference between lagless and MAME to be obvious, the latency difference between MAME and the PCB should be obvious to them as well. After all, in these kind of games, overall reaction time is king.

This is the enough to me.
But since I don't play for score no more then I prefer good graphics and good sound.

[Evilmaxwar]

Heres something interesting:
In Mushihimesama futari 1.5 on the 360 theres a setting to adjust the screen delay in arcade mode and the default setting is set to faster than the PCB.

you select Arcade mode , you go to screen options -> Maniac mode -> extra settings : Frame delay, default is set to 1(fast) , if you up it it says 2(PCB).

Would this imply that the 360 default setting is more responsive than the actual PCB? Which according to some here amounts to cheating?
Or maybe on the opposite, the 360 likely has other delays elsewhere ( wireless controller, LCD screen etc...) and using the default setting is in truth closer to the PCB ?

What if im using a wired controller and a CRT on my 360 ?

This topic certainly is an interesting one.

[system11]

I can think of words other than 'interesting' to describe some of the bits of this topic.

Is PC input hardware -> encoder -> USB hardware -> decoder/drivers -> OS -> application in any way similar to a simple logic high/low circuit with a near direct connection to a CPU purely running game code?

The answer is (obviously) no, and that's just the inputs.

[TrevHead (TVR)]

I think its all down to the individual scorer and whose running the score board

Thats why ive made my own FAIR score board http://shmups.system11.org/viewtopic.php?f=2&t=35987

[Zeether]

There's only one answer to this.

It's just you.

[Evilmaxwar]

think its all down to the individual scorer and whose running the score board

Thats why ive made my own FAIR score board viewtopic.php?f=2&t=35987

You forgot to mention the standardized distance one should be placed from the screen, Considering the light takes time to go from the screen to the eyes of the player, being placed farther from the screen induces lag.

[Despatche]

Playing 360 ports on a CRT is basically cheating then. The response of controls compared to a PCB on the same screen is noticeably faster.

Sarcasm aside, ports tend to have their own problems. Surely, there's more difference between port to PCB than there is between laglessmame to PCB.

[Despatche]

Topics like these bring to mind, yet again, that all these people seem to live atop an incredibly fragile web of contradictions, ready to fall apart as soon as they discover what they're doing or saying.

...hey... hey, maybe I'm one of them, man

[cools]

The only way of comparing scores fairly is for all competitors to play using an identical configuration.

The ideal way of achieving this is to use a single setup shared between all players. Second best is a standard platform. The more variables you introduce between players the less valid the score comparison becomes.

[Treasurance]

lagless is time travel
time travel is illegal

[Iori Branford]

Input delay is a bug, an undesired operation, for which lagless emulators are the patch. (In the case of Raizing's toaplan2 family of games where only the stick lags, it's probably just more of Yagawa's trolling. Respect for the developer's wishes can only go so far.)
Integrity of the game rules and game state is something everyone desires; any change to those would clearly be a cheat.

It's up to scoreboards to either dedicate themselves to particular configurations, or include relevant system and software information with every score and let viewers filter out configurations in which they're not interested.

[Despatche]

the only way to fairly compare scores is for every run to be publicly viewed and recorded and everyone must use the exact same computer/emulator/mouse+keyboard+control/etc and all scores made must be completely incomparable to scores made with any small change to the setup

[TrevHead (TVR)]

Basically Prom and austere have said all I want to say ill just add aslong as the emulator isnt massivly different from the original / majority of software then its fine. We arnt twin galaxies here, were here to have fun.

Although I cant compare the PCB with emulations personally but imo a few more tests with different PCs, games and rom versions need to be made so we can put this to bed once and for all. As it doesnt matter if shmupmame really is cheating or not the main thing is that some ppl are doubting it weather its their own score or someone elses. This undermines the whole leaderboads in and the hard work and achievements ppl have put into it.

[LeeVanCleef]

I think think the statements about the settings for 360 ports are missing the point in some way. Most leaderboards state that you must use default settings and you do not compare 360, or any console ports, to PCB or MAME and vice versa. As I and a few others have stated it should be up to the member running the thread or outside leaderboards to determine what settings or builds to allow. I am sure that with the tremendous amount of skill found on this site, players could use the version required and post scores with no problem.

[Evilmaxwar]

I think think the statements about the settings for 360 ports are missing the point in some way. Most leaderboards state that you must use default settings and you do not compare 360, or any console ports, to PCB or MAME and vice versa. As I and a few others have stated it should be up to the member running the thread or outside leaderboards to determine what settings or builds to allow. I am sure that with the tremendous amount of skill found on this site, players could use the version required and post scores with no problem.

In case you were quoting my post: In my example i used Futari 1.5 which has one scoreboard for both pcb and port. My point was to further demonstrated how a variety of factors can influence "control tightness" and how clearly many of the scoreboards are pretty much doomed to be not 100% perfect. Many scoreboards allow PCB, ports and MAME, some as you said are more specific. I also agree the bottom line is that its up mostly to the one starting the leaderboard.

At this point anyway i think this topic is pretty much exhausted, case has been analyzed to a point that it seems clear that if we start to make such a serious issue of which version of MAME we use, we then would need to redo most of the scoreboards to encompass all the little differences everywhere.
I think the topic is interesting but i pretty much totally agree with PROMETHEUS on this one.

[PROMETHEUS]

I am sure that with the tremendous amount of skill found on this site, players could use the version required and post scores with no problem.

In my opinion it only works if MAME is always recognized instead of just PCBs, because :
1) PCBs are too expensive to expect even all serious players to buy them (they aren't meant to be bought by players anyway).
2) Some players, like me, won't like to play with an arcade stick. It doesn't matter to me that the games were meant to be played with an arcade stick originally : they have never been made available to me through any other platform than MAME and I developped all my skill and experience on keyboard. I will not switch.

I regard MAME as the main platform for arcade gaming. The fact that it is unofficial is entirely irrelevant to me. It's close enough to perfection and adds a lot of extremely useful features (saved states being a MAJOR one, easy replay saving is another), making it a far better mean of practicing. It will also last forever, unlike the PCBs. Basically, I find it more perfect than arcade.