Shmup Input Lag Database

[Mark_MSX]

@komatik

I'll add a paragraph explaining that I double checked my ps4 and Switch numbers on my ASUS monitor. To clarify, the numbers on the lag page are the results from the HDGamer Fury cable as they were more responsive than the results of the ASUS monitor (again, only by about half a frame or so).

When I test Retroarch on Switch, I am going to use the exact same settings as I used on PC (basically going for least latency possible) and see how they compare to each other. I think this will be very interesting to see if there is a difference or not.

[Xyga]

One thing that I am certain of, though, is that the 360 port of Ketsui is a frame slower than Retroarch and PGM. So it could be possible that the PGM is only 1 frame and the 360 port is 2 frames.

Remember though that RA is not an emulator, it's either MAME or FBA, whatever, and the resulting delay depends on the settings and hardware.
Where a driver in an emulator is considered correct and is not calling an additional program or something like unusual sound or inputs emulation (which afaik in MAME can add a frame over the original hardware when they are required to assist the main driver) then the amount of delay produced matches that of the original game on its natural hardware (that we measure in frames because we're used to the way emulators work on computers)
This mean that in 'hands off' situation (no buffered sync method applied) in practice you are supposed to experience the real delay lenght. It's still better to have a sync method on top that doesn't add lag of course, since the correct behaviour is synced video anyway, you don't really want to check lag with portions of the frame hopping across the screen in a caterpillar movement fashion, or split in two areas, because the time is not presented to the screen in orderly linear fashion like it should.
So when you choose your hardware, emulator, and settings (assuming lag reduction running for the latter) it is absolutely crucial to be certain the entire chain is built/set so it doesn't add up anything on top of the game's natural delay.
For testing this Groovy is safer in that it doesn't permit to bring the input down to sooner than it should, at the very least it can get very close to the real thing, when run-ahead is much more hazardous in that it's set blindly in most cases (to be used correctly the proper delay should be investigated beforehand, and the settings -vsynced or not- applied in consequence)

note: I suspect a hands off Groovy would be valid to inquired about the raw delay, under the condition the reference location is the top of the screen, if so that would remove the need for hungry lag-reduced vsync only a powerful pc can really manage.
This might be your best source, most reliable method, to find out the original 'lag' for each - arcade - game, and use it as reference.

Frankly lag is one of the sneakiest things and there's so many variables it is very easy to get it wrong. Anyway good luck!

PS: for a good start with Groovy; a GPU with either a VGA or DVI-I+VGA adapter to your CRT monitor would be good. Ideally an AMD card with that, because you could install CRT_Emudriver, which would then allow you to run the games at their original framerates as well.

[Mark_MSX]

One thing that I am certain of, though, is that the 360 port of Ketsui is a frame slower than Retroarch and PGM. So it could be possible that the PGM is only 1 frame and the 360 port is 2 frames.

Remember though that RA is not an emulator, it's either MAME or FBA, whatever, and the resulting delay depends on the settings and hardware.
Where a driver in an emulator is considered correct and is not calling an additional program or something like unusual sound or inputs emulation (which afaik in MAME can add a frame over the original hardware when they are required to assist the main driver) then the amount of delay produced matches that of the original game on its natural hardware (that we measure in frames because we're used to the way emulators work on computers)
This mean that in 'hands off' situation (no buffered sync method applied) in practice you are supposed to experience the real delay lenght. It's still better to have a sync method on top that doesn't add lag of course, since the correct behaviour is synced video anyway, you don't really want to check lag with portions of the frame hopping across the screen in a caterpillar movement fashion, or split in two areas, because the time is not presented to the screen in orderly linear fashion like it should.
So when you choose your hardware, emulator, and settings (assuming lag reduction running for the latter) it is absolutely crucial to be certain the entire chain is built/set so it doesn't add up anything on top of the game's natural delay.
For testing this Groovy is safer in that it doesn't permit to bring the input down to sooner than it should, at the very least it can get very close to the real thing, when run-ahead is much more hazardous in that it's set blindly in most cases (to be used correctly the proper delay should be investigated beforehand, and the settings -vsynced or not- applied in consequence)

note: I suspect a hands off Groovy would be valid to inquired about the raw delay, under the condition the reference location is the top of the screen, if so that would remove the need for hungry lag-reduced vsync only a powerful pc can really manage.
This might be your best source, most reliable method, to find out the original 'lag' for each - arcade - game, and use it as reference.

Frankly lag is one of the sneakiest things and there's so many variables it is very easy to get it wrong. Anyway good luck!

PS: for a good start with Groovy; a GPU with either a VGA or DVI-I+VGA adapter to your CRT monitor would be good. Ideally an AMD card with that, because you could install CRT_Emudriver, which would then allow you to run the games at their original framerates as well.

I definitely know what you mean. The more I work with run-ahead the more I've realized that in some games like DDP, it actually works really well. But for other games, like DOJ, it kind of turns into a shit-show for whatever reason. I'm also skeptical about how it can support shmups beyond FBA. I'm not saying I dislike Retroarch and don't think it has its uses (especially for console games or on odd setups like hacked PS3), but I don't think it is the ultimate solution either. I am extremely interested in getting my hands on a grooveymame setup and seeing how it stacks up, I'd be very impressed if it is able to match the same lag as my PGM.

So some newbish questions, but once I get my grooveymame setup going, I do plan on making tutorials and stuff about it.

1. From the sound of it, Nvidia GPUs are not supported. I'm guessing that if I run grooveymame without switchres on windows 10 (which i have done), this is essentially just running MAME and does not have any lag reduction benefits. Is this right?
2. Is linux the best OS to use with grooveymame, in terms of accuracy and reduced lag?
3. In your opinion, what is the best combination of CPU and GPU for grooveymame overall? I do want to be able to run the later cave releases and stuff like that.

Cheers!

[komatik]

When I test Retroarch on Switch, I am going to use the exact same settings as I used on PC (basically going for least latency possible)

.... wait wait wait...
When you say "least latency possible", does that include enabling runahead with the correct number of frames, maxing out your fame delay, and messing with early/late usb polling and stuff? Because if so, how the flying heck are you still getting 3-4 frames of lag on PC? Turning all that stuff on (with a computer powerful enough to handle it) should effectively eliminate any input lag. When the RetroArch/libretro group created and implemented the whole runahead thing they were testing it against a real physical SNES hooked up to a CRT TV and were able to get less input lag on a PC desktop than the original hardware (1 frame vs 3, IIRC).

But for other games, like DOJ, it kind of turns into a shit-show for whatever reason.

The "use second instance" option can either fix or create a bunch of problems, ESPECIALLY if you have runahead set to the wrong number of frames. Not all games have consistent internal frame lag too so if you calibrated it during a busy part where the game had say 3 frames of lag and later played during a slow part where it only had 2, you'll get all sorts of wackyness. When I was configuring Daioh under FBA at first I was seeing an issue where I'd get hit, explode, and halfway through the explosion animation I'd snap back into existence like nothing happened because runahead's rewind-playback was set too aggressively. On a similar note, depending on core/game the audio can get hosed if runahead/2nd-instance is set wrong.

I'm not saying I dislike Retroarch and don't think it has its uses (especially for console games or on odd setups like hacked PS3), but I don't think it is the ultimate solution either.

If the UI wasn't a dumpster fire it would be the ultimate solution for NTSC-based consoles for sure. For PAL stuff or arcades that used something other than 60hz it could stand work. Speaking of arcades specifically, it would sure be nice if the libretro and MAME crowds could sort out their differences and give us some better cores.

2. Is linux the best OS to use with grooveymame, in terms of accuracy and reduced lag?

I don't know if it's still true, but as of a few years ago the Groovymame guys were having trouble with lag under Linux (even KMS) and recommended Win64. I have nothing to do with that project though so this is just what I heard through the grapevine.

[Mark_MSX]

Yes ha, when it comes to the Retroarch lag tests, I am using the correct number of run ahead frames. The only game I've tested so far is Ketsui, which came at 2 frames (same as PCB). Pretty much all the Retroarch games are going to come in at 2 frames I'll test DDP here pretty soon, but I think all the RA games will be 2 frames.

As far as DOJ goes, it's not too bad in RA, but I think FBA itself really struggles with the game overall, it has some stuttering and stuff even with 2nd instance. It just feels a little off to me compared to DDP in RA.

[komatik]

when it comes to the Retroarch lag tests, I am using the correct number of run ahead frames.

I'm confused how you're still getting upwards of 4 lag frames overall then. Are you 1,012% certain you don't have any extra triple-buffering shit going on with your drivers or something? The whole point of all the runahead+framedelay+polling stuff is specifically to prevent this exact problem. Something doesn't add up here.

Pretty much all the Retroarch games are going to come in at 2 frames

Uhh, that is not a guarantee. It depends a lot on what core you're using among other things. For example on my machine, Daioh has a constant 5 frames using the MAME core but variable 2-3 under the FBA core.

As far as DOJ goes, it's not too bad in RA, but I think FBA itself really struggles with the game overall, it has some stuttering and stuff even with 2nd instance. It just feels a little off to me compared to DDP in RA.

So, wait... when you say "in RA", what exact combination of software are you referring to? RetroArch itself is a framework not an emulator, the actual emulation is done by cores. Are you using the FBA core in RetroArch, or are you using FBA standalone exe? What cores are you using in RetroArch exactly? Are you using one of the MAME cores in RetroArch? Because unless things have changed recently only the FBA core properly supports runahead. In MAME 2003 plus it's unstable and the other MAME cores don't implement it at all.

......did you turn on runahead but then use a core that doesn't actually support runahead? Because that would explain the numbers you're getting.

edit:
Just to be clear, what I'm wondering about here is "compared to DDP in RA". Did you mean "compared to DDP under the FBA core" or "compared to DDP under the MAME core" or something else?

[komatik]

a GPU with either a VGA or DVI-I+VGA adapter

As an AV guy, I second this, strongly.

HDMI is nice and modern and all but having to use a powered converter to get it into a CRT is sub-optimal since you want as few stuff in between as possible. A direct VGA port is obviously easier, but a DVI-I ("integrated", not DVI-D "digital only") port is almost the same thing since the card can switch to analog mode and produce a VGA signal directly and all you need is a passive pin converter.

The main difference between a native VGA port and DVI-I is that some cheapo Chinese DVI-I > VGA pin converters don't connect the pins that do com negotiation, so instead of your computer auto-detecting an "ASUS XYZABC desktop monitor, serial number 123456" it will show up as a "new unknown VGA device" and you'll have to re-configure the resolution and refresh rate every time you plug it in or reboot. Trust me when I tell you to get one of the better adapters, the $5 you save on the cheap one is not worth the time you'll waste.

[Xyga]

some newbish questions, but once I get my grooveymame setup going, I do plan on making tutorials and stuff about it.

1. From the sound of it, Nvidia GPUs are not supported. I'm guessing that if I run grooveymame without switchres on windows 10 (which i have done), this is essentially just running MAME and does not have any lag reduction benefits. Is this right?
2. Is linux the best OS to use with grooveymame, in terms of accuracy and reduced lag?
3. In your opinion, what is the best combination of CPU and GPU for grooveymame overall? I do want to be able to run the later cave releases and stuff like that.

.MAME's lacking a number of things which Groovy specifically makes up for and that includes lag performance even by default yeah
.for Linux afaik these days there's no discrepancies, the performance should be the same, there's a dedicated GroovyArcade you can look into (not maintained by Calamity and never used myself)
.I am not familiar with setting it up for VGA/31Khz and Win 10, therefore...

...definitely register at BYOAC and go to groovymame forums, create a new thread explaining you're looking into making raw lag measurements of the game's original delay using GroovyMAME, detail your setup (CRT model, OS, CPU, GPU, your connectivity options, stick, converters where applying etc)
Then also explain/link to your measurement method.

FYI this is where the people most knowledgeable about how MAME and input delay gather, they've made measurements for years also.
Before thinking of writing tutorials get ready to unlearn stuff you might have though about before, and completely forget about RA for now bc seeing how you've used it I think that'll only confuse you. Plus there are many different ways to use and set GM for different hardware environments, and one mastered does not explain everything about the others, so focus on your own case and purpose first. I don't know many people who can explain all aspects of each different case with confidence (probably only Calamity and a handful of others)
There's many dedicated tutorials (tho so stuffy that they scare people away), but they've got a forum and it's the best place to start, just ask, most people who wonder about GroovyMAME don't I can't figure why.

[Xer Xian]

HDMI is nice and modern and all but having to use a powered converter to get it into a CRT is sub-optimal since you want as few stuff in between as possible. A direct VGA port is obviously easier, but a DVI-I ("integrated", not DVI-D "digital only") port is almost the same thing since the card can switch to analog mode and produce a VGA signal directly and all you need is a passive pin converter.

Is there any reason why an average built-in RAMDAC would be different than an average external DAC in terms of latency.

[komatik]

In theory, having everything internal to the card allows all sorts of optimizations since the hardware already knows what it's doing whereas an external device can't make assumptions. In practice, they should be more or less equal but I have encountered a few crappy HDMI>VGA converters that really really suck, and a lot of average converters that kinda suck. This is especially true if you're trying to do something like use an HDMI port to drive a VGA at 1280x1024 85hz; some converters choke on non-4:3 ratios and non-60 refresh rates and insist on live resampling the feed, leading to all sorts of bullshit. If your card has built-in VGA or DVI-I it will always work, whereas HDMI converters can be a coin toss sometimes.

[Mark_MSX]

@komatik

Hey man I think there might be some confusion about the results on my input lag page. The only game in RA that I've tested is Ketsui, at 2 frames. All the other tests are on different platforms, so I'm not sure which 4 frame result you are referring to. Also, I do understand that I would get varying results if I used the MAME core, but I do not plan on testing the MAME core, just FBA core.

Hope this helps clear stuff up

[komatik]

Hey man I think there might be some confusion about the results on my input lag page.

Yes :)

On your page, you have a section labeled "Windows PC" which as of this writing has the following entries:
- Blue Revolver
- Crimson Clover World Iginition
- Devil Engine
- Ketsui (Shmuparch)
- Ketsui (Shmupmame)
- ZeroRanger

You wrote:

As far as DOJ goes, it's not too bad in RA, but I think FBA itself really struggles with the game overall, it has some stuttering and stuff even with 2nd instance. It just feels a little off to me compared to DDP in RA.

The "2nd instance" setting is specific to runahead. AFAIK, the only emulation environment that supports runahead is RetroArch, and furthermore the only arcade core that supports runahead in RetroArch is the FBA core.

The only game in RA that I've tested is Ketsui,

When I test Retroarch on Switch, I am going to use the exact same settings as I used on PC

Right, so, how did you test DOJ and DDP in RetroArch if you only tested Ketsui in RetroArch? How can you test these games with RetroArch on the Switch using "the exact same settings" if you're not even using RetroArch for the other PC games in the first place? Except you are using Retroarch for all your PC games, because you're using runahead with the FBA core? Do you see what I'm getting at here?

so I'm not sure which 4 frame result you are referring to

Specifically Crimson Clover World Ignition, but the others at 3 aren't better. You wrote "same settings as I used on PC (basically going for least latency possible)" but you clearly haven't gone for the least latency possible based on your results. The fact of the matter is that if you have RetroArch properly set up on a PC with runahead and everything, you should not be seeing 3 and 4 lag frames on games. (At least, not according to the people who wrote and tested the runahead code).

Ultimately what I'm saying is that these "Windows PC" lag numbers aren't useful right now because you're giving conflicting statements about exactly what software you're using and how that software is configured.

[Mark_MSX]

Hey man I think there might be some confusion about the results on my input lag page.

Yes

On your page, you have a section labeled "Windows PC" which as of this writing has the following entries:
- Blue Revolver
- Crimson Clover World Iginition
- Devil Engine
- Ketsui (Shmuparch)
- Ketsui (Shmupmame)
- ZeroRanger

You wrote:

As far as DOJ goes, it's not too bad in RA, but I think FBA itself really struggles with the game overall, it has some stuttering and stuff even with 2nd instance. It just feels a little off to me compared to DDP in RA.

The "2nd instance" setting is specific to runahead. AFAIK, the only emulation environment that supports runahead is RetroArch, and furthermore the only arcade core that supports runahead in RetroArch is the FBA core.

The only game in RA that I've tested is Ketsui,

When I test Retroarch on Switch, I am going to use the exact same settings as I used on PC

Right, so, how did you test DOJ and DDP in RetroArch if you only tested Ketsui in RetroArch? How can you test these games with RetroArch on the Switch using "the exact same settings" if you're not even using RetroArch for the other PC games in the first place? Except you are using Retroarch for all your PC games, because you're using runahead with the FBA core? Do you see what I'm getting at here?

so I'm not sure which 4 frame result you are referring to

Specifically Crimson Clover World Ignition, but the others at 3 aren't better. You wrote "same settings as I used on PC (basically going for least latency possible)" but you clearly haven't gone for the least latency possible based on your results. The fact of the matter is that if you have RetroArch properly set up on a PC with runahead and everything, you should not be seeing 3 and 4 lag frames on games. (At least, not according to the people who wrote and tested the runahead code).

Ultimately what I'm saying is that these "Windows PC" lag numbers aren't useful right now because you're giving conflicting statements about exactly what software you're using and how that software is configured.

Retroarch is not able to run PC shmups like ZeroRanger, Blue Revolver, or Crimson Clover. The results I have under the "Windows PC" section are of their PC ports that you buy off steam. Like I wrote in my previous posts, the only game on the list that is in Retroarch is "Kestui (shmuparch), the rest are either other emulators or just the PC releases. I have setup my PC GPU settings to be as responsive as possible (VSync disabled and all that) with the PC ports. However, there is no way to use run-ahead with Crimson Clover or any of the other Steam games. Retroarch is only able to use run-ahead with shmups that are supported by Final Burn Alpha, so older arcade titles DDP, Ketsui, DOJ, etc.

As for the issues with DOJ, that's kind of a separate conversation than input lag. I was more speaking about emulation accuracy in that case, as FBA seems to struggle with that game in general. Though it is possible to run DOJ at low levels of latency in RA, I'm still concerned with accuracy and slowdown issues. But like I said, that was a comment regarding some previous posts and isn't related to input latency.

[komatik]

Retroarch is not able to run PC shmups like ZeroRanger, Blue Revolver, or Crimson Clover. The results I have under the "Windows PC" section are of their PC ports that you buy off steam.

Oh ok, that explains that part. I'm still not familiar with all the games out there and how you run them, especially most of the bullet-hell ones.

the rest are either other emulators or just the PC releases.

Well you do need to list these out so people can compare properly. ie; "(Steam)" or "(groovymame)" or whatever you're using.
Different emulators/forks can have different lag numbers so its kinda important that people know what you were doing instead of having to guess.

FBA seems to struggle with that game in general.

Yeah FBA isn't perfect for a lot of games for sure. More than once I've encountered a game where MAME plays fine but in FBA the music is 1/2 speed or everything's choppy or some other weirdness.

But anyway, I'm still confused about what you were talking about in regards to the Switch stuff:

Tonight I am going to test Shmuparch/Retroarch games on switch, so I'm super super curious how that will compare to shmuparch tests on PC

Which games (plural) were you going to compare if Ketsui was the only one you tried? Was that a typo and you're only comparing Ketsui+RA+PC with Ketsui+RA+Switch?



Oh btw, I forgot to add my 2¢ on this:

Speaking about the xbox 360 numbers being a frame too high, this might be possible. It might be possible that all my results could be a frame too slow for some reason based on the way the light is wired or something. I'm not exactly sure how that could be possible, but I'm not closed off to the idea.

- It's not clear from your info page but it appears the camera you're using to capture the LED+screen is only 60fps? If so you might be experiencing a 'camera vsync' issue where you're losing a frame. If possible, try these tests again but film with like a phone whose camera can do 120 fps slowmo mode or something.

- People think of LEDs as being purely digital on or off, but all things in the universe are fundamentally analog when you get down to the subatomic level. Regardless of the technology used to make it, there will always be some kind of a 'warm up lag' on a light when you apply voltage, it's just a question of speed. You also have to consider what other circuitry you have in that equation as things like capacitors will introduce delay and temperature will affect battery response. And of course with LEDs you have to worry about the resistor you're using to regulate it. I don't know the exact circuitry you used or how much timing is an issue for it, but when we're talking about times under 20ms that's the range where I might start to consider these things. Try connecting a bunch of different LEDs to the same circuit and film them all turning on at once and see of any illuminate faster.

[DietSoap]

Do you think you might do a test on the PS4 and Switch versions of ESPrade?

[Shelcoof]

Just discovered this thread now

Great work! Now I can blame inplut lag for failing