XRGB Mini + Extremecap U3 Colour and Artifacting Issues

[Jademalo]

I've asked about this before about a year ago, but ultimately I'm still having problems. So far I've thought it was caused by at least three separate issues, but on further testing each one of these is definitely not the cause. At this point I'm frankly out of ideas, so I figured it might be worth asking again as well as trying to organise my understanding by writing it out.
I also don't have a very good camera, so apologies for the really bad photos. It's actually worse in person though.
For reference, My N64 is RGB modded and it's running through the Framemeister at 720p into an Extremecap U3. For the purposes of these tests I haven't got anything else hooked up.


To start with, my issue is similar to this - https://blz.la/rgb/#post_6
However, it's not as simple. Blizzz's post explains sampling issues due to a non-square PAR, but I've got reason to believe that isn't my issue even though the results are extremely similar. This post is relevant in a couple of ways though.
As far as I can tell, the N64 uses a square PAR. That should mean that (regardless of the N64's inbuilt anti-aliasing) I should be able to get an output from the Mini that is comprised of entirely sharp square pixels that are each 3x3 at 720p. The easiest thing to test this with as far as I can tell is the interface on Super Mario 64.
After a lot of testing, I can't get the N64 to align perfectly like this. Things like H_POS and H_WIDTH don't seem to be able to fine tune it enough to align it. This isn't too much of a problem by itsself, but I believe it may be amplifying the other issue.
I'm still fairly sure this is possible though, thanks to an image I found somewhere in the XRGB Mini thread - http://i.imgur.com/QpCdItm.jpg. I'm just not good enough at configuring it, and it's especially difficult since I don't have a good capture to reference when configuring.

With these current settings, the image that I'm getting displayed on my monitor connected directly to the mini is gorgeous. The pixels are almost perfectly sharp, the colours are all fantastic, and all in all it's a great experience. I'm the type of person who likes uncompromised console processor out pixel accuracy rather than "how it was" accuracy, but that's just me.

However, when viewing it through my Capture card, things get... Messy.
Vertically, every pixel has the right hard edges, as would be expected. However horizontally, it very quickly generates an absolute ton of artifacting and what I can only describe as a "glow". This is definitely not something to do with the console, and as far as I can tell not something to do with the mini. It seems this is to do with my capture card.
My capture card, as I mentioned, is an Avermedia Extremecap U3. My understanding of this card is that it's essentially the most barebones thing imaginable, but because of that it doesn't do much to the signal it's capturing. According to the information I could find, it decodes whatever signal, then samples the singal into 4:2:2 and that's what is seen by the PC. This obviously lowers the colour resolution, but literally every capture card does that unless you're in the $1k+ range of professional cards. I mean, the LiveGamer Portable which almost everyone uses to stream is only 4:2:0, and it doesn't seem to cause as many artifacts as mine.

In the blog post I linked above, blizzz has this image - https://blz.la/rgb/img/2014/mario_framemeister_zoom.png
From my understanding, the SC-500N1 that he took it with also has 4:2:2 chroma subsampling, but that image has absolutely no artifacting or glow aside from the "delibrate" sampling issue which is correct based on what is being output from the framemeister. The actual image from the framemeister is captured extremely cleanly and accurately. Every few pixels there is a nice sharp edge exactly where it would be expected based on the sampling.

In contrast, let's look at mine. (You will need to download some of them to get a good close look at the pixels) -
Direct from the Mini - http://imgur.com/bwIAR7E
Through the Extremecap - http://imgur.com/wUzUkGp
Screengrab through the Extremecap - http://imgur.com/wd0F77M
Full screengrab - http://imgur.com/CI0MAbB

The most important area is to the right of the camera. As you can see, there is a colossal amount of blue glow around it that was not present at all in the direct mini output. If you then look at the full image, every single pixel horizontally is divided up substantially into lines. There's absolutely no uniformity anywhere where the pixels should be, it's all just one horrible, blurry mess. There's also the blue glow to the right of pretty much every single dark area, even places like the star's eyes. This is absolutely not present anywhere on the direct mini output.
I could understand it being a pixel or two due to the chroma subsampling, but this glow in some places extends 3-4 pixels away from the object. While this might not seem much and most people will go "Eh, that's just what it is", it gets amplified to an insane level when capturing video or streaming. Further scaling for people using 1080p monitors combined with the H.264 codec combined with movement makes the glow not just noticable, but distracting and ugly.

This isn't just an issue with the N64, and it isn't just an issue with the right hand side of dark objects looking a little blue.
Here are two captures from Yoshi's island. A direct capture, and a zoomed in look at the life counter
1 - http://imgur.com/UDgoZxw
2 - http://imgur.com/0gOCZI3
You can very clearly see the 2-3 pixel blue halo around the right hand side, but there is also a 2-3 pixel red halo around the left. This artifacting is present absolutely everywhere, and extends multiple pixels rather than the single one that would be expected from the snes as detailed in the blog post. Once again, there isn't one pair of horizontal pixels that share a colour, everything has been blurred together to an extreme degree.
And once again, this is not an issue in any way with the direct output from the Mini. This is not visible with the naked eye on a 24" 1080p monitor.


Now from my understanding, (Please correct me if I'm wrong,) the Mini can output two different types of signal over HDMI. RGB, and YCbCr.
RGB outputs at 8bit or 12bit, and has the choice of either Full or Limited. However, this mode has an issue where the greens are all crushed when using an RGB source.
YCbCr outputs at 8bit or 12bit 4:4:4. The output range on mine has to be set to full for some reason, even though it was my understanding that YCbCr didn't support full (and my cap card has to be set to 16-235 to look right, so it obviously is using limited). Is this just the mini being weird?

As far as I am aware thanks to things Fudoh has said, the Mini's internal processing uses YCbCr 4:2:2 Rec. 709 while it's doing it's thing. It then decodes this for the final output. For RGB, it decodes it with Rec.601 which is why all of the greens get ruined. For YCbCr, it decodes correctly with Rec.709 and then reencodes it at 4:4:4 Rec.709.

This should all mean that regardless of the output set, the Mini is outputting it's final video without any further chroma subsampling in full quality.



Hence, I'm at a loss. I'm hesitant to buy a different capture card since I can't tell if I will just get exactly the same results, I don't know if it's a chroma subsampling issue, I don't know if it's something to do with the settings on the mini, I'm absolutely clueless. All I know is that some SM64 streamers who are using cheap capture devices and S-Video from the N64 are getting better results than I am since while it is a bit blurrier, there isn't a colossal amount of ghosting and halos around every single dark thing in the image.


Sorry about the length of that, I got a bit carried away since I've gotten fairly frustrated at this. Why is it so damn difficult to perfectly capture a 720p60 HDMI feed? D:

Thanks!
Jade

[Guspaz]

Some notes:

- 4:2:2 and 4:2:0 have the same horizontal chroma resolution.

- The glow is grey, not blue, according to photoshop.

- The lack of fixed pixels would appear to be a problem on the Framemeister, not the capture card, so you'll want to tweak those settings.

- The problem is virtually invisible on your Yoshi's Island screenshot. While it would be good to figure out how to fix it, you seem to be greatly exaggerating the problem. Nobody is going to notice this on a stream.

- YCbCr that have a limited range, but it's not all 16-235: YCbCr has a 16-235 range on Y and a 16-240 range on Cb/Cr.

- Your monitor is likely not in the same display mode for the direct-display versus the captured display. The direct display would have your monitor in YCbCr mode, while your PC would be converting the capture to RGB and then displaying over RGB. You can switch your PC to YCbCr and try using a program that can output natively in YCbCr (like a good video renderer) to avoid the colourspace conversion, and maybe see if that changes anything.

In terms of the worse results with the capture card, regardless of crushed greens, does the "glow" problem occur when the Framemeister is outputting RGB with the capture range matching the Framemeister setting?

[Jademalo]

Thanks for the reply!

- 4:2:2 and 4:2:0 have the same horizontal chroma resolution.

I'm aware of that, I'm just stating that the LGP is objectively a lower quality yet I've never seen bleed issues like this before.

- The glow is grey, not blue, according to photoshop.

Huh, it looks distinctly blue to me, and isolating the blue channel in photoshop highlights the glow substantially more than the other colours.

- The lack of fixed pixels would appear to be a problem on the Framemeister, not the capture card, so you'll want to tweak those settings.

As I've said, while the edges aren't perfect, blocks of the same colour display as pure on my monitor when connected directly. Only after putting it through the capture card does the issue become substantial. I would like to try someone elses 720p N64 settings though.

- The problem is virtually invisible on your Yoshi's Island screenshot. While it would be good to figure out how to fix it, you seem to be greatly exaggerating the problem. Nobody is going to notice this on a stream.

I'll get a video recorded soon, essentially a combination of all factors plus h.264 makes it a lot more noticable than a static screenshot. It's almost like ghosting on a bad monitor when moving mario to the left. My main annoyance with it all though is the discrepancy between what I can see directly and what is getting captured.
And yes, while the average viewer will probably not notice, there probably won't be many people watching. It's mainly for myself, and ultimately I'm noticing it.

- YCbCr that have a limited range, but it's not all 16-235: YCbCr has a 16-235 range on Y and a 16-240 range on Cb/Cr.

Oh, I didn't know that, that's pretty interesting. Thanks!

- Your monitor is likely not in the same display mode for the direct-display versus the captured display. The direct display would have your monitor in YCbCr mode, while your PC would be converting the capture to RGB and then displaying over RGB. You can switch your PC to YCbCr and try using a program that can output natively in YCbCr (like a good video renderer) to avoid the colourspace conversion, and maybe see if that changes anything.

Just did this test and it made absolutely no visible difference.

In terms of the worse results with the capture card, regardless of crushed greens, does the "glow" problem occur when the Framemeister is outputting RGB with the capture range matching the Framemeister setting?

Just tested - Yes. It's pretty much exactly the same, slightly brighter due to the ground and grass being brighter in general thanks to the bug.

[Guspaz]

Huh, it looks distinctly blue to me, and isolating the blue channel in photoshop highlights the glow substantially more than the other colours.

Only because the background has less blue, making the glow seem more intense in the blue channel.

If I sample starting at the Yoshi's Island circle, first pixel being black, you get:

RED/GRN/BLU
002/000/000
133/120/105
184/157/116
195/157/083
198/155/076
203/156/070

At that point you're in the background colour. This is not a shift to blue, as you can see, since the blue channel is always the least. I think only the chroma channels are smearing, and only to the right. If I shift the chroma channels (but not luma) to the left, then I see the same effect on the left side, but with a much sharper outline.

[Jademalo]

Oh, that's interesting. Any chance you can post the modified image that you're getting with the chroma channels shifted to the left?

[Fudoh]

However, it's not as simple. Blizzz's post explains sampling issues due to a non-square PAR ... As far as I can tell, the N64 uses a square PAR.

the problem aren't really square or non-square pixels, it's that every system has a different horizontal pixel count and the Mini doesn't really adjust for that. The OSSC for example does and can therefore deliver sharp pixel edges for a number of source systems.

But this being said, it's your N64's fault that you don't at least get the same results as with your SNES or SFC. There are different N64 revisions and some of them - unfortunately - don't play very well with the Mini. Mine for example (japanese N64 with RGB mod) produces the same contouring on edges as long as I keep the H_SCALER where it usually belongs for 240p content (4-6 range). Instead I have to set H_SCALER to 10 or 11 to keep down the false contouring. This will result in less sharp pixel edges, but it will also heavily reduce the effect you're seeing. Maybe give that a try.

[Guspaz]

Oh, that's interesting. Any chance you can post the modified image that you're getting with the chroma channels shifted to the left?

Here is the original at 3x:



And here is the chroma channels shifted two pixels to the left:



EDIT: I did it by converting to lab colour and back. It's not at all the same as YCbCr, but the important thing is that the first channel is still luminance, and the second and third channels still store chrominance, so shifting them should have the same effect.

[Jademalo]

However, it's not as simple. Blizzz's post explains sampling issues due to a non-square PAR ... As far as I can tell, the N64 uses a square PAR.

the problem aren't really square or non-square pixels, it's that every system has a different horizontal pixel count and the Mini doesn't really adjust for that. The OSSC for example does and can therefore deliver sharp pixel edges for a number of source systems.

But this being said, it's your N64's fault that you don't at least get the same results as with your SNES or SFC. There are different N64 revisions and some of them - unfortunately - don't play very well with the Mini. Mine for example (japanese N64 with RGB mod) produces the same contouring on edges as long as I keep the H_SCALER where it usually belongs for 240p content (4-6 range). Instead I have to set H_SCALER to 10 or 11 to keep down the false contouring. This will result in less sharp pixel edges, but it will also heavily reduce the effect you're seeing. Maybe give that a try.

I was under the assumption that H_POS and H_WIDTH were used to counteract that, so the mini could be manually tuned for each console in order to properly display the pixels.
Also what do you mean by the OSSC? I did a quick google but couldn't come up with anything relevant.

The thing I can't understand, as I mentioned, is that direct into my monitor the N64 feed from the mini looks absolutely fantastic. It's the discrepancy that I don't understand.
I've never actually tried messing with H_SCALER, I'll give that a try. What exactly does the H_SCALER and V_SCALER do on a more technical level? It's easy to see the big jump between V 5 and 6 and GAME/VIDEO, but I've never really been able to see a difference with any other number.



Also thanks Guspaz, that's really interesting!


EDIT: Just tested H_SCALER higher, and there is literally no difference. As in, testing it in photoshop with layers, there's actually objectively no difference between H_SCALER 6 - 17. Absolutely nothing. Now that has me confused.

[Shuco13]

The thing that I ended up doing was seperately set colors, aspect and sharpness for each input. It takes some time but is defininitely worth it since every signal is different.

[Jademalo]

The thing that I ended up doing was seperately set colors, aspect and sharpness for each input. It takes some time but is defininitely worth it since every signal is different.

I've been doing that, but this issue extends beyond simply setting a console specific profile with the mini. If I can't even get one console looking right, then multiple profiles aren't going to help matters.

[Fudoh]

Also what do you mean by the OSSC?

http://shmups.system11.org/viewtopic.php?f=6&t=52158

EDIT: Just tested H_SCALER higher, and there is literally no difference. As in, testing it in photoshop with layers, there's actually objectively no difference between H_SCALER 6 - 17. Absolutely nothing. Now that has me onfused.

that's not possible unless you're doing something wrong. H_SCALER sets the horizontal scaling engine. Basically the blurriness. H_SCALER 17 makes the picture looks like composite video instead of RGB.

Here's my N64 on my Mini: http://pms.hazard-city.de/n64mini.jpg

You can see three different H_SCALER settings here. Everything else is the same. Top one is 5, which is usually great for (all other) 240p systems. I think the third one is 10 or 11. As you can see, the image is little bit more blurry, but the contouring effect is basically gone. You can increase the sharpness setting from 0 to 1 if you like to counter the high H_SCALER effects.

[RGB32E]

1. The only real way to get sharp pixels from the Nintendo 64 is to use the UltraHDMI mod. No amount of tinkering with XRGB-mini settings will compare to what can be achieved using the Sharp Pixels and VI De-Blur settings available with the UltraHDMI. Period. On top of that, 240p/480i transitions are instantaneous, unlike the mini's 5 second re-sync.

2. Capture cards kind of suck. I'm actually interested in replacing my PEXHDCAP with something that won't chroma sub sample. It's an issue with the XRGB-mini as well, as many horizontal pixel transitions result in artifacts. I ended up developing an application that "corrects" subsampling artifacts. Here's a before and after when using the UltraHDMI (Integer+ and de-blur on) as a source at 720p.



See the artifacts on top of Mario's hat?

Here's the same algorithm applied to your SMW2 capture. Perhaps not perfect, but it goes a long way in reversing the artifacts.

[FBX]

What exactly does the H_SCALER and V_SCALER do on a more technical level? It's easy to see the big jump between V 5 and 6 and GAME/VIDEO, but I've never really been able to see a difference with any other number.

Game and Video just apply a default fixed value setting in the H_Scaler and V_Scaler entries. You can actually manually apply the same settings Game and Video use when you turn Auto_Scaler to OFF, so you might as well completely disregard Game and Video, and always use manual values.

As for what H_Scaler and V_Scaler do, the way I like to describe it is they adjust the 'focus' on their given axis. My own experiements indicate H_Scaler: 4 is the sharpest possible horizontal focus without introducing artifacts for most systems. A value of 5 is a hair sharper, but does introduce artifacts. Sharpest setting on V_Scaler is 6 or 7 (cannot really see any difference between the two values).

[Jademalo]

that's not possible unless you're doing something wrong. H_SCALER sets the horizontal scaling engine. Basically the blurriness. H_SCALER 17 makes the picture looks like composite video instead of RGB.

Here's my N64 on my Mini: http://pms.hazard-city.de/n64mini.jpg

You can see three different H_SCALER settings here. Everything else is the same. Top one is 5, which is usually great for (all other) 240p systems. I think the third one is 10 or 11. As you can see, the image is little bit more blurry, but the contouring effect is basically gone. You can increase the sharpness setting from 0 to 1 if you like to counter the high H_SCALER effects.

Thanks for the clarification on the OSCC, that looks pretty interesting!

Hmm, that's super weird. H_SCALER has absolutely no effect for me until I get to 15, when it immediately goes extremely blurry. I'm running the latest firmware, that's extremely strange. Here's a collection from 6-17 to show you exactly what I mean - https://dl.dropboxusercontent.com/u/523 ... SCALER.zip
The 10-11 in that image does look good though!

1. The only real way to get sharp pixels from the Nintendo 64 is to use the UltraHDMI mod. No amount of tinkering with XRGB-mini settings will compare to what can be achieved using the Sharp Pixels and VI De-Blur settings available with the UltraHDMI. Period. On top of that, 240p/480i transitions are instantaneous, unlike the mini's 5 second re-sync.

Does the VI blur affect the output on a subpixel level? I'll be honest in saying I totally forgot about that, I didn't even know about it until about a week ago, and apparently I forgot in the meantime. While it doesn't explain the halo, that would definitely explain a lot of the lines and "artifacts" if it was subpixel.
I've just had a look into the UltraHDMI - While it's far too expensive for me to consider having to deport it to the US and then import it back into the UK, if there's a UK installer in the future then I might look into it. I can't argue with the precision of that VI deblur + Sharp Pixels!

2. Capture cards kind of suck. I'm actually interested in replacing my PEXHDCAP with something that won't chroma sub sample. It's an issue with the XRGB-mini as well, as many horizontal pixel transitions result in artifacts. I ended up developing an application that "corrects" subsampling artifacts. Here's a before and after when using the UltraHDMI (Integer+ and de-blur on) as a source at 720p.

That's definitely what I've taken from this whole thing, lol. The problem is that it will be so far down the list of things that matter for the consumer capture card companies I expect we'll get 4k capture before RGB or 4:4:4. Any idea what the cheapest pro card that can do 4:4:4 is?

Holy crap, that's amazing! Are you planning on releasing it in some form?
The Yoshi's Island image is also fascinating, mainly because while it fixed all of the other artifacts, it left that glow perfectly intact. I do remember Fudoh mentioning something to me a while ago about the SNES's output being too "sharp" and the Mini's internal processing compounding the glow issue, but that still begs the question why it would happen on the N64.



Thanks for the clarification FBX, it definitely seems like something is up with that setting on my Mini. I understand what it's supposed to do now, but it definitely doesn't seem to want to do it.

[Guspaz]

Remember that no streaming or video site will do 4:4:4, so nobody will ever see 4:4:4 captures but you: YouTube, for example, is 4:2:0. I'm not sure what Twitch uses, but it isn't 4:4:4, since Flash doesn't support 4:4:4 and Twitch still uses Flash.

[blizzz]

The problem is that it will be so far down the list of things that matter for the consumer capture card companies I expect we'll get 4k capture before RGB or 4:4:4. Any idea what the cheapest pro card that can do 4:4:4 is?

We actually got both at the same time, at least in the prosumer segment. The Blackmagic Intensity Pro 4K (~$190 / 230€) lists 4:4:4 support for HD resolution and 4:2:2 for UHD. The specs sound awesome, but I don't think I've read a single positive review about it. There are also the Magewell cards that list 4:4:4 support and even multi-channel audio. But those are also not perfect and don't even come with software that can record multi-channel audio.

So in the end there's still not a great non-pro card that I know of.

[Jademalo]

Remember that no streaming or video site will do 4:4:4, so nobody will ever see 4:4:4 captures but you: YouTube, for example, is 4:2:0. I'm not sure what Twitch uses, but it isn't 4:4:4, since Flash doesn't support 4:4:4 and Twitch still uses Flash.

As I mentioned in one of the above posts, I know, the extra accuracy is mostly personal. I mean as I said, it's not like anyone will be watching anyway.

We actually got both at the same time, at least in the prosumer segment. The Blackmagic Intensity Pro 4K (~$190 / 230€) lists 4:4:4 support for HD resolution and 4:2:2 for UHD. The specs sound awesome, but I don't think I've read a single positive review about it. There are also the Magewell cards that list 4:4:4 support and even multi-channel audio. But those are also not perfect and don't even come with software that can record multi-channel audio.

So in the end there's still not a great non-pro card that I know of.

Haha, that's amazing. The price isn't even unreasonable.
What exactly are the negative reviews for the Intensity Pro 4k complaining about? I've only ever heard good things about Blackmagic.

[blizzz]

Let me just quote the headlines of the first three reviews on Amazon:

Wonderful list of specs to read! NONE of which are functional!

UPDATED 04-14-2015 - Does not work for PS4 or XB1 capture (as of 03-14-2015)

Currently unusable. Do not buy!

On top of that it also has an annoyingly loud fan.

[Jademalo]

Hahaha, that's absolutely insane. Is it a software/driver issue or is the hardware totally pants?
It's a shame that it doesn't work, those specs look absolutely perfect, especially at that price.

Most reviews I'm seeing post about august last year look positive, have they maybe fixed it since?