Do consoles horizontally blur the output?

[setiawan]

I'm running a Raspberry Pi 4 at 240p over composite to a SDTV.

If I'm in a game, and I use no shaders, the text has this artefact where small parts of it change colour. I think this probably is happening to any part of the image that has fine details and is high in contrast.
If I use the 'tvout-tweaks_sharp.glsl' shader, the problem is 'corrected', and the colours go away (mostly anyway). It appears that all the shader does is slightly blur the image horizontally.

I've taken the liberty of putting together this image to compare the differences:



Disregarding the non-integer scaling along the horizontal axis for a moment (since it happens with integer scaling too), can anyone explain to me why this happens?

I grew up on PS1, playing it on SDTV over composite, and I never recall seeing this kind of artefact on text. Text that was supposed to be black always just looked black, and text that was supposed to be white always just looked white. This leads me to think that, at some point after the PS1 (or any other console) renders the frame, some kind of processing takes place to blur the image horizontally, before outputting it.

Would this be correct? Is there a name for this? I'm just trying to better understand what's going on here. Like, is this being done in software by these consoles, or do the D/A converters in these consoles do it? Is there a reason the Raspberry Pi doesn't do it, even though these older game manufacturers understood that it needed to be done?

[setiawan]

Looks like the thread was drowned to the second page before it was approved (my first thread), so posting this to bump!

[NewSchoolBoxer]

Too bad I'm not smart enough to get the Img feature to work. Interesting to ask a composite video SDTV question in the land of RGB.

I can't find the 'tvout-tweaks_sharp.glsl' source online but it seems to be a combination of the 'tvout-tweaks' and 'sharp-bilinear' shaders. Some talk whether it's better to apply the bilinear sharpening in the linear RGB color space where it's more accurate or the gamma (composite) video space where some colors get interpolated more with gamma being non-linear but where pixel wobble is guaranteed to be removed.

The rainbow artifacts (sorry US spelling) definitely existed in composite video and still do if you hook up to LCD or CRT. Just hard to perceive from a distance and we all got so used to it that it felt natural. I definitely notice it now switching from RGB on PVM to composite. You're right that it happens in areas of high contrast, especially where dithering was used and this composite blur would conveniently replace the dithering with a transparency effect. Rainbowing (is that name?) is the tradeoff.

I took these screenshots from capture card in OBS that is straight from the video signal and not processed by a television or filter: https://i.postimg.cc/8CnBg56q/OBS-C-to-S.png
If you zoom in, you can see rainbowing in top left composite image on every letter but the L and E for obviously being the least distorted ones. Little to none in S-Video due to the separation of luma and chroma on different wires straight from the video chip that drastically reduces interference between the two.

But sure, rainbowing in your image was never that bad IRL. Let's think about where your composite video comes from. Open to corrections. Video encoder from $40 computer creates digital RGB image via emulator instruction that gets digitally processed by tvout-tweaks function to become artificially generated composite. Code commentary states no crosstalk between luma and chroma is simulated. Does clamp the 0-255 color values of RGB to 16-255 for more accurate CRT representation, apparently meaning colors cannot be as dark. Converts RGB to luma and chroma color space but is not an exact process since RGB has many more colors available and the digital to analog process itself adds noise to the signal. I guess conversion to S-Video would use identical code.

Does use real composite video fact of being on a single wire and signal travels to 3.5mm audio out jack where it may get hit with EMI from chips and capacitors along the way and take a small impedance mismatch from this part of the PCB not being 75 ohms when it hits the 75 ohm cable. Cross talk between video and audio on the cable due to being so close together versus on 3 separate cables from actual console. CRT then processes signal.

To answer the subject question, consoles didn't add blurring to the video output. Was blurry enough as is. The artificially created blurring in the image on the right is horizontal and vertical. A bilinear filter uses linear interpolation on x and y axis so is actually quadratic. Forms a square of 4 known values and calculates what the middle value should be from these 4. I still see some rainbowing but it's greatly reduced. I could imagine interpolating dulls the colors since the new value of every pixel is an average of other pixels, yet this process sharpens the image overall.

[NewSchoolBoxer]

I should have discussed the first two black and grey screenshots. The filter indeed softens the image by blurring it a little. The source image is free of video interference so there is no net gain of sharpness by reducing said interference. Instead the purpose of the filter is to remove pixel wobble, which I believe the filter author defines to mean the incorrect scaling of certain groups of pixels. You can see the filter does fix the shape of ( and ) and , that are a little distorted in the unfiltered image. The author considered this tradeoff desirable and I agree.

[Unseen]

Composite video is an interesting hack to add color to a signal that was originally black-and-white only, while staying compatible with existing black-and-white TVs: It re-uses a very high-frequency part of the signal to embed the color information. Visually, this means that the composite signal removes a bit of sharpness but gains color instead, which was a very acceptable trade-off in the middle of the 20th century when this system was invented.

Of course this process can also work in reverse: If the signal-generating device outputs a luma signal with no frequency restrictions, some high-frequency components of it can be (mis)interpreted as color by the display - mathematically, the sharper an edge between two pixels is, the more high-frequency components it contains. This has been used intentionally on the Apple II to generate a color video signal with minimal hardware: By aligning its pixels with the color carrier, it can output predictable colors by filling the frame buffer with specific patterns.

Of course such shimmering text is not really wanted in many cases, so devices that generate a composite signal sometimes use measures that minimize them. One method is called a luma trap, a filter that specifically removes the frequencies used for color from the luma signal, which technically means blurring the signal slightly, though in a very specific way. Another is to generate a blurry signal in the first place, so the signal transitions between pixels are slower, reducing the high-frequency components there. Of course this is just another way of blurring and exactly what the shader you use does. It also applies a bit to old game consoles because older IC technologies require more complex circuits to generate signals with very sharp edges and since the console should be cheap, the designer would not invest extra work to do that if it only makes the picture look worse.

Yet another way to visually reduce the color artifacts is to ensure that they change color in every frame, which is what Nintendo does in the NES and SNES: By adding one extra pixel every second frame, the alignment between the luma and chroma signal changes all the time, which changes the color that is generated at the edge, so it looks "cleaner" to the player.

I suspect the Pi's composite output does not do any filtering in hardware to avoid color artifacts, but instead assumes that filtering has already been applied on the software side - after all the original version of the SoC is meant for cheap set-top boxes and has a relatively powerful video chip that can do it in a shader.

[Fudoh]

Can you tell me where to find the shader (tvout-tweaks_sharp.glsl) you're using?
Is it a standard shader included with Retropie and other packages?

[fernan1234]

@Unseen, that's some really interesting stuff. Thank you for sharing it.

[setiawan]

Thanks for the information everyone, it's been very insightful! So what I gather is that the blurring (whether introduced by design, or 'left in' by the hardware (which may in turn be 'by design')) helps to reduce this effect, which is inherently, mathematically existent due to the way colour is encoded in a composite signal.

It's very interesting then, that the effect is also mildly apparent in NewSchoolBoxer's composite screen capture. I guess TV's and capture cards have to decode composite signals in a similar way, and as such are similarly affected. Of course, the blur in your screen capture probably mitigates it a bit, which is why the effect is so mild.

I wonder if it would be possible to modify a Raspberry Pi, so that it creates this blur in hardware. The shader that I'm using does bring me pretty close to the kind of picture produced by consoles, but I can't help but think that the blurring in software may be making a noticeable difference when compared to hardware blur. After all, the software blur creates discrete pixels of intermediary colours (i.e. the darker grey in my shader screenshot) that the CRT then has to try to reproduce (after D/A processing by the pi), as opposed to blurring by hardware which I imagine would create a more continuous/gradual shift between what is internally rendered as contrasting pixels.

For anyone wondering about this elusive shader I'm using, I got it from this repository: https://github.com/Sakitoshi/retropie-c ... sharp.glsl

[fernan1234]

Pretty sure the tvout shaders are included by default, you may just need to explore the different shader folders to find it.

[NewSchoolBoxer]

Insightful indeed. I knew the history and Apple II trick but nothing else. PAL came after NTSC and made no attempt to be compatible with black-and-white.

Glad it was clear I captured footage from real SNES. I didn't mean to omit. I did another Composite to S-Video comparison with Ogre Battle: https://i.postimg.cc/QdXZvtJX/OB-C-Top-S-Bottom.png
If you compare to Legend of Zelda, capital letters are same exact height and 3 pixel width but Composite here has fewer artifacts. With what we learned, I think entire reason is the blue border around the letters in Ogre Battle is light blue versus the dark blue border in Legend of Zelda. The greater white to dark blue contrast causes more rainbowing and you really only see it in Ogre Battle on the rounded parts of the letters. Wizard is visibly sharper in S-Video.

Thanks for the shader link. Is unfortunate the code was script-generated from a compiled source that isn't named. It certainly has the tvout-tweaks.cg code in it but I'm not sure what sharpening is being used or if it clamps luma and chroma: https://github.com/libretro/common-shad ... -tweaks.cg
I found a really good explanation of the clamping on https://blizzz.ovh/rgb/ in RGB Range section. Full Range 0-255 has 2^24 = 256^3 = 16.7 million colors. Limited Range, applying to Composite/S-Video is 16-235 for 220 steps at 220^3 = 10.6 million colors. I was wrong in that 16 is true black and 235 true white. The greater granularity of YCbCr/Component and RGB is only a minor gain on consumer grade television. I noticed MS Paint caps me at 0-239 hue and 0-240 saturation and luma so not sure if Limited Range is a codified standard or best practice.

Seems you can pull Composite video directly from a pin on the Pi to avoid noise sources from audio circuit. Not sure how it gets generated: https://www.retrorgb.com/rpi240p.html
Hardware blur has to exist to some extent by analog signal moving along a tightly packed circuit board. Way we added noise in college lab was to put laptop power cord on top of breadboard. I agree with you about wanting a hardware solution to simulate console better even though DSP filtering in software is generally superior.