RGB over Ethernet?

The place for all discussion on gaming hardware
Post Reply
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

RGB over Ethernet?

Post by DirkSwizzler »

Hello people of Shmups! I'm fairly new to retro gaming and rgb video. Only becoming interested since January of 2017.

I've been seeing the problems of the masses with fresh eyes. And one of the things that fascinates me is all the work it takes just to manage a good scart setup outside the consoles themselves.

So far, I've been firmly avoiding HDMI upgrade kits because:
1. Those don't scale to all my console equally. So I would still need a scart setup for wide coverage.
2. it seems like every kit is trying to make video tradeoffs for me that I might not like.
3. 1080p seems like it's on the cusp of being obsolete, and video standards in general have a much shorter life than these consoles will. As the Bob at RetroRGB mentioned a few times, it seems a bit weird to make permanent modifications to a console that will probably not be used for the lifetime of the system.

HDMI kits do have 1 property that I do really want though. A noise-free cabling solution. Digital signals over mass-produced shielded cable would be a huge win in any setup.

I've been ruminating about Bob's desire to have a "fan spec" connector for consoles that don't have any rgb-capable pinout. One of the points he brings up is that any mass produced connector with good cables and switches available is probably going to be mistaken for something else in the future. Not good. He also wanted the "fan spec" to be something rounded to make console body modification as simple as drilling a hole that's easy to get right.

I think I have a possible solution. For the purposes of illustration, assume we're talking about a SNES with the A/V multi-out.

Essentially, a small processing device like an arduino or a pi should be able to be hooked up to the RGB (and possibly a sync pins) of the multi-out. Digitize the samples to a byte per pin, and wrap that data in a proper ethernet broadcast packet.

Then, another device that only needs to be on the same ethernet network (assume that you have an isolated network just for this purpose with no other traffic) can listen for broadcast packets of rgb data and convert them back into a scart pinout.

You've instantly made cable length a non-issue. And switching (assuming 1 broadcast device at a time) is completely trivial. RJ-45 ethernet cable is so entrenched that it's not going to completely die in the forseeable future even if it's superceded by newer tech. So the hardware will be well supported for a long time.

I did some ballpark math and I believe a 480p60 signal would only need around 50 MB/s to transmit all it's raw data. Which is well within the realm of consumer gigabit switches these days. Although I'm not sure if there are limitations on what an arduino or pi can encode.

The connector would technically be still within the realm of it's intended purpose, so there's no worry about damaging the console or other equipment if they're connected. And RJ-45 is close enough to round for an easy installation in my opinion.


Just some thoughts I wanted to bounce off the smart people around here.
User avatar
Guspaz
Posts: 3136
Joined: Tue Oct 06, 2015 7:37 pm
Location: Montréal, Canada

Re: RGB over Ethernet?

Post by Guspaz »

tl;dr: basically you either need full gigabit ethernet and suffer from iffy image quality due to misaligned samples, or you use some sort of compression to allow sampling fast enough to make this work.

So, you're basically proposing that the initial ADC stage doesn't actually interpret the video signal in any way, so you avoid all the video sampling and sync issues by just doing pure ADC and letting some video processing device farther down the chain handle it? That could work, although I'd imagine you'd want a really high bandwidth ADC (and DAC at some point) to avoid any noticeable reduction in image quality.

Your solution has the advantage of using an ethernet cable that is doing actual ethernet, so there's less confusion. People will think they can plug it into an ethernet switch because of the connector, and... they'd be right, since it's actual ethernet. And your switcher setup would just involve actual ethernet switches, you'd actually select different inputs on the analog output device (the one with the DAC) rather than the switch itself.

You're going to want more than 8-bit sampling to preserve quality, and you're going to need to oversample by a large amount if you don't want to have to synchronize to the console's pixel clock (the goal is to avoid having to handle video processing in the ADC, right?), so I'm not sure your 50 Mbps estimate is really practical. You'll also need four channels to get RGBS (whatever the source of the sync is).

I started out trying to see how much bandwidth we'd need to recreate a 480p analog signal on the other end with high precision, but was coming up with numbers that would need enterprise-grade ethernet switches, so let's do the reverse and see what we can fit in your 50 Mbps target. Let's also ignore ethernet overhead for the sake of doing some basic estimation (imagine that you're using maximum size ethernet frames so that the overhead isn't much anyhow).

We've got four channels, so right off the bat, we'll divide by four, giving us 12.5 mbps. Now you said 8-bit samples, I don't think that would be enough (I think you'd want more precision than that), but let's go with it for the sake of estimation. That means we divide the bitrate by 8 to get the sampling rate, which is 1.5625 msps. Now let's divide by the scan rate for VGA (480p with some time taken for blanking), which is 31.469 kHz, which gives us 49.7 samples per scanline.

So, errm, that's not going to fly. You'd need 15x that just to get one sample per pixel, but since this approach doesn't want to know anything about the pixel clock or video sync, you would need to oversample several times more than that to accurately recreate the signal, since your samples won't align with pixels and you'd need multiple samples per pixel. And unfortunately, you can't really do that with even a full gigabit per second.

In the end, you're ultimately going to need to sync to the pixel clock on the source so that you can conserve bandwidth and just do one sample per pixel (and even that is way more than 50 Mbps), but that's now opened a whole can of worms. You're either doing hyper-specific mods to individual consoles to get the pixel clock from each one, or you're trying to do proper video ADC, at which point you're basically using most of an OSSC for every input, which would be insanely expensive, and even that doesn't fully solve the problem because you can't tell where the pixels are from the sync signal since you don't actually know exactly how many pixels there are per scanline, and experience with the OSSC "optim" modes shows that even when you *do* know, getting the samples perfectly aligned with the pixels is damned near impossible.
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

Re: RGB over Ethernet?

Post by DirkSwizzler »

My initial estimate was for 3 channels, with sync assumed to be either sync on a color channel or a sync pin that easily boils down to a logic low or logic high.

My math was 24-25 bits per pixel, with 307,200 pixels in a 480p image (I don't have any knowledge of overscan or things in that realm). And accommodating for 60 pixel updates per second.

That should be roughly 3 * 307,200 * 60 = 55,296,000 bytes/sec. A consumer gigabit switch should hit up to the 100 megabyte/sec range in my experience. Maybe there's something I'm missing there.

You're correct that the goal is to ensure the ADC side is super cheap and efficient. I had assumed that the ADC could cheaply work out the sync rate and optimize for that. Maybe that was a bad assumption.
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

Re: RGB over Ethernet?

Post by DirkSwizzler »

Also, if it's all done over ethernet. That immediately builds in 2-way communication to the ADC side. It's possible that the DAC side could ocassionally send back a packet to sample just 1 pin at an arbitrary frequency for a while and work out the sync. Then apply tweaks to the ADC side.

The ADC side also doesn't have to be completely generic. There's no reason it couldn't have a design that specifically accommodates exactly 1 console model. We're already in that realm for some of the SCART cables.
User avatar
Guspaz
Posts: 3136
Joined: Tue Oct 06, 2015 7:37 pm
Location: Montréal, Canada

Re: RGB over Ethernet?

Post by Guspaz »

Sorry, I misread your 50 MB/s as 50 Mb/s. I think the fault here is that you're assuming one sample per pixel, and you can't really do that without access to the pixel clock of the console. For one thing, you don't know how many pixels there are per scanline (the sync signal only tells you when scanlines and frames start) and for another thing, even if you know how many pixels there are per scanline, synchronizing the sampling to the pixel output is really hard (as mentioned, the OSSC can do this, but getting it lined up is damned near impossible).
Wolf_
Posts: 387
Joined: Sat Jun 25, 2016 10:10 pm

Re: RGB over Ethernet?

Post by Wolf_ »

What about usb 3.1 type c? Lots more bandwidth. Pretty idiot proof too. Can be plugged into any other/future usb connector without causing harm and it's even reversible.
viletim
Posts: 550
Joined: Mon Aug 07, 2006 6:44 am
Location: Sydney, Australia
Contact:

Re: RGB over Ethernet?

Post by viletim »

DirkSwizzler,

Only a few consoles have digital RGB available to connect to, others require ADC, others still require more complex solutions (NES). Putting fast ADC into consoles is difficult due to space and power constraints. Custom hardware would be required in the console and on the other end of the data line to convert to usable video (HDMI, RGB, etc). A lot of expense for only slight benefit. I would put the time/money towards making some good analog cables instead.
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

Re: RGB over Ethernet?

Post by DirkSwizzler »

Sorry for the necromancy. But I just had a flash of insight that may prove somewhat interesting. Even if only as a thought exercise.

Specifically trying to digitize the signal super close to the console cheaply. And dropping the whole idea of making a simple standard connector.

Imagine for a second that the timing for when to digitize a pin was both perfect and free. That’d be a significantly cheaper/easier per-console ADC, right?

It seems to me that the sync signal is probably the most robust part of the analog output by one or more orders of magnitude.

What if the the console ADC units sent the unmodified analog sync signal along a separate wire to the DAC buddy box? Then gave the buddy box a wire or two back to the ADC to tell it when to digitize?

You only need one of those DAC buddy boxes (or zero if integrated with the OSSC) so it’s reasonable to make it arbitrarily complex. And the analog sync signal and digital return signals are already robust enough to not care about shielding.

Then you get the rgb digitization right at the console and theoretically don’t have to obsess quite as much about interference and boosting signals.

Just a thought.
thebigcheese
Posts: 707
Joined: Sun Aug 21, 2016 5:18 pm

Re: RGB over Ethernet?

Post by thebigcheese »

What's the latency expected from any of these proposed solutions? Because if you're adding latency, you're adding input lag, and after a while it doesn't matter how universal and wonderful your solution is if I can't play the game.
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

Re: RGB over Ethernet?

Post by DirkSwizzler »

thebigcheese wrote:What's the latency expected from any of these proposed solutions? Because if you're adding latency, you're adding input lag, and after a while it doesn't matter how universal and wonderful your solution is if I can't play the game.
It should be sub-millisecond I believe.
thebigcheese
Posts: 707
Joined: Sun Aug 21, 2016 5:18 pm

Re: RGB over Ethernet?

Post by thebigcheese »

DirkSwizzler wrote:
thebigcheese wrote:What's the latency expected from any of these proposed solutions? Because if you're adding latency, you're adding input lag, and after a while it doesn't matter how universal and wonderful your solution is if I can't play the game.
It should be sub-millisecond I believe.
Hmmm. I had thought that network connections, particularly through routers/switches, add noticeable latency. But maybe I am wrong. Worth considering though.

Some other thoughts I had on this subject recently, having less to do with the connections themselves and more to do with the end goal:
On the one hand, I actually dig the idea of HDMI mods for each console, at least in theory. One of the annoyances with upscalers is that you have one device for all your consoles, but each console has its own set of unique needs, at least if you want perfect scaling. For example, some Genesis games have a different horizontal resolution than others, the SNES has an off-spec refresh rate, etc. So to really make each console look the best, you need a separate profile on the upscaler for each console.

Alternatively, if you have an HDMI mod installed in the console, it can be perfectly tailored to that console's needs. Ultra HDMI for N64, for example, includes deblur, a feature that no other console really needs. In this way, each console takes care of its own special needs. However, as you mentioned, if it's designed now with 1080p in mind, it's likely to go obsolete very shortly. And who is to say that HDMI will even be around that much longer?

The benefit of RGB mods, then, is that, although they use outdated connections, you're at least getting the original signal that can be converted to whatever future format you need. Much like old movies shot on film can easily be converted to 1080p/4k, but TV shows shot in digital require a lot more effort to be cleanly scaled up (and even then won't look that great). In a sense, they are more future proof.

I will say, though, that this is all a little moot since you can get 1080p with HDMI mods now and it should scale pretty nicely to 4k (depending on your TV) since it's an even multiple. Though I'd probably use 720p for the even scaling factor. The next move will probably be to 8k, even though that's pretty useless for home use, which would be awesome because we will finally have an integer scale for 240 AND 480. The biggest downside with HDMI mods right now is that they have very limited availability in terms of the number of consoles supported.
User avatar
DirkSwizzler
Posts: 548
Joined: Fri Apr 28, 2017 8:23 pm
Location: Bellevue, Washington, USA
Contact:

Re: RGB over Ethernet?

Post by DirkSwizzler »

thebigcheese wrote:Hmmm. I had thought that network connections, particularly through routers/switches, add noticeable latency. But maybe I am wrong. Worth considering though.
Definitely not a noticeable amount. Most noticeable latency when sending live video over a network comes from the compression and decompression phases.

This approach is different in that it's just a stream of quantized pixel values. Which should just about fit if you have a dedicated gigabit network just for the stream. So there's no compression needed.
Post Reply