DExx-vd_isl video digitizer add-on for Intel FPGA dev boards

The place for all discussion on gaming hardware
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

DExx-vd_isl video digitizer add-on for Intel FPGA dev boards

Post by marqs »

Decided to spin off this topic from OSSC Pro thread since despite common HW & features, DExx-vd_isl is a different board which anyone with some soldering skills can build already today. The board was originally designed to be used in early stage of OSSC Pro development, but it is possible to backport features from OSSC Pro project to this system now that development has transitioned to the dedicated OSSC Pro HW.

Currently supported FPGA dev boards include DE2-115 and DE10-Nano. PCB files and code is found on github. Documentation is scarce at this point, unfortunately, as the board has been used just internally so far.

Below is a short feature list and picture of preliminary version 1.3 of the board. Finished v1.3 version which is on github does not require the rework which is apparent in the picture.

* Digitizes analog video sources in RGBS, RGBCS (TTL csync), RGBHV and YPbPr formats connected to SCART connector
* Up to ~100MHz sampling clock support (e.g. 1280x1024@60Hz). Output up to ~185MHz (e.g. 1920x1440@60Hz)
* Analog and digital audio inputs via SCART and toslink (SPDIF not connected to HDMITX on DE10-Nano)
* Character OLED and OSD as UI. IR control using OSSC remote control.

Image
User avatar
donluca
Posts: 852
Joined: Sat Feb 28, 2015 8:51 pm
Location: Italy
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by donluca »

Now, something I feel is really missing is a good, low lag digitizer so that instead of messing around with scalers, extra monitors and what not, I can just use my PC/Mac to play old RGBs consoles (and maybe even stream them).

I'll be following this thread closely.
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by fernan1234 »

donluca wrote:Now, something I feel is really missing is a good, low lag digitizer so that instead of messing around with scalers, extra monitors and what not, I can just use my PC/Mac to play old RGBs consoles (and maybe even stream them).

I'll be following this thread closely.
Agreed! Sometimes simple is best, and really the most essential thing we need is good ADC.

Any chance of a slightly revised board with a DE-15 connector instead of SCART, eventually?
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

A kind person recently provided me with a bare PCB, so I assembled a vd_isl board and thought I'd share a bit of footage and draw some comparisons to the OSSC Classic.

New features over the OSSC Classic include:
• Adaptive line multiplication. This always outputs a standards-compliant resolution, significantly improving display compatibility, while framelock and less than 40 scanlines of latency is maintained.
• 240p ➤ Line6x, 480p ➤ Line3x (1920x1440), in adaptive line multiplication mode
• Noninterlace restore mode. This can restore the original 240p format of some arcade ports that output an idiosyncratic "240p via 480i", examples include Sengoku Blade and Dragon Blaze on the PS2
• Indications are that the H-PLL of the ISL51002 video ADC is more robust than the one from TVP7002 used by the OSSC Classic, improving display compatibility to some degree in some problematic cases like the NES/SNES jittery sync

Some photos of the device (click for larger version):

ImageImage
ImageImage

For the OLED display, instead of the NHD-0220CW-AW3 which will be used by the OSSC Pro, I used the NHD-0216CW-AW3 as I couldn't easily get hold of the former one at that time. The latter provides 2 rows and only 16 columns instead of 2 rows and 20 columns; this means 0-4 letters will be cut off depending on the current selection (e.g. 'AV1-3 video in opt.' becomes 'AV1-3 video in o'), but here that isn't a big deal to me. There are also other illumination colors available, the 'X' in the '-AX3' suffix of the part number denotes the color. The display is really pleasant to look at (looks even better than the photo is able to capture), the increased contrast improves readability compared to the original OSSC LCD display and underlines the 'pro' aspect of the upcoming device.

Cost for parts without the bare PCB was around 90€ (around 110 USD currently), shipping excluded – the ISL51002 being the most expensive part by far with 44€. I also ordered a small amount of bare PCBs, which amounted to 10.80 USD per piece.

With SW1 you decide which SCART pin is connected to the SOG0 pin on the ISL; position 'CSYNC' connects pin 20, 'SOG' means pin 11 is connected – the latter is also needed for YPbPr. The sync slicer behind the SOG pins accepts 'bare' CSync, sync on green and sync on luma, I have yet to test if composite video works as well. Make sure there is no SD card inserted that's bootable by HPC, otherwise this may interfere with configuration/operation of the FPGA.

With the OSSC Classic, both the monitor and capture card used for these tests cannot handle the SNES jitter in Line5x mode, the latter even has difficulties with Line4x mode. With DE10-vd_isl, they process the 5x output signal flawlessly in both line multiplication and adaptive line multiplication mode. :) The jitter is still slightly visible near the top edge of the screen in LM and ALM mode as it temporarily affects the H-PLL, if you look for it.

Some lossless screenshots, I used default settings everywhere except where otherwise noted, OSSC FW version is 0.88a, DExx-vd_isl FW version is 0.41:

SFC, 240p, Super Mario World

DE10-vd_isl, Line4x ALM, Generic 4:3 (1920x1080) | cropped | cropped and R,G,B gain set to 290
DE10-vd_isl, Line5x ALM, Generic 4:3 (1920x1080)
Phase looks off, the current FW doesn't provide an option to manually adjust sampling parameters:
DE10-vd_isl, Line5x ALM, SNES 256col. optimized (1920x1080)
OSSC, Line4x, Generic 4:3 (1280x960) | cropped

PS1, 240p, Crash Bandicoot 2

OSSC, Line5x, Generic 4:3 (1920x1080) #1
OSSC, Line5x, Generic 4:3 (1920x1080) #2
DE10-vd_isl, Line5x ALM, Generic 4:3 (1920x1080) #1
DE10-vd_isl, Line5x ALM, Generic 4:3 (1920x1080) #2

Crash Bandicoot 2 uses 512 dots per line, but the PSX 512col. mode is currently blurrier than Generic mode:
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #1
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #2
OSSC, Line5x, 512x240 opt. (1920x1080) #1
OSSC, Line5x, 512x240 opt. (1920x1080) #2

Line4x ALM combined with PSX 384, 512 and 640col. modes is currently broken:
DE10-vd_isl, Line4x ALM, PSX 384col. optimized (1920x1080)

GC, 480p via GCVideo & HDMI-to-YPbPr DAC, Super Mario Sunshine

GCVideo hardware is a Carby, FW version is 3.0e, all settings are on their default values, OSSC has Pre-ADC-Gain set to 9 and Sampling phase set to 315deg for the DTV sampling pics only. DAC is a Portta (this one).

Since you can use a DAC+OSSC combination to reduce the degree to which your display has to scale the picture, thereby increasing overall sharpness, I was especially excited for this. The OSSC only gets you to 960p however. With 480p Line2x ALM, 1920x1080p is output, and the resulting picture looks gorgeous on my 1080p monitor. The OSSC Pro will then further be able to do away with the DAC and ADC steps thanks to its digital A/V input. Unfortunately I couldn't yet test 480p Line3x (where the picture will then also vertically fill the screen - more or less, depending on the game), as the displays I currently have access to only accept up to 1920x1200, and while the Magewell USB Capture HDMI Plus should on paper be able to record that timing, it doesn't for some reason I have yet to find out.

In an attempt to mimic how a typical 1080p display would process and show the signal, I also upscaled each respective screenshot to 1080p with GIMP using bilinear interpolation. The results are very close to what it looks like on my monitor.

R= & GCVideo & GCVideo ➤ DAC ➤ OSSC Passthrough & GCVideo ➤ DAC ➤ OSSC Line2x & GCVideo ➤ DAC ➤ DE10-vd_isl Line2x ALM R= 480p in & [url=https://6t8k.de/images/mariosunshine_gcvideo_480p_1x_720x480p_in.png][link][/url] & [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_1x_dtv_720x480p_in.png][DTV sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_1x_vesa_720x480_in.png][VESA sampler][/url] & [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_2x_dtv_720x480p_in.png][DTV sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_2x_vesa_720x480p_in.png][VESA sampler][/url] & [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_de10-vd_isl_480p_2xalm_generic_1920x1080.png][generic sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_de10-vd_isl_480p_2xalm_dtv480p_4_3_1920x1080.png][DTV sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_de10-vd_isl_480p_2xalm_vesa640x480_60_1920x1080.png][VESA sampler][/url] R= 1080p display simulation & [url=https://6t8k.de/images/mariosunshine_gcvideo_480p_1x_720x480p_in_scaled.png][link][/url] & [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_1x_dtv_720x480p_in_scaled.png][DTV sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_1x_vesa_720x480p_in_scaled.png][VESA sampler][/url] & [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_2x_dtv_720x480p_in_scaled.png][DTV sampler][/url] [url=https://6t8k.de/images/mariosunshine_gcvideo_ypbpr_ossc_480p_2x_vesa_720x480p_in_scaled.png][VESA sampler][/url] & output is already at 1920x1080 :]


PS1, 240p test suite, color bar pattern

Colors are a bit too bright on current default settings, crushing the brightest sections:
DE10-vd_isl, Line5x ALM, Generic 4:3 (1920x1080)
DE10-vd_isl, Line5x ALM, Generic 4:3 (1920x1080) R,G and B gain adjusted, values are exemplary, not necessarily optimal


If you have any questions or suggestions, ask away :)

A few more tests following soon.
Last edited by 6t8k on Mon Dec 14, 2020 8:38 pm, edited 3 times in total.
User avatar
donluca
Posts: 852
Joined: Sat Feb 28, 2015 8:51 pm
Location: Italy
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by donluca »

I mean... if we're digitizing to a PC there's absolutely no need of any kind of processing on the board whatsoever (which is likely to add lag as well).

Once it's digitized you can do everything in software on your PC/Mac and the possibilities become literally endless.

I just want a good, cheap, "open source" digitizer compatible with win/*nix/macOS so that I can grab a 240p/480i signal and get it to my PC screen with the lowest amount of lag possible and THEN scale it to whatever I feel like.
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

donluca wrote:I mean... if we're digitizing to a PC there's absolutely no need of any kind of processing on the board whatsoever (which is likely to add lag as well).

Once it's digitized you can do everything in software on your PC/Mac and the possibilities become literally endless.

I just want a good, cheap, "open source" digitizer compatible with win/*nix/macOS so that I can grab a 240p/480i signal and get it to my PC screen with the lowest amount of lag possible and THEN scale it to whatever I feel like.
You can always output in passthrough mode which has <2 scanline latency, and if you don't like HDMI, DE10-Nano has Gbps ethernet which could be used to push out video stream (at least up to 480p uncompressed). That'd of course need someone to create required logic and accompanying SW on PC/Mac.
User avatar
donluca
Posts: 852
Joined: Sat Feb 28, 2015 8:51 pm
Location: Italy
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by donluca »

The ethernet solution sounds good, hopefully someone will step up to the task.

I really feel that after the OSSC we need something similar specifically tailored to digitize RGBs/Component signals to a PC so that we can use our PCs to acquire original hardware video output without needing an additional monitor and, as I said before, it would be an amazing solution for streamers.
paulb_nl
Posts: 340
Joined: Sat Feb 20, 2016 5:05 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by paulb_nl »

donluca wrote:I mean... if we're digitizing to a PC there's absolutely no need of any kind of processing on the board whatsoever (which is likely to add lag as well).

Once it's digitized you can do everything in software on your PC/Mac and the possibilities become literally endless.

I just want a good, cheap, "open source" digitizer compatible with win/*nix/macOS so that I can grab a 240p/480i signal and get it to my PC screen with the lowest amount of lag possible and THEN scale it to whatever I feel like.
This doesn't make sense. What you want is a capture card and that will always add lag. Even if the capture card has zero lag, the Windows desktop adds 3 frames of lag to everything.

So if you want the least amount of lag then you need OSSC+monitor.
User avatar
Fudoh
Posts: 13015
Joined: Mon Mar 06, 2006 3:29 am
Location: Germany
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by Fudoh »

@6t8k:
• Noninterlace restore mode. This can restore the original 240p format of some arcade ports that output an idiosyncratic "240p via 480i", examples include Sengoku Blade and Dragon Blaze on the PS2
would you be able to provide a short video capture (e.g. 960p, 60fps, 10 sec) for this one ?
User avatar
donluca
Posts: 852
Joined: Sat Feb 28, 2015 8:51 pm
Location: Italy
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by donluca »

paulb_nl wrote:This doesn't make sense.
Nope, it just doesn't exist yet and still, it would be a wonderful devices for gamers and streamers alike.
What you want is a capture card and that will always add lag.
Current capture cards add lag. This doesn't need to always be the case. We just need the correct device.
Even if the capture card has zero lag, the Windows desktop adds 3 frames of lag to everything.
So all emulators running in windowed mode have 3 frames of lag? This sounds like news to me, you might want to share your sources on that.
User avatar
Extrems
Posts: 540
Joined: Sat Jan 30, 2016 5:01 pm
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by Extrems »

Even then, Direct3D9Ex hardware overlays still work in Windows 10.
User avatar
orange808
Posts: 3196
Joined: Sat Aug 20, 2016 5:43 am

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by orange808 »

Fudoh wrote:@6t8k:
• Noninterlace restore mode. This can restore the original 240p format of some arcade ports that output an idiosyncratic "240p via 480i", examples include Sengoku Blade and Dragon Blaze on the PS2
would you be able to provide a short video capture (e.g. 960p, 60fps, 10 sec) for this one ?
+1
We apologise for the inconvenience
User avatar
orange808
Posts: 3196
Joined: Sat Aug 20, 2016 5:43 am

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by orange808 »

donluca wrote: So all emulators running in windowed mode have 3 frames of lag? This sounds like news to me, you might want to share your sources on that.
Sure, but we are making calls natively. Importing video is more tricky.

I don't know how we would import and get each line into a graphics API quicky.
Blurbusters has a frame slice beam racing implementation, but that's not guaranteed to remain compatible with driver updates is it? Will Nvidia and AMD let us have direct access to the raster? Does ultra low latency already do that? How much latency does Nvidia/AMD hardware scaling add? Do the AMD/Nvidia cards perform hardware scaling without a full internal frame buffer? A frame buffer is a frame buffer, rather we directly implement one or not. Are the cards/drivers using an optimised scaling algorithm? If they are using a simple full frame buffer, it's never going to be under a frame of latency. I've never seen any scientific tests on Nvidia or AMD hardware scaling.

Do I have to build an entire frame for the GPU using my graphics API and wait another entire frame for the GPU to buffer and scale the video?

Don't sleep on audio, either. If you recall, audio latency was a often bigger issue than video for Byuu with SNES emulation.

https://higan.readthedocs.io/en/stable/guides/drivers/

There's so many "moving parts" with Windows.
We apologise for the inconvenience
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by fernan1234 »

orange808 wrote:There's so many "moving parts" with Windows.
May we hope for any better with Linux/Mac?
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

There is also the issue of refresh rate if you want to minimize latency in such setup. Basically the SW should run in exclusive fullscreen and adjust monitor refresh rate to exactly match input to avoid buffering. In practice that doesn't work without VRR, not even for fixed mode sources. Scaling is another challenge since most APIs require full frame to be available and audio is a can of worms as well.
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

DE10-Nano image has been updated to v0.42:

* fixed previously reported PSX optimal mode issues
* added support for 48kHz audio in addition to default 96kHz
* added H-PLL loop gain option, thanks to paulb_nl for noticing it on datasheet (increasing this should remove visible jitter with SNES)

The 48kHz option is untested as I'm using older board revision that has audio MCLK generated in a different way.
jotheripper
Posts: 10
Joined: Thu Jan 02, 2020 8:55 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by jotheripper »

thank you @marqs
with the new fw v0.42 and 48khz, i finally have sound on my both tvs.
my next wish would be the scaler-mode and scanline-support. :D
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

Thanks a lot for the update.
marqs wrote:* fixed previously reported PSX optimal mode issues
Yes, ALM Line4x PSX 384, 512, 640col. are fixed:
DE10-vd_isl, Line4x ALM, PSX 384col. optimized (1920x1080)

ALM Line5x 512col. only looks right when I switch to it like this: 'MD 320col.' → ... → 'PSX 512col.':
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #1
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #2

When I switch to it like this: 'PSX 512col.' ← ... ← 'N64 320col.', it's less sharp:
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #1
DE10-vd_isl, Line5x ALM, PSX 512col. optimized (1920x1080) #2

I didn't further look into it yet, but I'd wager it's caused/influenced by the ISL's automatic sampling phase adjustment (at least that gets activated for every preset video mode currently). I've previously noticed phase to be somewhat unintuitive to predict.
Edit: quickly did a downgrade to confirm that the inconsistent sharpness was exactly like that before, which it was. But it's still still [sic] suboptimal :-)
marqs wrote:* added support for 48kHz audio in addition to default 96kHz [...] The 48kHz option is untested as I'm using older board revision that has audio MCLK generated in a different way.
Both sample rates work with the monitor seen above and my capture card. Independently from that, what was the rationale for 96kHz in the first place?
marqs wrote:* added H-PLL loop gain option, thanks to paulb_nl for noticing it on datasheet (increasing this should remove visible jitter with SNES)
The effect was most visible in LM 5x 256x240opt. mode. Setting this to the value of 1 in my case completely removes it everywhere for all practical intents and purposes. Very nice!
Fudoh wrote:
6t8k wrote:• Noninterlace restore mode. This can restore the original 240p format of some arcade ports that output an idiosyncratic "240p via 480i", examples include Sengoku Blade and Dragon Blaze on the PS2
would you be able to provide a short video capture (e.g. 960p, 60fps, 10 sec) for this one ?
Of course! Results of that mode also interest me very much.

It doesn't seem to work quite right yet though. My observation so far is that the picture still vertically bobs like it does without it. PS2, Dragon Blaze, 480i (PAL version set to its native NTSC/60Hz mode):
240p restore mode (720x240) ➤ YouTube (scaled to 1080p using the capture card for better encoding by YT)
Dint+L4x mode (1920x1080) ➤ YouTube (for comparison)

I've also fed the 240p restore/720x240 signal into the Portta HDMI-to-YPbPr DAC and plugged that into an Olympus OEV203. It bobs just like it does when I plug the PS2 directly into a CRT as I've previously shown here (best watched in the highest quality setting and frame-stepped through using the , and . keys).

Also in my previous post, 'HPC' is 'HPS', and 'LCD display' is 'LCD' ._.
User avatar
orange808
Posts: 3196
Joined: Sat Aug 20, 2016 5:43 am

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by orange808 »

@6t8k
Could you also try testing 240p restore using PS2 Fantasy Zone Collection? That game lets you toggle the flicker filter On/Off in the menu when using 480i. You can also toggle between 240p and 480i for direct comparison between 240p and 480i (240p restore).
We apologise for the inconvenience
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

6t8k wrote:I didn't further look into it yet, but I'd wager it's caused/influenced by the ISL's automatic sampling phase adjustment (at least that gets activated for every preset video mode currently). I've previously noticed phase to be somewhat unintuitive to predict.
Edit: quickly did a downgrade to confirm that the inconsistent sharpness was exactly like that before, which it was. But it's still still [sic] suboptimal :-)
The automatic phase adjustment is not very effective on these optimal modes where multiple samples are taken for each source pixel (7 in case of PSX 384x240). Once the manual phase option is added, MSBs will select the sample to be used (instead of always using the middle one like now) while LSBs will change the actual ADC sampling phase.
6t8k wrote:Both sample rates work with the monitor seen above and my capture card. Independently from that, what was the rationale for 96kHz in the first place?
I had constant 24.576MHz MCLK wired for the audio ADC in an earlier board revision which results to 96kHz sampling. V1.3 routes MCLK via clock generator that allows dividing that by 2 for 48kHz.
6t8k wrote:The effect was most visible in LM 5x 256x240opt. mode. Setting this to the value of 1 in my case completely removes it everywhere for all practical intents and purposes. Very nice!
You could also try if compatibility is degraded on the higher values since higher loop gain can result to larger instantineous change in output clock too.
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

@orange808: I don't have that disc unfortunately. :[ Wouldn't the DE10-vd_isl also just switch to 240p proc. when selecting 240p in that menu? But I can test PAL Sengoku Blade (i.e. Tengai) and G.Darius from the Taito Memories Gekan collection later. We weren't sure whether the latter one would look right due to the filtering used, weren't we?

It's probably just that lm_deint_mode=1 (=noninterlace restore) still uses an additional, deliberate vertical offset, since, if I understood it correctly, the only missing ingredient you need if you have bob deinterlacing is to omit the additional deliberate offset.
marqs wrote:
6t8k wrote:The effect was most visible in LM 5x 256x240opt. mode. Setting this to the value of 1 in my case completely removes it everywhere for all practical intents and purposes. Very nice!
You could also try if compatibility is degraded on the higher values since higher loop gain can result to larger instantineous change in output clock too.
Will further experiment with this setting/higher values and report if I find something to that effect.

Thanks for clearing up the other two, I had suspected something along those lines.
User avatar
Fudoh
Posts: 13015
Joined: Mon Mar 06, 2006 3:29 am
Location: Germany
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by Fudoh »

It doesn't seem to work quite right yet though. My observation so far is that the picture still vertically bobs like it does without it
thanks for the demo videos. There is no adjustable vertical offset compensation yet (I guess). This way the option will probably work on a bunch of titles, but not on the rest, since the programmers hardly stuck to any standard when doing "raw" 480i from 240p titles.
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

Yep, that option doesn't exist (yet).
User avatar
orange808
Posts: 3196
Joined: Sat Aug 20, 2016 5:43 am

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by orange808 »

Following up Nvidia GPU scaling, Nvidia RTX cards appear to use a proper optimised dynamic frame buffer. It's not a lazy fixed full frame buffer.

I chained the Time Sleuth through a RTX 2080 machine with a capture card. I used 480p on the Time Sleuth and forced the capture preview software to VGA using compatibility settings (this forces all video output from the computer to VGA while the program is running). When I compare GPU scaling of the video capture VGA full screen display and my display's VGA scaling, the lag is minimal.

It doesn't make me much more confident about performing video processing on Windows, but it's good news for gamers.
We apologise for the inconvenience
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

This is how a ToaplanV2 game that has the sync quirk looks through the DE10-vd_isl: YouTube. The capture card has problems picking it up.

On the monitor, the picture is perfectly stable in ALM mode with Line3x and lower, and in LM mode with Line4x and lower. The capture card has dropouts universally. When H-PLL loop gain is set to 1, the skew gets much better, it's not perfect but I'd consider the game playable just fine this way. The capture card completely bails out then though (I was setting it to 1 the moment the video switches to a constant 'Unsupported Signal' message), but this so far only happened in this scenario – other, unproblematic signals worked with the capture card using values >0 so far. With the value of 2 and up I'm getting an intermittent picture on the monitor. If I e.g. change the line multiplication factor while H-PLL loop gain is >0, the monitor fails to resync until I set it to 0 again. I could not (noticeably) improve the results using the AV1-3 sync opt. menu. Playing it with H-PLL loop gain set to 1 might work well in indvidual cases, but the scaler/non-framelocked route will surely be the go-to option for affected-and-unpatched games.


GBI-hf 360p/22.5kHz Line2x/3x works out of the box - without having to adjust any sync-related options and without any compatibility problems, at least in my case. There is no 384/400p proc. for ALM, so that is skipped and pure LM steps in, which seems sensible from GBI's point of view. With Line2x/3x 240x360 opt., there are already GBI-aware presets :) Here's a screenshot from 160p test suite's checkerboard pattern in Line3x 240x360 mode:
Spoiler
Image
As long as the Adv. timing menu is not available yet, it'll have to stay this way, but in a pinch there's the equivalent generic presets until then which while blurrier, are at least uniform.


The Taito F3 with its constant even field indicator also works offhand, no workaround / HW modification / extra HW necessary: YouTube.

I tinkered a bit with the H-PLL loop gain setting in combination with some SNES games. With the value of 3 (the maximum), but not lower, the monitor sporadically loses sync/the picture (every ~10-30s for a few seconds), sometimes audio keeps playing. The capture card seems completely unaffected by this and picks up the signal just fine. Changing the line multiplication factor etc. when H-PLL loop gain is >0 is not an issue.

Sengoku Blade/Tengai and G.Darius on PS2 expectedly didn't bring any news on the table with regards to noninterlace restore. The latter has a softening/flicker filter so it probably won't look as good if/when noninterlace restore works with it.
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

The results from ToaplanV2 indicate that the internal clock can be used at least for sampling even if output clock has to be generated externally. That makes things easier and more robust compared to a case where external clock would need to be used for sampling too.

About noninterlace restore, I just added an option for alternative even field offset. The different configurations and their effect is summarized in the table below:

R= Resulting even field offset & LM deinterlace mode & NI restore Y offset R= 0 input lines & Noninterlace restore & 0 R= 0.5 input lines & Bob & N/A R= 1 input lines & Noninterlace restore & 1

As mentioned earlier, the feature only works in full effect with games that don't use flicker filter.
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

Noninterlace restore works now with Dragon Blaze and Sengoku Blade/Tengai when 'NI restore Y offset' is set to 1. :)

Dragon Blaze (PS2/480i), ALM Dint+L4x Generic (1920x1080), through capture card: YouTube
Dragon Blaze (PS2/480i), ALM 240p restore mode Generic (720x240), through HDMI-to-YPbPr DAC into PVM 2053-MD: YouTube

I managed to borrow Fantasy Zone Complete Collection in the short term, it needs 'NI restore Y offset' set to 0 (had I tried this instead of Dragon Blaze earlier, I would have seen that this worked already):

Fantasy Zone II DX (PS2/240p), ALM L4x Generic (1920x1080): YouTube | lossless screenshot
Fantasy Zone II DX (PS2/480i, deflicker=off), ALM Dint+L4x Generic (1920x1080): YouTube | lossless screenshot

Ideally they would look identical, and they almost do; the restored version is very slightly sharper even due to better alignment, which may be related to automatic phase adjustment.

In the case of Dragon Blaze and Sengoku Blade/Tengai, the first and last active input lines effectively flicker in line with the input refresh rate (actually the total input lines bob according to 'NI restore Y offset'). I suppose this could be hidden with a mask feature, although a little bit of picture information would get lost this way. The first active line is a copy of the line below it, the last line contains unique picture information.

For G.Darius on the Taito Memories Gekan collection, noninterlace restore deinterlacing is not useful, unfortunately, due to the filtering/line mapping it uses. The 0.5 input lines offset of bob deinterlacing is a better fit:
G.Darius (PS2/480i), ALM Dint+L4x Generic (1920x1080): YouTube

To elaborate a bit, the ALM 240p restore mode (720x240) has the deinterlacing method hard-coded to noninterlace restore, while the other line processing modes that incorporate deinterlacing are influenced by the 'LM deinterlace mode' option - i.e. they can combine noninterlace restore with line multiplication. Contrary to the overview table here, the pure LM modes can also use noninterlace restore deinterlacing.


Two other observations:
- Pure LM Deint+L4x mode is currently faulty: when I switch to it like this: 'Line3x (laced)' → 'Deint + Line4x', both my monitor and capture card show 'Unsupported signal', when I switch to it like this: 'Deint + Line4x' ← 'Passthru', both show this.
- H frequency display shows twice the actual rate with interlaced input signals (at least I can confirm this for 525i and 625i). For example, a PAL PS2 idling in the system menu causes DE10-vd_isl to show 31.25kHz, while an oscilloscope measurement results in ~15.62kHz, which the OSSC Classic also shows.
marqs wrote:The results from ToaplanV2 indicate that the internal clock can be used at least for sampling even if output clock has to be generated externally. That makes things easier and more robust compared to a case where external clock would need to be used for sampling too.
It really is a delight that the ISL has such a robust H-PLL +accompanying settings. When I experimented with the respective register/s on the TVP, I couldn't really improve the situation at all, and could play whack-a-mole on top of a poor basis at best.
User avatar
Fudoh
Posts: 13015
Joined: Mon Mar 06, 2006 3:29 am
Location: Germany
Contact:

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by Fudoh »

beautiful results - thanks for the captured clips and snapshots!
User avatar
6t8k
Posts: 496
Joined: Wed Aug 14, 2019 2:44 pm

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by 6t8k »

Some 1920x1440p footage – using RGB24 pixel format I'm getting ~50fps with non-scaled 1920x1440 output, and so far haven't managed to increase that by choosing a different (i.e. chroma-subsampled) pixel format. Letting the capture card downscale the 1920x1440 input to e.g. 1920x1080 or 1600x1200 facilitates capturing RGB24 at 60fps. The 30fps recordings below do not really set themselves apart from hypothetical 60fps recordings of the same pixel dimensions in the case of the games chosen, as both seem to render gameplay at 30fps. That's just some side issues specific to my setup though – the intent is to demonstrate that DE10-vd_isl's ALM 1920x1440p output can work reliably. Generic sampling has been used throughout:


PS1 / Crash Bandicoot, 240p Line6x ALM (1920x1440): 1920x1440p30 recording | 1600x1200p60 recording | lossless screenshot
During the 1600x1200p60 recording, starting with Line2x (720x480) I increment the 240p processing mode until Line6x is reached, while the capture card's output resolution stays the same.

GameCube ➤ GCVideo ➤ HDMI-to-YPbPr DAC / Super Mario Sunshine, 480p Line3x ALM (1920x1440): 1920x1440p30 recording | lossless screenshot
No audio because the RCA-to/from-SCART adapter I'm currently using cannot passthrough audio, and I didn't separately hook the RCA audio out from the DAC (or audio from the GC's MultiAV connector) into the 3.5mm microphone jack on the Magewell or the PC for this test. Routing audio around DE10-vd_isl would technically have distorted the result of the test too.

PS1 / 240p test suite, 240p Line6x ALM (1920x1440): 1600x1200p60 recording
To represent content rendered at 60fps.
User avatar
marqs
Posts: 1034
Joined: Sat Dec 15, 2012 12:11 pm
Location: Finland

Re: DExx-vd_isl video digitizer add-on for Intel FPGA dev bo

Post by marqs »

I've now spent some with SiI1136 HDMI TX daughtercard connected to DE2-115, and managed to get it to run at 270MHz from Cyclone IV PLL. At that frequency only the test pattern can be realistically output as anything more soptisticated is too much for the old FPGA model. Basic 12-bit H/V counters already make it hard to meet timing across all conditions. I/O timing at high frequency was another challenge since there is no adjustable input clock delay on SiI1136, and very limited adjustment on FPGA side. Cyclone V GX Starter Kit should be much better platform for this experiment so I may need to acquire the board and add support for it to get a better idea if it makes sense to consider SiI1136 for OSSC Pro. HSMC IO level is fixed to 2.5V on that kit (3.3V could be selected on DE2-115) which is not optimal, but should still work. On a positive note, I managed to find full programming guide for SiI1136 so much less guesswork is now needed compared to using the partial documentation provided by Terasic.
Post Reply