That said, I was still drawn to the analog output problem. It was clear that some amount of capacitance was causing the blur, but the amount of blur varied between different pixel transitions. For example:
Here you can see a pixel transition from zero, to full scale, back to zero (0-31-0). The rise time of the pixel is so slow that it doesn't reach full scale in time (~186 ns), but it drops back to zero very quickly (~7 ns). This variable rise/fall time is what made the problem difficult to solve. Attempting to compensate the slowest pixel transitions would overcompensate faster ones, causing them to overshoot/ring.
To address the variable blur, we have to go one step further up the signal chain. The problem with our previous attempts was the fact that they were all post-filters. Once the signal is buffered, it outputs a uniformly low-impedance signal. Normally that's a good thing in most cases, but not here. A while ago, I theorized that the sensitive, unbuffered DAC output had a variable impedance which depended on the pixel transition. That, combined with the parasitic capacitance in the IC package, formed a low pass filter. If the pixel transition was high impedance, the LPF would be strong (low cutoff frequency, slow rise/fall time). If the transition was low impedance, would have a weak LPF (high cutoff frequency, fast rise/fall time). I searched for active compensation techniques for parasitic capacitances and came across the concept of negative impedance converters. If the parasitic capacitance in the IC was constant, it could be countered by injecting "negative capacitance" back into the signal. I built a prototype to test this theory, with promising results. The second prototype result looks decent enough to publish. Here's a screenshot:
Spoiler
Here are some scope plots showing the changes in the signal based on the amount of capacitance compensated. I used the 240p Test Suite Sharpness test as a reference for some of the slowest pixel transitions, mid-zero-mid (15-0-15) and mid-full-mid (15-31-15):
0 pF compensation:
20 pF compensation:
28 pF compensation:
30 pF compensation (I settled with this):
33 pF compensation:
An overlay between 0 pF and 30 pF compensation:
The compensation doesn't solve all the DAC issues though. There are still glitches that may be intrinsic to the DAC design. You can see some issues where 4-5 bits have to flip at once:
Spoiler
Finally, here's a simplified schematic of my current implementation of the capacitance compensation circuit, with 75-ohm output (the video buffer LPF must be off). It was one of the simplest implementations I could think of:
Note that it is not designed to work with the rest of the stock video circuit like the video encoder and AV multi-out ports. It is completely isolated and should output directly off the mod board. A fair bit of understanding of electric circuits will be needed to implement it to your needs. I think the SNR suffers a little due to the 2.9V reference instead of the original 5V.
Looking forward to seeing if anyone is interested in making a viable mod out of this. I may update the thread with a video playlist of comparisons I have recorded, but that will take a while. I'll continue researching for improvements, pushing the upper limit as far as I can.
Edit: I got a comparison video up. Watch in 4k to minimize color bleed from chroma subsampling: https://youtu.be/Hbk3wf0Ek8k
Edit: Created a playlist and added another video. More comparisons will trickle in over time: https://www.youtube.com/watch?v=Hbk3wf0 ... dR7-EDcP-I