This is a cross post from Neo-geo.com, but thought I would share my findings here as well:
Further analysis:
Tested composite video (non RGB) with stereo audio:
Audio seems similar with the buzzing. Much of the buzzing seems to coincide with access to the SD card. During SD card access I hear an increase in background audio noise.
Composite video sadly seems to have the same kind of interference in that I am seeing in RGB mode. Obviously whatever is affecting RGB output is also affecting regular composite video.
I plugged my RGB modded white PCE into the SSDS3 for some more testing.
Comparing it's internal RGB amp to the SSDS3 shows quite a big difference. The internal amp had none of the wavy noise nor the diagonal interference.
Here is a quick video capture of the RGB output between the SSDS3 and an internal RGB amp.
https://www.youtube.com/watch?v=xmb2JPiu7e4
My thoughts: The RGB amp has good sharpness and color rendition. It's however let down by substantial noise in the video.
I am leaning towards this not being an issue with the RGB amp design (though broken composite sync is a different issue entirely) and more towards some kind of grounding or voltage leaking issue. Or some other kind of noise generating error.