What's my best option for playing PC engine cd games?

The place for all discussion on gaming hardware
Jonesyy
Posts: 23
Joined: Sun Apr 29, 2018 2:48 pm

Re: What's my best option for playing PC engine cd games?

Post by Jonesyy »

fernan1234 wrote:
Jonesyy wrote:Question: If I plan to play a lot of pce cd and sega cd games, would I be better off with a 512gb sd card or a 2-4tb external hard drive for about the same price? Are there any downsides to using an external hard drive vs an sd card?
No disadvantage, other than the external USB drive taking up physical space. The MiSTer framework is capable of loading user files from both the SD card and an external USB at the same time. Game folders for each core are exclusive however, so if you have a TGFX16-CD folder in both the SD card and the external USB, then the MiSTer will default to look for CD files on the internal SD card (or at least this was the case last time I set up my MiSTer).
Great thanks for that.
Would that mean, when I come to set the mister up, I use the external hard drive instead of an sd card? Or do I still use an sd card for the system files etc and just use the external hdd for games folders?
I’m sure I’ll find all this out when it arrives but I’m just asking ahead of time so I know what to expect
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: What's my best option for playing PC engine cd games?

Post by fernan1234 »

Jonesyy wrote:use an sd card for the system files etc and just use the external hdd for games folders?
It's like that by default, but it is also possible to configure the system to run entirely off the external USB, if I'm not mistaken.

The DE-10 Nano comes with a small SD card. I just use that for system files, and USB only for cores' Games folders.
SamIAm MkII
Posts: 20
Joined: Sat Feb 13, 2021 3:54 am

Re: What's my best option for playing PC engine cd games?

Post by SamIAm MkII »

There is something I've wanted to write up for a while now, and I might as well do it here.

A lot of people talk about how using a MiSTer lets you get "the correct color palette" for the PC Engine. Now, if I had a MiSTer myself, I'm pretty sure I would use the composite palette because it is a very reasonable approximation of authentic PC Engine composite video. However, it is not necessarily perfect, and I'd like to explain why.

A quick overview for those who don't know: While most of its contemporaries do a direct conversion of RGB in the analogue domain to create composite video, the PCE does things differently. First, it converts the 9-bit RGB in which the graphics are originally processed to 15-bit digital YUV via a lookup table. This inevitably results in rounding errors for literally every color; however, that's only the first of a few reasons why PCE composite is so different.

Here is a section of the PCE schematic showing the HuC6260 chip and a few relevant components creating the PCE's composite video:

Image

The R-Y and B-Y signals come out of the chip as sine waves which sum together, and Burst and Y are also mixed in to form the complete composite signal. A critical thing to understand is that the colors that result from this do not line up exactly with the 15-bit YUV lookup table. There is some analogue funny-business going on during the encoding phase that we don't completely understand. What we do understand, however, is that if you create a palette based solely on the YUV lookup table, the colors are clearly incorrect. MiSTer's current composite palette is based not so much on the lookup table as on vectorscope readings of an actual PCE system outputting all 512 possible colors in composite video.

Having said all that, let me introduce a couple reasons why this isn't necessarily perfect.

First of all, notice that there are five resistors between Y, R-Y, B-Y, Burst, and the base of the first transistor, which is part of the video amp and is essentially the final mixing point for all signals. NEC seems to have used 1% tolerance resistors in this section, which is nice...however, even a tiny degree of variance results in colors being different from system to system. Without getting too far into the nitty-gritty, suffice to say there are other factors in the video amp itself that create even more system-to-system variance as well.

On the 240p Test Suite discord server, we spent time looking at stock composite video output from over a dozen PCE systems with oscilloscopes and vectorscopes. What we found was that even if you allow for things like automatic gain control and automatic chroma control in the receiver to partially normalize the signal, PCE video can be quite visibly different between systems. 

Here is an example. In the image below, think of the middle horizontal color as the base. The upper horizontal color is the base with a 10% saturation reduction, and the lower horizontal color is the base with a 10% saturation reduction and a 4% brightness reduction. Finally, the vertical stripe on the right is the base with a 10% saturation reduction, a 4% brightness reduction, and a 1% hue rotation (i.e. it's displaced about three degrees on a hue circle).

Image

This is a realistic representation of the degree of variance possible from stock PCE composite even with automatic gain control and automatic chroma control in the receiver. Without those features, the variance becomes even greater.


But that's not all. Here is a shot of the signal coming out of the R-Y pin of the HuC6260 when displaying a color that basically maxes it out. 

Image

Notice that this is not a perfect sine-wave, but more like a funky triangle wave. I regretfully don't have shots of B-Y and Burst, but you can see pretty much the same thing on both of them, and critically, the Burst wave actually has a slightly unique shape to it. 

R-Y, B-Y and Burst together form the 3.58MHz color subcarrier signal that we'll call C, and that gets mixed with Y to create the full composite signal. On the receiver end, your TV or whatever decodes the composite signal by first pulling apart Y and C, and it basically does this with filters. C is separated out by taking what's left after blocking anything that isn't 3.58MHz...or, more specifically, a 3.58MHz sine-wave.

When the triangle-ish waves coming out of the HuC6260 get pushed through the receiver's filtering to isolate C, they get distorted into something more sinusoidal, and this causes a slight phase-delay in them. Since Burst has a different shape, it gets affected a little differently.

How differently? Finally, we come to the key point: That depends on the TV. From the first color TVs in the 50s to the totally digital stuff in the 2000s, there are countless NTSC decoder circuits out there, and even though they have the same goal, the way they might deal with the PCEs non-sinusoidal subcarrier may be different in each case. The phase-delay added to R-Y and B-Y versus the phase-delay added to Burst, which is used as a reference for phase, is quite likely different from TV from TV. 

All of this boils down to two things:

1. There is a very noticeable degree of variance in the stock composite signals of PCE systems.

2. Two otherwise perfectly calibrated TVs may decode composite from the same PCE differently depending on how they do Y/C separation.

Thus, there really is no such thing as a perfect composite palette for the PCE. 


Now, let me reiterate that the current MiSTer palette is a very reasonable approximation; it represents certain tendencies in PCE composite well. I would use it. However, I can (and have!) done a side-by-side comparison with that palette and a real PCE on the same monitor and found differences. This will pretty much be true for any PCE that isn't the one the MiSTer palette was based on, and even that one could appear differently depending on how different the monitor's Y/C separation is from the device used to decode the signal for analysis for the MiSTer palette.

One thing I will say is that there is a possibility to improve the MiSTer palette. This would take an exhaustive analysis of dozens of PCE systems and at least a few different NTSC decoders, until we get enough data to see something like a normal distribution, but it's not impossible. The resulting tweaks would probably be minor, too, but if we're trying to be perfectionist about it, this is the way to go. I hope it happens someday.

Now, let me close this out with a personal opinion: The difference between the RGB palette and the composite palette is not that huge. It's great to have the composite palette preserved and available, but in most cases, your eyeballs just aren't going to care. Don't let it trip you up and diminish your enjoyment of the games.
Last edited by SamIAm MkII on Tue Nov 30, 2021 3:37 am, edited 1 time in total.
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: What's my best option for playing PC engine cd games?

Post by fernan1234 »

First of all, that's a really neat write up.
SamIAm MkII wrote:All of this boils down to two things:

1. There is a very noticeable degree of variance in the stock composite signals of PCE systems.

2. Two otherwise perfectly calibrated TVs may decode composite from the same PCE differently depending on how they do Y/C separation.

Thus, there really is no such thing as a perfect composite palette for the PCE. 

This has all been known and it is indeed not difficult to realize with a simple side-by-side comparison. Even if you throw away all other confounding factors, a video signal displayed via RGB/component on a CRT will never display colors quite the same as the "same" signal via composite. The goal was never an exact reproduction of composite (which would involve simulating its artifacts as well), but a "best of all worlds" solution where colors and shades are represented with the highest* degree of accuracy possible while also having the crisp clarity of a least lossy analogue video signal.

*highest here can be achieved either via a subjectively selected single reference, or an averaging of samples as you've suggested. I don't think the latter is necessarily better than the former approach, especially since, as you've mentioned as well, there is noticeable variation among PCE units. This means that people have only experienced specific units, but never an "average" unit that is just an abstraction. Personally, I'd rather the basis be a chosen reference, even if I never experienced it myself before, than an average that never existed on actual PCE units.

SamIAm MkII wrote:Now, let me close this out with a personal opinion: The difference between the RGB palette and the composite palette is not that huge. It's great to have the composite palette preserved and available, but in most cases, your eyeballs just aren't going to care. Don't let it trip you up and diminish your enjoyment of the games.
I would also go back to this point, because in the opinion of many, myself included, the difference between the raw RGB colors and this kind of corrected colors is quite significant, not just in obvious stuff like the tone of skin and foliage to which we are naturally sensitive, but in the reproduction of color gradations. The example that has been used a lot is the sky in Startling Odyssey, but there are a ton of other cases like that which I've been noticing for the past year that can be equally startling when you revisit games you had only played with RGB-modded consoles. Of course, we did enjoy those for years, but there is such a thing as not being able to unsee things and now there really is no going back.
User avatar
bobrocks95
Posts: 3460
Joined: Mon Apr 30, 2012 2:27 am
Location: Kentucky

Re: What's my best option for playing PC engine cd games?

Post by bobrocks95 »

Ignorance is bliss, and I'd rather play on original hardware than with an approximated palette from some odd NTSC circuitry.

If the composite palette mimics the composite color (from their one sample) exactly that's weird to me too. On SNES, the first console I got RGB cables for, colors in RGB are much more vibrant and saturated than the composite output. I expect RGB colors to be different, though cutting out parts of gradients because of a lookup table is definitely an extreme, if isolated, example.
PS1 Disc-Based Game ID BIOS patch for MemCard Pro and SD2PSX automatic VMC switching.
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: What's my best option for playing PC engine cd games?

Post by fernan1234 »

bobrocks95 wrote:Ignorance is bliss, and I'd rather play on original hardware than with an approximated palette from some odd NTSC circuitry.

If the composite palette mimics the composite color (from their one sample) exactly that's weird to me too. On SNES, the first console I got RGB cables for, colors in RGB are much more vibrant and saturated than the composite output. I expect RGB colors to be different, though cutting out parts of gradients because of a lookup table is definitely an extreme, if isolated, example.

With the SNES, like with most systems, composite is encoded from RGB so none of this is an issue for it.
SamIAm MkII
Posts: 20
Joined: Sat Feb 13, 2021 3:54 am

Re: What's my best option for playing PC engine cd games?

Post by SamIAm MkII »

fernan1234 wrote:First of all, that's a really neat write up.
Thanks! Just so you know, I didn't mean to single you out or anything for bringing up the palette in this thread. :)
This has all been known and it is indeed not difficult to realize with a simple side-by-side comparison. Even if you throw away all other confounding factors, a video signal displayed via RGB/component on a CRT will never display colors quite the same as the "same" signal via composite. The goal was never an exact reproduction of composite (which would involve simulating its artifacts as well), but a "best of all worlds" solution where colors and shades are represented with the highest* degree of accuracy possible while also having the crisp clarity of a least lossy analogue video signal.
I think we're mostly on the same page.

Artifacts from color transitions are totally unrepresented in the MiSTer palette, and for what it is, I think that's fine. However, the two main issues I mentioned...the inconsistency between systems and the inconsistency in how different NTSC decoders deal with the PCE's funky subcarrier...apply even if you are displaying a single solid color across the entire screen. The conclusion, then, is still the same: you can't really say that there is a single "best" set of colors. For the time being, it's more like you have to settle for a reasonable compromise.
*highest here can be achieved either via a subjectively selected single reference, or an averaging of samples as you've suggested. I don't think the latter is necessarily better than the former approach, especially since, as you've mentioned as well, there is noticeable variation among PCE units. This means that people have only experienced specific units, but never an "average" unit that is just an abstraction. Personally, I'd rather the basis be a chosen reference, even if I never experienced it myself before, than an average that never existed on actual PCE units.
Here I'd have to disagree.

There are millions of PCE units in existence (hopefully); the one that is closest to the average of all of them is probably so close as to be indistinguishable from it. There seems to be a lot of variance, yes, but there is also probably a normal distribution, so the majority of systems are probably fairly close to the average.

The system that was used as the basis for the MiSTer palette was not checked to see if it was relatively average among the systems that were looked at. Neither was the NTSC decoder. For all we know, both the system and the NTSC decoder are outliers.

Surely we could at least agree that if the system closest to the average is indistinguishable from it, then we ought to use that system as the basis for the palette? This is in part for preservation, after all. For the time being, however, we were only able to check about 12 stand-alone systems, and several combinations with expansions like the IFU and the AV Booster, and this just isn't enough data to say where that average really is. The next step would have to be to gather that data.


Here are a few more things to think about:

First, if you eliminate all expansions and only look at "stand-alone" systems that output composite - Coregrafx I and II, Supergrafx, Duos, and the Shuttle IIRC - the range of variance seems to narrow.

Second, unless you allow automatic gain control based on sync-tip-to-blanking voltage, which does not seem to have been all that common in televisions, it seems that most stand-alone PCE systems output a maximum luminance of ~650mV relative to blanking when NTSC standard should have that at 714mV. In other words, a calibrated TV displaying a 100% white screen from a stand-alone PCE is only going to show that screen at 91% of the brightness of the TV's true maximum.

Third, even if you allow for automatic chroma control based on the peak-to-peak of the colorburst, which does seem to have been common in televisions, all PCEs display all of their colors undersaturated in composite. There is not a single color in the entire 512 palette that will display at 100% saturation. On something like a BVM that doesn't have automatic chroma control, the saturation levels are so low, it's like the image is halfway to black-and-white.

However, the current MiSTer palette "stretches" both luminance and saturation. The brightest white color is, in 24-bit RGB, 255,255,255. Also, the most saturated colors are set at 100% saturation, and only the proportions of saturation are preserved (e.g. the magentas are more saturated than the yellows, which is generally true across all PCE systems).

In all honesty, I would say this looks better, but you can't really say that this is a true reproduction of an existing PCE system. Not unless you suppose that the user was turning up the contrast and color dials on his television every time he played PCE.

You've got variance in systems, variance in NTSC decoders, presence or absence of AGC and ACC, and also the possibility of "stretching" the final palette. (And let's not even get into the 7.5 IRE pedestal that North American TVs expect but don't get.) With so many variables, deciding on a non-RGB palette is an art as much as a science. It could be that in the end, the best thing would be to give people a variety of choices.
I would also go back to this point, because in the opinion of many, myself included, the difference between the raw RGB colors and this kind of corrected colors is quite significant, not just in obvious stuff like the tone of skin and foliage to which we are naturally sensitive, but in the reproduction of color gradations. The example that has been used a lot is the sky in Startling Odyssey, but there are a ton of other cases like that which I've been noticing for the past year that can be equally startling when you revisit games you had only played with RGB-modded consoles. Of course, we did enjoy those for years, but there is such a thing as not being able to unsee things and now there really is no going back.
There are certainly some things that do look better in the composite palette. Legend of Xanadu II has perhaps the prettiest pixel art on the system, and I would say the game as a whole looks worse in RGB.

On the other hand, the overall differences between RGB and the composite palette are minor enough that I, for one, would not want to bet that I could spot which palette is being used during any scene of any game.
fernan1234
Posts: 2175
Joined: Mon Aug 14, 2017 8:34 pm

Re: What's my best option for playing PC engine cd games?

Post by fernan1234 »

SamIAm MkII wrote:Thanks! Just so you know, I didn't mean to single you out or anything for bringing up the palette in this thread.
Didn't feel that way, no worries. And I totally agree with the overall sentiment that we're talking about a compromise solution. Everything will be a compromise of some kind.
SamIAm MkII wrote:First, if you eliminate all expansions and only look at "stand-alone" systems that output composite - Coregrafx I and II, Supergrafx, Duos, and the Shuttle IIRC - the range of variance seems to narrow.
Yeah, systems like these would also be the best references most likely. I'd still go with a Duo as my reference of choice, in part because they remain the best way to play the CD library on 100% original hardware.
SamIAm MkII wrote:In all honesty, I would say this looks better, but you can't really say that this is a true reproduction of an existing PCE system. Not unless you suppose that the user was turning up the contrast and color dials on his television every time he played PCE.
I think this is a good way to look at the whole thing. I also imagine that quite a few people did do something like this, since most people played on consumer sets with dials turned up for a more "vivid" picture in general.
SamIAm MkII wrote:On something like a BVM that doesn't have automatic chroma control, the saturation levels are so low, it's like the image is halfway to black-and-white.
And the saturation differences can make for quite a dramatic effect. If you've played around with ACC on/off on monitors that support it, the differences are quite clear. ACC on is supposed to yield more accurate results in general, though subjectively no ACC can be quite pleasing with some sources, and may even be closer to what was expected.
SamIAm MkII wrote:(And let's not even get into the 7.5 IRE pedestal that North American TVs expect but don't get.)
This I don't think needs to be a factor at all. The PC Engine is a very Japanese system, in fact 100% when it comes to the CD library. So all references should be with IRE 0 assumed.
SamIAm MkII wrote:There are certainly some things that do look better in the composite palette. Legend of Xanadu II has perhaps the prettiest pixel art on the system, and I would say the game as a whole looks worse in RGB.
Completely agree with this too. Especially when it comes to pixel art, composite "degradation" can be a really happy thing. Fortunately on MiSTer there is the possibility of applying composite filters on the fly for cases that benefit from it, though unfortunately these filters often go too far in blurring the picture, especially in the case of a system like the PCE which has one of the crispiest composite outputs.
Post Reply