There is something I've wanted to write up for a while now, and I might as well do it here.
A lot of people talk about how using a MiSTer lets you get "the correct color palette" for the PC Engine. Now, if I had a MiSTer myself, I'm pretty sure I would use the composite palette because it is a very reasonable approximation of authentic PC Engine composite video. However, it is not necessarily perfect, and I'd like to explain why.
A quick overview for those who don't know: While most of its contemporaries do a direct conversion of RGB in the analogue domain to create composite video, the PCE does things differently. First, it converts the 9-bit RGB in which the graphics are originally processed to 15-bit digital YUV via a lookup table. This inevitably results in rounding errors for literally every color; however, that's only the first of a few reasons why PCE composite is so different.
Here is a section of the PCE schematic showing the HuC6260 chip and a few relevant components creating the PCE's composite video:
The R-Y and B-Y signals come out of the chip as sine waves which sum together, and Burst and Y are also mixed in to form the complete composite signal. A critical thing to understand is that the colors that result from this do not line up exactly with the 15-bit YUV lookup table. There is some analogue funny-business going on during the encoding phase that we don't completely understand. What we do understand, however, is that if you create a palette based solely on the YUV lookup table, the colors are clearly incorrect. MiSTer's current composite palette is based not so much on the lookup table as on vectorscope readings of an actual PCE system outputting all 512 possible colors in composite video.
Having said all that, let me introduce a couple reasons why this isn't necessarily perfect.
First of all, notice that there are five resistors between Y, R-Y, B-Y, Burst, and the base of the first transistor, which is part of the video amp and is essentially the final mixing point for all signals. NEC seems to have used 1% tolerance resistors in this section, which is nice...however, even a tiny degree of variance results in colors being different from system to system. Without getting too far into the nitty-gritty, suffice to say there are other factors in the video amp itself that create even more system-to-system variance as well.
On the 240p Test Suite discord server, we spent time looking at stock composite video output from over a dozen PCE systems with oscilloscopes and vectorscopes. What we found was that even if you allow for things like automatic gain control and automatic chroma control in the receiver to partially normalize the signal, PCE video can be quite visibly different between systems.
Here is an example. In the image below, think of the middle horizontal color as the base. The upper horizontal color is the base with a 10% saturation reduction, and the lower horizontal color is the base with a 10% saturation reduction and a 4% brightness reduction. Finally, the vertical stripe on the right is the base with a 10% saturation reduction, a 4% brightness reduction, and a 1% hue rotation (i.e. it's displaced about three degrees on a hue circle).
This is a realistic representation of the degree of variance possible from stock PCE composite even with automatic gain control and automatic chroma control in the receiver. Without those features, the variance becomes even greater.
But that's not all. Here is a shot of the signal coming out of the R-Y pin of the HuC6260 when displaying a color that basically maxes it out.
Notice that this is not a perfect sine-wave, but more like a funky triangle wave. I regretfully don't have shots of B-Y and Burst, but you can see pretty much the same thing on both of them, and critically, the Burst wave actually has a slightly unique shape to it.
R-Y, B-Y and Burst together form the 3.58MHz color subcarrier signal that we'll call C, and that gets mixed with Y to create the full composite signal. On the receiver end, your TV or whatever decodes the composite signal by first pulling apart Y and C, and it basically does this with filters. C is separated out by taking what's left after blocking anything that isn't 3.58MHz...or, more specifically, a 3.58MHz sine-wave.
When the triangle-ish waves coming out of the HuC6260 get pushed through the receiver's filtering to isolate C, they get distorted into something more sinusoidal, and this causes a slight phase-delay in them. Since Burst has a different shape, it gets affected a little differently.
How differently? Finally, we come to the key point: That depends on the TV. From the first color TVs in the 50s to the totally digital stuff in the 2000s, there are countless NTSC decoder circuits out there, and even though they have the same goal, the way they might deal with the PCEs non-sinusoidal subcarrier may be different in each case. The phase-delay added to R-Y and B-Y versus the phase-delay added to Burst, which is used as a reference for phase, is quite likely different from TV from TV.
All of this boils down to two things:
1. There is a very noticeable degree of variance in the stock composite signals of PCE systems.
2. Two otherwise perfectly calibrated TVs may decode composite from the same PCE differently depending on how they do Y/C separation.
Thus, there really is no such thing as a perfect composite palette for the PCE.
Now, let me reiterate that the current MiSTer palette is a very reasonable approximation; it represents certain tendencies in PCE composite well. I would use it. However, I can (and have!) done a side-by-side comparison with that palette and a real PCE on the same monitor and found differences. This will pretty much be true for any PCE that isn't the one the MiSTer palette was based on, and even that one could appear differently depending on how different the monitor's Y/C separation is from the device used to decode the signal for analysis for the MiSTer palette.
One thing I will say is that there is a possibility to improve the MiSTer palette. This would take an exhaustive analysis of dozens of PCE systems and at least a few different NTSC decoders, until we get enough data to see something like a normal distribution, but it's not impossible. The resulting tweaks would probably be minor, too, but if we're trying to be perfectionist about it, this is the way to go. I hope it happens someday.
Now, let me close this out with a personal opinion: The difference between the RGB palette and the composite palette is not that huge. It's great to have the composite palette preserved and available, but in most cases, your eyeballs just aren't going to care. Don't let it trip you up and diminish your enjoyment of the games.