Interpreting color difference data - a practical discussion of the CRF

My avid readers (yes, both of you) will realize that I have been on a mission, a holy quest, for process control of color. The holy grail that I seek is a technique for looking at the color data of a color-making process, and distinguishing between normal and abnormal variations in the process. This is subtly different from setting acceptance tolerances. The first is inward looking, seeking to improve the process. The second is outward-focused, with the ultimate goal of getting paid by a happy customer.

I'm not talking about getting paid. Unfortunately, getting paid is rarely the outcome of this blog!

In this blog post, I talk about a tool for color geeks. This tool is appropriate for analysis of whether the color is acceptable. I will discuss whether this tool is acceptable for identifying changes in the process. I promise to write about this tool for customer satisfaction later.

Recap of exciting events leading up to this

Just to recap, I wrote a lengthy and boring sequence of blogs on statistical process control of color difference data. The four part series was aptly called Statistical process control of color difference data. (Note that the link is to the first part of the series. For those who tend toward masochism, each of the first three posts have a link to the next in the series, so feel free to indulge.)

The topic for today's blog post is a tool from statistics called the CPDF (cumulative probability density function). At least that's the name that it was given in the stats class that I flunked out of in college. It is also called CPD (cumulative probability distribution), CDF (cumulative distribution function), and in some circles it's affectionately known as Clyde. In the graphic arts standards committee clique, it has gone by the name of CRF (cumulative relative frequency).

I blogged on the CPDF/CPD/CDF/Clyde/CRF before, and by golly, I just realized that I used two of the names in that blog post. This violates rule number 47 in technical writing, which states that you have to be absolutely consistent in your use of technical terms. This is also rule number 7 in patent writing, where it is generally assumed that different words mean different things. I will try to be consistent in calling this tool the CRF. Please forgive me if I slip up and call it Clyde.

Now that the nomenclature is out of the way, today's blog is an extension of the previous blog post on CRF. Today, I want to discuss the practical aspects. Looking at a CRF, what can we discern about the nature of a collection of color differences? And importantly, what conclusions should we avoid coming to, however tempting they may be.

Brief refresher

Below we have a picture of a CRF. The horizontal axis is color difference. The purplish blue curve represents the percentage of color differences that are smaller than that particular DE.

Picture of Clyde from another blog post of mine

It is rather easy from this plot to determine what are called the rank order statistics. The red arrows show that the median (AKA the 50th percentile) is a shade under 1.5 DE. The green arrows show the 95th percentile as being 3.0 DE.

So, one use of the CRF is to quickly visualize whatever percentile is your favorite. You like the 68th percentile? No problem. Your neighbor's dog prefers the 83rd? Yup, it's there as well.

The plot above is real data from one set of real measurements of real printed sheets. The plot below shows nine different sets of real data. The different data sets show medians of anywhere from 0.3 DE to 4 DE. It's clear from looking at the plots that some are tighter distributions than others. Most people would say that the tighter ones are better than the broader ones, but I try not to be so judgmental. I try to love all CRFs equally.

Picture of Bonnie, from the same blog post, who exhibits multiple personality disorder

So, we see that another use of the CRF is to visualize the overall magnitude of a set of color differences. Of course, the median or 95th percentile also give you that impression, but the CRF plot is visual (great for people who are visual learners), and it incorporates all the percentiles into one picture.

Do we need more than one number?

This begs a question. If I know the median, do I need the 95th percentile as well? Is there any additional information in having both numbers?

I assessed this in that prior blog post that I keep referring back to. Based on the data sets that I had at my disposal, I found that there is a very strong correlation between the median and the 95th percentile (r-squared = 0.903). You could get a pretty good guess at the 95th percentile just by multiplying the median by 1.840. That's actually good news for those who have only a small amount of data to draw conclusions from. The median is a lot more stable than trying to infer the median from (say) 20 points.

But I should add a caveat -- the data that I was analyzing was pretty much all well-behaved. If the data is not well behaved, then the ratio between the two will probably not be close to 1.840. So, having both numbers might provide exactly the information that you are looking for. I mean, why would you be wasting your time looking at color difference data, if not to identify funkiness???!?

The next graph illustrates this point. I started with that same set of 9 CRFs from the previous graph. I scaled each of them horizontally so that they had a median color difference of 2.4 DE. If all the different percentiles have the same basic information, then all the curves would lay right atop one another. 

But they don't all lay atop each other. After doing the rubber band thing to adjust for scaling, they don't all have the same shape. The orange plot is below the others in the lower percentiles, and mostly above the others at higher percentiles. The red plot is kinda the opposite.

What does this observation about the two CRFs tell us about the DE values from which the CRFs were created? And more importantly, what does this tell us about the set of color differences that went into creating the CRF?

(That was a rhetorical question. Please don't answer it. If you answered it, it would steal the thunder from the whole rest of this blog post. And that would make me sad, considering the amount of time that I am going to put into analyzing data and the writing this post! You may actually learn something, cuz I think that no one in the history of the known universe has investigated this topic to the depth that I did to write this post.)

The simple answer to the rhetorical question is that the orange plot is closer to having color difference values that are all the same, and the red plot has more of a range of color difference values. But the answer is actually more nuanced than that. (I recently heard the word nuanced, and I admit that I have been waiting for the opportunity to show it off. It's such a pretentious word!)

Here is the third thing we learn by looking at the CRF: Not all CRFs are created equal. The shape of the CRF tells us something about the color difference data, but we aren't quite sure what it tells us. Yet.

Looking at cases

To help build our intuition, I will look at a few basic cases. I will show the distribution of points in an a*b* plot, and then look at the associated CRF. Conclusions will be drawn and we will find ourselves with a deeper appreciation of the merits and limitations of CRFology as applied to color difference data.

About the data

The data that I use for the different cases is all random data generated deep in the CPU of my computer. I used a random number generator set up to give me DL*, Da* and Db* data that is normally distributed (Gaussian). Note that while DE values are definitely not normally distributed, the variations in the individual components of L*a*b* are more likely to follow a normal distribution, at least when the color process is under "good control". Whatever that means.

A peek inside my computer as it generated data for CRFs

In most of the cases following, I have looked at data that is strictly two-dimensional (only a* and b* values, with the ridiculous assumption that L* has no variation). I will bring three-dimensional variation in on the last two cases. Those two will blow your mind.

All of this work is done with the 1976 DE formula, simply because it is sooooo much easier to work with. The conclusions that we draw from analysis of these cases will not change if we use the DE2000 color difference formula. This is not immediately evident (at least not right now), but trust me that I kinda have a feeling that what I said was true.

I should mention one other thing. The figures below all show scatter plots and CRF plots. I decided to use 200 data points for the scatter plots, since that made a plot where the scatter was pretty clear. If I went larger, the dots all merge into a blob, which is bad. But worse than that is the fact that the size of the blob depends on the number of points in the scatter plot. (I have a solution to that, but it it several blog posts away).

For the CRF plots, 200 points would give a jaggy appearance that would mask the true underlying shape. So, for the CRFs I chose the luxury of enlisting a few more CPU cycles to generate 10,000 data points. My computer didn't seem to mind. At least it didn't say anything.

In all the scatter plots below, I have plotted Da* and Db*, and not a* and b*. The values are relative to some hypothetical target a*b* value.

Case 1

This first case is the most basic. We assume that the distribution of measurements in a*b* is a circular cloud. The values for Da* and Db* are both normally distributed with standard deviation of 1.0 DE, and Da* and Db* are uncorrelated. You can probably see this; I show the scatter plot in the upper left hand corner of the figure, and the associated CRF below.

Case 1, uncorrelated variation of equal size in a* and b*

The lower right corner of the figure introduces a new metric for analysis of the CRF; the median-to-95th percentile ratio. I intend this to be a parameter which describes the shape of the CRF. Dividing by the median makes it independent of the overall spread of the CRF. I have decided to give this ratio a sexy name: MedTo95. Kinda rolls off the fingertips as I type it. 

Note that a lower value of MedTo95 corresponds to a CRF that has an abrupt transition (high slope at the median), and a lower number indicates a more laid back shape.

(The astute reader will note that the number 1.840 that I described in a previous blog as a way to estimate the 95th percentile from the median was a MedTo95 metric based on real DE data. The really astute reader will note that 1.840 is not equal to 2.11. The really, really astute reader will draw an inference from this inequality. The CRF shown in Case 1 is a bit abnormal.)

Case 2

The most natural question to ask is what happens if the cloud of DE data is moved over a little bit. To put this into context, this is a DE cloud where there is a bias. We have the usual variation in color, but we are also not quite dead-on the target value.

The Figure below shows the Case 1 data in black and this second DE data in green. I thought it was pretty clever of me to come up with a figure like this that is so incredibly intuitive. If you had to scratch your head for a while before catching on, then I guess I am not as clever as I would like to think.

Case 2, which is the same as Case 1, except shifted over

It should come as no surprise that the CRF has been expanded outward toward larger DE values. We have added a second source of color difference, so we expect larger DE values.

Important note for the analysis of the CRFs: The bias in the color (of production versus target a*b*) caused a reduction in MedTo95.

I should also point out that it makes no difference whether the cloud in a*b* has shifted to the left by 2 DE, to the right by 2 DE, or upward or downward. The DE values will all be the same, so the CRF will be the same.

Case 3

It is hard to compare the shape of the black and green plots of Case 2, since one has been stretched. I have altered the cloud of points in the figure below so that the CRFs have the same median. Note that this scaling of the CRF required a commensurate scaling of the a*b* cloud. So, the green cloud is more compact than the black cloud of points. The standard deviations, shown at the bottom left corner of each of the scatter plots, were cut in half.

Case 3 - two data sets with the same median, but different offset

The MedTo95 ratio was 1.70 in Case 1, and is 1.69 in this case -- almost identical. That's reassuring. I mean, that's why I introduced this shape parameter as a ratio.  

Tentative conclusion #1: MedTo95 kinda works as a parameter identifying the shape.

We see that introducing a bias in our process (the color is varying about a color somewhat different than the target color) changed the shape of the CRF. The CRF of the biased data makes a faster transition, that is, it has a higher slope at the median, that is, MedTo95 is smaller.

Tentative conclusion #2: A lower MedTo95 is an indication of bias - that you're not hitting the target color.

(Please note that I underlined and boldfaced the word tentative. This might be foreshadowing or something.)

Case 4

The next most obvious change we could make to the distribution is to change the relative amount of variation in a* and b*, in other words, to make the scatter plot elliptical. This is to be expected in real data. Imagine that the plot below is of the variation in a* and b* measurements of a solid yellow throughout a press run. The predominant cause of variation is the variation in yellow ink film thickness, which is reflected mostly in the b* direction. 

I will take this a step further. The eccentricity (as opposed to circularity) of the scatter plot is an indication that one cause of variation is predominant over the others. 

The figure below shows the effect that a 5:1 aspect ratio has on the CRF.

Case 4 - elliptical variation

This is cool. The shape of the CRF has changed in the opposite direction. I would call this a mellow, laid-back, sixties kind of cool and groovy CRF, one which makes the transition gradually. The slope at the median is smaller. Graciously, MedTo95 has responded to this by getting much larger. Once again, MedTo95 has given us an indication of the shape of the CRF.

Perhaps we can tie the change in MedTo95 back to the variation in a*b*? 

Tentative conclusion #3: A higher value of MedTo95 is an indication of a departure from circularity of the scatter plot. 

Now we're getting somewhere! We can look at MedTo95 and understand something important about the shape of the scatter plot. If we see MedTo95 go up, it is a sign that we have one source of variation which is rearing its ugly head.

But once again, the word tentative is bold faced and underlined. It's almost like I am warning the reader of some broader conclusion that may eclipse this one.

Case 5

Case 4 looked at eccentricity in the b* axis. This is typical of variation in yellow, but magenta (for example) tends to vary a lot more in a*. What if the angle of the eccentricity changes? 

I played with my random number generator to simulate a variation which has been rotated so that the major axis is in the a* direction. I show the figures below.

Case 5 - a comparison of eccentricity in two different directions

This is reassuring. The CRFs are pretty much identical, and a so are the MedTo95 values. This shouldn't be much of a surprise. A moment's consideration should convince one that the color difference values (in DE76) would be the same, so the CRF shouldn't change. This will not be the case for DE2000 values.

This is good news and perhaps not-so-good news. The good news is that the CRF and MedTo95 of DE76 values is irrespective of the predominant direction of the variation. The bad news is that the CRF and MedTo95 of DE76 values are irrespective of the predominant direction of the variation.

Tentative conclusion #4: CRF and MedTo95 don't know nothing from the direction of variation. 

Case 6

We have looked at a bias and eccentricity of variation in isolation. How about if both occur? In the figure below we look at one possible example of that. The blue cloud has been shifted to the right, and squished horizontally. It was also squished just a bit vertically, just so the median is that same as all the other CRFs. 

Case 6, in which we encounter both sliding over and squishing

From the figure, it is clear that the combination of these two effects causes a CRF that is more uptight than the standard scatter plot that we started with. The new CRF is reluctant to change initially, but changes rapidly once it decides to change.

Thus, we re-iterate tentative conclusion #2: A lower MedTo95 is an indication of bias - that you're not hitting the target color. And, of course, we can forget about tentative conclusion #3: A higher value of MedTo95 is an indication of a departure from circularity of the scatter plot.

How does elliptical distribution with offset (Case 6) compare Case 3, where the scatter plot shows a circular distribution with offset? The two are compared in the figure below.

A comparison of two biased distributions

Here we see two CRFs that are pretty darn close if you look at the area above the median. The MedTo95 of the two are also (to no one's surprise) very close. If I may remind you, the CRFs represent a collection of a whopping 10,000 data points where the initial distributions were algorithmically designed to be normal distributions. You ain't never gonna see no real CRF that is as pristine as these CRFs.

Our tentative conclusions are starting to unravel. :(  

Tentative conclusion #5: There ain't no way, no how, that you can use MedTo95 to diagnose ellipticity when there is a bias.

But, these  conclusions are based on what's going on above the median. There is still some hope that the stuff below the median might pan out. We would need, of course, an additional parameter. Maybe MedTo25? 

Case 7

In Case 6, we looked at elliptical variation that is perpendicular to the bias. The bias was off to the right, and the principle axis of the variation was up and down. Let's look at bias and variation that are both along the a* axis. This is shown in the next figure.

Case 7 - comparison of variation parallel and perpendicular to bias 

The new curve - the one in violet - kinda looks like red curve shown in Case 4. Things have certainly gotten complicated! I will try to capture this in another tentative conclusion.

Tentative conclusion #6: In the presence of both elliptical variation and bias, elliptical variation in the direction of the bias looks similar to elliptical variation. Elliptical variation perpendicular to the direction of the bias looks like bias. 

Ummm... I didn't stop to consider what happens when the elliptical variation is at 45 degrees to the bias. Presumably, it looks a lot like circular variation with no bias. That ain't so good. 

I probably should actually show an example of this. I think the CRF of elliptical variation at 45 degrees to direction of the bias would look a lot like the black CRF that we have been using as a reference, at least above the waist. But, rather than head further down the rabbit hole, I have one more consideration.

Case 8

All of the examples so far have made the assumption that the variation is strictly two-dimensional, that is, in a* and b*. That's a simplification that I made in order to aid in our understanding of the interpreting of a CRF. One would expect that three-dimensional variation is more likely to be encountered in the real world.

In the cyan of the figure below, I modeled spherical variation which is of equal magnitude in L*, a*, and b*.

Case 8 - the effect of dimensionality on the CRF

By comparing the cyan CRF to the black (two-dimensional), we see that adding a third dimension has the effect of making the transition sharper and of decreasing MedTo95. The red CRF has been added to suggest the effect of reducing the dimensionality to something closer to one.

(Some readers may be thinking something along the line of "chi-squared distribution with n degrees of freedom, where n is some number between 1 and 3, where that number might not be an integer." If those words are gobbledy-gook, then that's ok.)

This next figure compares the CRF of a three dimensional spherical distribution with a two dimensional circular distribution with a little bias. 

Comparison of spherical distribution with circular distribution with bias

I think that this might be just a tad scary for those who wish to use the CRF to divine something about the scatter of points in color space. We see two distributions that are nothing like each other, but yet have CRFs that are very similar. 

In a theoretical world, one might be able to tell the difference between these two. But, there are two things fighting against us. First, we never have CRF plots that are as clean as the ones I have created. 

Second, this blog post shows that we have a lot of knobs to play with. The shape of the CRF is effected by the length of all three of the axes of the ellipsoid, as well as by the magnitude and direction of the bias with respect to the axes of the ellipsoid. Without a lot of trying, I have twice come up with pairs of dissimilar distributions where the CRFs are similar. And I haven't even considered variations that are non-normal. Given a bit more time, I think I could get some pairs of CRFs that would boggle the mind. 

The non-tentative conclusion

If two CRFs are different, we can pretty definitively make the statement that there is some difference between the distributions of color differences. But, one might just as well look at tea leaves to divine the nature of the difference in the distributions. Furthermore, if the CRFs of two sets of color difference data are similar, one cannot come to any conclusions about the similarity between the underlying color variation in CIELAB space.

This blog post and the exciting conclusion was not an Aha! experience for me. The Aha! moment occurred earlier, when I was coming to grips with the fact that a bias will mask variation in CIELAB. This was described in one of my blog posts about process control of color difference data. 

So, what's the answer? How can we look at a cloud of points in CIELAB and draw any inferences about it? The following diagram is a clue -- drawing ellipsoids that "fit" the variation of points in CIELAB. The ellipses are an extension of the standard deviation to three-dimensions. This involves the co-variance matrix of DL*, Da*, and Db* values. It involves a strange concept of taking the square root of a matrix - not to be confused with the square root of the components of a matrix. And it involves principle component analysis. And it is related to Hotelling's T-squared statistic.

The ellipses below were generated using this technique. I will get around to blogging about this eventually!

I gratefully acknowledge the comments are proofreading from a good friend, Bruce Bayne. He won't admit it, but he is a pretty sharp guy.