The color of a bunch of dots, Part 5

It's about time!  People have been banging on my door, tackling me in parking lots, and calling me at 3:00 AM to demand that I finish my series of blog posts about halftones. Well, maybe the part about being tackled in the parking lot is a bit of a hyperbole. But, there were two comments on the previous blog post in this series.

Picture taken from out of my window

In the first four parts of this series, I described four different mathematical models that allow one to estimate the color of a halftone.

1. Murray-Davies equation

The Murray-Davies equation estimates the spectrum of a halftone from three pieces of information: the spectra of the paper and of the solid, and the percentage of area that is covered by the halftone. The model assumes that the halftone area has either the reflectance of the solid, or the reflectance of the paper, and that the ratio between them is the halftone area on the plate or in the digital file.

2. Murray-Davies equation with dot gain correction

In this model, the straight Murray-Davies equation is augmented with a further assumption - that the halftone dots grew somewhere between the plate (or file) and the substrate. Note that I didn't have to change much in the equation. I just changed Ain (the input dot area in the file) to Aout, which is the effective dot area on the substrate.

3. Yule-Nielsen equation

The Yule-Nielsen equation assumes that there is a leakage of color from the halftone dots to the substrate between the dots. This acts to boost the color beyond what the Murray-Davies formula predicts. There is a parameter, called the n-factor, which is used to adjust the amount of leakage. In the equation below, n is used to characterize the richness of the halftone.

4. Noffke-Seymour

The Noffke-Seymour equation assumes that the volume of ink is equal to the halftone area on the plate or in the digital file. That amount of ink is squished out to some greater or lesser degree. The ink film of the dots is smaller, but the dots cover more area. Beer's law is used to predict the reflectance of the dots. In the equation below, Aout is used to characterize the richness of the halftone.

Assessment of the equations

In part 3 of the series I pointed out that the two Murray-Davies formulas provide a poor prediction of the color of a halftone. (Parents, please cover your children's ears. I am about to spew some apostasy.) It is unfortunate, but Murray-Davies remains the predominant mindset. It has been enshrined even in ISO 12647-2, which provides us with curves for what the TVI should be, despite the fact that the predictions from this are just plain lousy. Yes, I said it. ISO 12647-2 promulgates a dumb idea.

OK, so maybe that's an over-statement? The idea of TVI is good in principle. Using Murray-Davies, you get one number from which to assess how rich the color of a halftone is. Having one number is great for process control. But there are much better alternatives: either equation 3 or 4. Either of these equations have a single parameter to express the richness at which a halftone is printing. The benefit is that they both provide a reasonably accurate estimate of the spectrum of that halftone. Given the spectra or paper and solid, along with one number to describe the richness, you can reconstruct the spectrum of the halftone.

Why does this matter?

Maybe it doesn't matter that we use TVI, which is based on a poor model of reality. Everything is working, right? Well... I would beg to differ.

You really think your TVI formula is working?

First problem: If I compute the TVI of AM and FM tone ramps, TVI would lead me to believe that a simple plate curve can make one look like the other. Or that I could make a magenta tone ramp printed offset look like a magenta tone ramp that was printed gravure or ink jet. Getting the proper TVI is a necessary, but not sufficient, condition to get a color match.

Second problem: If you compute TVI in one region of the spectrum, you won't necessarily get the same TVI from another region. This problem led to the SCHMOO initiative. (SCHMOO stands for Spot Color Halftone Metric Optimization Organization.)

Third problem: How do you compute the spectrum of a halftone? You can't accurately estimate the spectrum (or color) of a halftone using TVI and Murray-Davies.

Which is better?

So, the natural question is, which equation is more better? Yule-Nielsen or Noffke-Seymour?

First, I need to acknowledge my own affiliations. Many of you may be under the impression that my last name is "Math Guy". This is not quite correct. My middle name is "the Math Guy", and my last name is Seymour. Yes... the Seymour of the Noffke-Seymour equation.

Second, I will make a totally impartial statement which can be proved with algebra. At the extremes, the two equations (YN and NS) are identical. At the end of minimum richness, they both simplify to the Murray-Davies equation. At the other end (maximum richness) they both simplify to Beer's law.

Beer is my law

Both equations are based on verifiable assumptions about the underlying physics. Light spreads in the paper, and that causes halftones to be richer. Halftones dots squish out and that causes halftones to be richer. Which one is the predominant effect?

I will make another totally impartial statement. It doesn't really much matter which physical effect is larger. It has been demonstrated through looking at a bunch of data that numerically, the effects are very similar. In between the two extremes, the two equations act very similarly. I suppose some math guy could figger some way to figger just how close the two equations are. But, experience says they are close.

In short, it doesn't much matter which equation (YN or NS) is used. I prefer mine, of course, because. Just "because".

Call to action

What to do about this? To be honest, I don't know. But, the first step to recovery is a trip to the bookstore to buy a shelf full of self-help books. (While you are there, ask about my latest book, How I Recovered from My Addiction to Self-Help Books.)

Part of the problem is that Murray-Davies is such a wonderfully simple equation. You can easily solve it going forward. As it is written above, you can plug in the spectrum of the solid, the substrate and a dot area, and it will give you an estimate of the spectrum of that halftone. But (and here is the cool part) anyone who remembers some of their high school algebra can solve the Murray-Davies equation for the dot area. Plug in the three spectra (solid, substrate, and halftone) and you can solve for the apparent dot area. You can poke this into one cell of a spreadsheet even after a whole evening of experimenting with Beer's law.

This does not hold true for the YN or NS equations. :( These are both "trap door" equations. You can go one way easily, but going the other way requires a bunch more cells, maybe even a whole page. They both require an iterative approach to solving.

If only I knew a math guy who could figger this out!

The color of a bunch of dots, part 2

In the color of a bunch of dots, part 1, I focused on one simple equation for the prediction of the reflectance of a halftone, the Murray-Davies equation. This equation is reasonable and readily understood, but it does not do such a good job at predicting what happens when ink meets paper. The Murray-Davies  equation does such a poor job of predicting reflectance that the prediction error has become one of the most common standard process control parameters for printing.

Dot gain in the headlines

That comment is important enough to repeat. Please read this slowly, carefully articulating every word: The Murray-Davies  equation does such a poor job of predicting reflectance that the prediction error has become one of the most common standard process control parameters for printing. It is called "dot gain" by old pressmen, and "tone value increase" or "TVI" by the intellectual elite. Feel free to decide which group you belong to and use the appropriate phrase.

Of course, we knew one source of error in the Murray-Davies approximation. Halftone dots are bigger in real life than they were in the image file. Ink squishes out between the plate and the blanket and then again when it transfers from blanket to paper, so the dots on the paper are bigger [1]. To be fair to those using the Murray-Davies equation to compute TVI, the degree to which the dots spread is a valid control parameter indicative of how dots squish out.

But, to be fair to people who make brash, negative comments about the Murray-Davies equation (and then have the gall to repeat them in italics), how much the dots squish is only part of the prediction error.

Optical dot gain

The Murray-Davies equation was published in 1936. It was known at least by 1943 that it did not work well for halftone dots on paper. I quote from Yule: "Experimental results do not agree exactly with the theoretical relationships except for screen negatives and positives with sharp dots."

Enter John A.C. Yule and W. J. Neilsen. They presented a paper at the 1951 TAGA conference [2] entitled "The Penetration of Light into Paper and Its Effect on Halftone Reproduction", where they described another reason for the discrepancy. [3]

Yul Brynner, Leslie Nielsen, and Dodd Gayne
(from The King and I in the Cockpit) [4]

The Murray-Davies formula makes the assumption that light either hits a halftone dot or paper. Furthermore - and this is the critical part - that the dots of ink don't effect the color of the paper. Yule and Neilsen point out that this is just not the case. Anyone who says otherwise is itching for a fight.

The diagram below shows the Yule-Neilsen effect. When we look at a halftone, some of the light follows path #1. It passes through the ink once, reflects from inside the paper, and then exits for us to behold its marvelous hue. The ink acts like a filter, so two passes make it a richer color.

Some of the light, however, passes through the ink once and then scatters within the paper. This hapless light then exits from between halftone dots. Since it has only passed through the filter (the ink) once, it will not take on quite as rich a hue.

Dramatization of the  Yule-Neilsen effect

The next diagram illustrates what the result looks like. The paper between halftone dots takes on a richer hue as a result. The magnitude of the effect depends on a few factors. First, obviously upon the amount that the paper scatters light. This is related to the opacity of the paper. A paper that is translucent will tend to scatter light further, enhancing the Yule-Neilsen effect. 

Second, and perhaps not so obvious, is the screen ruling. If the dots are closer together, the light doesn't have to scatter as far to infuse the whole area between halftone dots.

Equally dramatic demonstration of the effect of the Yule-Neilsen effect

These gentlemen, Mr. Yule and Mr. Neilsen, were pretty sharp guys. They knew some math. The graph below shows the equation that they came up with to model this effect. From this equation, it was now possible to predict the reflectance (or density as in the graph) of a halftone from the dot area on the paper and the densities of the paper and solid.

The celebrated Yule-Neilsen equation as it originally appeared

The equation above is written in terms of density and not reflectance. This makes it a bit hard to relate to the Murray-Davies equation. Here is the equation written in a way that makes the correspondence obvious.

Yule-Neilsen equation

Comparing this back to the Murray-Davies equation, we see that the only difference is that in the Yule-Neilsen equation, all the reflectance values are raised to the power of 1/n. The Murray-Davies equation is a special case of the Yule-Neilsen equation with n = 1.

Murray-Davies equation

The appropriate value for "n" depends on the translucency of the paper. One researcher (Pearson) said that it should be between 1.4 and 1.8. Another set of researchers (Qian et al.) had it at 1.3 for their substrate.

Now it gets complicated

This thing that has become known as "tone value increase" thus is comprised of two parts: 

1) The dots on the paper are richer in color than expected because the dots squish out to cover more paper. This is known as physical dot gain.

2) The light spreads between the dots to make the paper take on a tint of the color. In doing so, they also make the color of a halftone richer than expected. This is known as optical dot gain.

That was the easy part. Now for the complicated part. 

Optical dot gain is not as easy to measure as physical dot gain. You need to take a picture of the dots, and assign each pixel to either dot or paper. This isn't all that hard, but it can't be done with a standard spectrophotometer. A second instrument is used, called a planimeter. For "hard" dots - dots that have crisp, well defined edges, the assignment of each pixel isn't that hard. You simply set a threshold gray value somewhere around half way between "paper" and "dot". But for softer dots, the measurement you get depends a lot on how the threshold is chosen.

So, basically, no one in a production environment ever measures physical dot gain. The two types of dot gain get rolled into one. The combination of the two, TVI has become the process control parameter of choice. Any difference between the tone value in the file and the tone value on the paper is undifferentiated.

But researchers generally use the Yule-Neilsen equation to model the relationship between CMYK tone values and reflectance. I have a short list of such papers below [5]. The Holy Grail for these folks is to find a formula that will allow them to compute the whole shooting match. CMYK tone values, along with some press parameters, go into the magic black box. CIELAB values come out.

The Yule-Neilsen equation (and the n value that go with it) are kind of a one-way street. It can be used in prepress to predict what a halftone will look like, but you can't use it in the press room to verify that the print is correct. 

So, for the time being we are stuck with the Murray-Davies equation. Maybe that's not so bad? I will address this issue in the next post in this series. Maybe the current way of measuring halftones is not the panacea that we think it is.

-------------------------------------

[1]  I am taking a very web-offset-centric view of this. The same principles apply to other types of printing.

[2]  TAGA (Technical Association of the Graphic Arts) has been running an annual conference since the ten commandments were inscribed via a lithographic process without the help of Yule Brynner. The 2014 call for papers is out. Submissions dues by July 19th, 2013.

[3]  I missed this conference and hence the landmark paper, but I don't quite recall why.

[4] Seriously for the moment. Yule is the print scientist. Yul is the actor who my wife has a crush on. Someday I will shave my head to compete. Neilsen is another print scientist.The actor's last name is spelled "Nielsen". My wife does not have a crush on him, but regardless, I will someday grow my hair white.

The insidious misspelling of Neilsen's name is so pervasive that Google's patent search returns 11,400 results searching on "Yule Nielsen factor" and only 88 on the correct spelling. This just ticks me off.

Scan from the 1951 TAGA Journal, showing correct spelling

[5] Here is a list (incomplete) of papers where the Yule-Neilsen formula is featured as a way to predict the color of a halftone.

Pearson, M. (1980). N-value for general conditions. In TAGA Proceedings, (pp. 415–425).

Viggiano, J. A. S. (1985). The color of halftone tints. In TAGA Proceedings, (pp. 647–663).

Pope, W. (1989). A practical approach to N-value. In TAGA Proceedings, (pp. 142–151).

Rolleston, R., & Balasubramanian, R. (1993). Accuracy of various types of Neugebauer model. In IS&T and SID’s Color Imaging Conference: Transforms and Transportability of Color, (pp. 32–37).

Arney, J. S., Arney, C. D., & Engeldrum, P. G. (1996). Modeling the yule-nielsen halftone effect. Journal of Imaging Science and Technology, 40(3), 233–238.

Hersch, R. D., & Crt, F. (2005). Improving the Yule-Nielsen modified spectral Neugebauer model by dot surface coverages depending on the ink superposition conditions. In IS&T Electronic Imaging Symposium, Conf. Imaging X: Processing, Hardcopy and Applications, SPIE, vol. 5667, (pp. 434–445).

Gooran, S., Namedanian, M., & Hedman, H. (2009). A new approach to calculate colour values of halftone prints. In IARAGAI.

Rossier, R., & Hersch, R. D. (2010). Ink-dependent n-factors for the Yule-Nielsen modified spectral Neugebauer model. In CGIV – Fifth European Conference on Colour in Graphics, Imaging, and MCS/10 Vision 12th International Symposium on Multispectral Colour Science.

Qian, Yiming, Nawar Mahfooth, and Mathew Kyan, (2013) Improving the Yule-Nielsen modified spectral Neugebauer model using Genetic Algorithms, 45th Annual Conference of the International Circle

The color of a bunch of dots, part 1

First off, most printing today is done with dots. Halftone dots. Oh, you knew that already? Just in case you didn't, have a look at magazine with a magnifying glass. You'll see that highlight areas are made of tiny dots, and shadows are made of dots that are so big they run all together.

Scan from a catalog showing halftoning

A formula to predict the reflectance of a halftone

Way back in 1936, a fellow by the name of A. Murray of Eastman Kodak [1] was pondering the measurement of halftones. Murray sought out his friend, E.R. Davies [2] to ask if there were a simple formula. Davies gave him such a good enough answer that Murray decided to publish the results. Naturally, by Stigler's law,  the resulting equation isn't known as the Davies equation, but rather, as the Murray-Davies equation.

  

Contrary to popular belief, the Murray-Davies formula
was not developed by Bill Murray and Geena Davis

Suppose that you have an area that is halftone dots that cover 20% of the area, something like the area below. Of the light shining on this area, 20% will hit ink, and the remaining 80% will hit paper. If there are 100 photons playing this game, 20 will hit the ink. Suppose further that the ink will reflect 5% of the light that hits it. That means that, of the initial 100 photons in the game, only 1 will reflect back from the ink, since 5% of 20 is 1. 

Now, let's look at the fate of the other 80 photons. Suppose that the paper has a reflectance of 80%. That means that 80% of those 80 photons (64) will reflect. All told, 1 + 64 = 65 of the original 100 photons will reflect.   

The pattern on my jammy bottoms

Did that all make sense?  If so, then congratulations! You understand the Murray-Davies equation. If the various R's in this equation stand for the reflectance of the various things, and A stands for the dot Area, then this simple formula will give an approximation for the reflectance of the halftone:

The celebrated Murray-Davies equation

You may have seen an uglier version of this equation, one involving logarithms or 10 raised to some power.  Trust me. IT's all the same equation. People put that kind of complication in just to make themselves sound smart. Sometimes they are actually smart. Especially when I do it. But the point is, the simple equation above is the basic Murray-Davies equation.

Trouble in Paradise

It didn't take long to see that the formula does not do a fabulous job of estimating the reflectance of a halftone. And it didn't take long to find an explanation. If one compares the size of the dots on the plate with the size of the dots on the paper, it is obvious that the dots get bigger when you print. There is dot gain. And if the dots are bigger, then one would expect the reflectance to go up as well.

Clearly if one would like midtones to be the right color, dot gain must be measured and ultimately controlled. The Murray-Davies equation provides a way to measure the dot gain. First, you solve the Murray-Davies equation for the area term, A. This revision of the Murray-Davies equation (shown below) answers the question "How big must the dots be in order to get a certain reflectance?" 

You measure the reflectance of the paper, the solid, and the halftone, plug them into the equation below, and you have an  indirect measurement of the size of the dots. The difference between this indirectly measured dot area and the area on the printing plate (or the area requested in the digital file) is called the dot gain.

Aside from changing the name from "dot gain" to "tone value increase", this basic formula has survived to this day as a control metric. The image below is a screenshot from the most recent (unpublished, as yet) version of the main ISO standard for printing, ISO 12647-2. The horizontal axis is the tone value (dot area) going from 0% (paper) to 100% (solid ink). The vertical axis is the expected amount of increase in tone value for each of five different types of printing. For printing type A, a 50% tone value in the image file is supposed to print like a 66% halftone.

One of the jobs of the printer is to maintain the press so that these are the tone value increases that are seen on a daily basis. If the press is printing differently (maybe the 50% tone value measures as a 64% rather than a 66%) a look up table (known as a plate curve) is introduced between the digital file and the plate manufacture so that the combined system has the proper tone value increase. As a result, a contact proof can be printed or displayed on a calibrated monitor that accurately predicts what the press will print.

So, it would appear that all is well with the world. The printer has a tool to monitor the way that halftones are printed, and can digitally adjust this to a value that ISO has prescribed. By keeping this constant, the printed product will always be the correct color. 

Or so it would appear. Stay tuned for the next installment, where I show that this simplistic view of tone value increase is lacking, particularly when we go beyond the borders of conventional web offset printing with CMYK inks.

--------------------------------------

[1] Almost no one knows this guy's first name. It is ironic that his last name has gone down in the history of printing without a first name. Note to the historians: When you put my name down in the history books, please make sure my first name is there. And if you have enough space, I wouldn't mind having my middle name, "the Math Guy" included. I said, "almost no one" because I have found exactly one person, Dr. J. A. S. Viggiano, who has preserved the first name: Alexander. Alexander has that first name on at least 22 patents.

[2] Guess what? I don't know this guy's first name.