The color of a bunch of dots, Part 5

It's about time!  People have been banging on my door, tackling me in parking lots, and calling me at 3:00 AM to demand that I finish my series of blog posts about halftones. Well, maybe the part about being tackled in the parking lot is a bit of a hyperbole. But, there were two comments on the previous blog post in this series.

Picture taken from out of my window

In the first four parts of this series, I described four different mathematical models that allow one to estimate the color of a halftone.

1. Murray-Davies equation

The Murray-Davies equation estimates the spectrum of a halftone from three pieces of information: the spectra of the paper and of the solid, and the percentage of area that is covered by the halftone. The model assumes that the halftone area has either the reflectance of the solid, or the reflectance of the paper, and that the ratio between them is the halftone area on the plate or in the digital file.

2. Murray-Davies equation with dot gain correction

In this model, the straight Murray-Davies equation is augmented with a further assumption - that the halftone dots grew somewhere between the plate (or file) and the substrate. Note that I didn't have to change much in the equation. I just changed Ain (the input dot area in the file) to Aout, which is the effective dot area on the substrate.

3. Yule-Nielsen equation

The Yule-Nielsen equation assumes that there is a leakage of color from the halftone dots to the substrate between the dots. This acts to boost the color beyond what the Murray-Davies formula predicts. There is a parameter, called the n-factor, which is used to adjust the amount of leakage. In the equation below, n is used to characterize the richness of the halftone.

4. Noffke-Seymour

The Noffke-Seymour equation assumes that the volume of ink is equal to the halftone area on the plate or in the digital file. That amount of ink is squished out to some greater or lesser degree. The ink film of the dots is smaller, but the dots cover more area. Beer's law is used to predict the reflectance of the dots. In the equation below, Aout is used to characterize the richness of the halftone.

Assessment of the equations

In part 3 of the series I pointed out that the two Murray-Davies formulas provide a poor prediction of the color of a halftone. (Parents, please cover your children's ears. I am about to spew some apostasy.) It is unfortunate, but Murray-Davies remains the predominant mindset. It has been enshrined even in ISO 12647-2, which provides us with curves for what the TVI should be, despite the fact that the predictions from this are just plain lousy. Yes, I said it. ISO 12647-2 promulgates a dumb idea.

OK, so maybe that's an over-statement? The idea of TVI is good in principle. Using Murray-Davies, you get one number from which to assess how rich the color of a halftone is. Having one number is great for process control. But there are much better alternatives: either equation 3 or 4. Either of these equations have a single parameter to express the richness at which a halftone is printing. The benefit is that they both provide a reasonably accurate estimate of the spectrum of that halftone. Given the spectra or paper and solid, along with one number to describe the richness, you can reconstruct the spectrum of the halftone.

Why does this matter?

Maybe it doesn't matter that we use TVI, which is based on a poor model of reality. Everything is working, right? Well... I would beg to differ.

You really think your TVI formula is working?

First problem: If I compute the TVI of AM and FM tone ramps, TVI would lead me to believe that a simple plate curve can make one look like the other. Or that I could make a magenta tone ramp printed offset look like a magenta tone ramp that was printed gravure or ink jet. Getting the proper TVI is a necessary, but not sufficient, condition to get a color match.

Second problem: If you compute TVI in one region of the spectrum, you won't necessarily get the same TVI from another region. This problem led to the SCHMOO initiative. (SCHMOO stands for Spot Color Halftone Metric Optimization Organization.)

Third problem: How do you compute the spectrum of a halftone? You can't accurately estimate the spectrum (or color) of a halftone using TVI and Murray-Davies.

Which is better?

So, the natural question is, which equation is more better? Yule-Nielsen or Noffke-Seymour?

First, I need to acknowledge my own affiliations. Many of you may be under the impression that my last name is "Math Guy". This is not quite correct. My middle name is "the Math Guy", and my last name is Seymour. Yes... the Seymour of the Noffke-Seymour equation.

Second, I will make a totally impartial statement which can be proved with algebra. At the extremes, the two equations (YN and NS) are identical. At the end of minimum richness, they both simplify to the Murray-Davies equation. At the other end (maximum richness) they both simplify to Beer's law.

Beer is my law

Both equations are based on verifiable assumptions about the underlying physics. Light spreads in the paper, and that causes halftones to be richer. Halftones dots squish out and that causes halftones to be richer. Which one is the predominant effect?

I will make another totally impartial statement. It doesn't really much matter which physical effect is larger. It has been demonstrated through looking at a bunch of data that numerically, the effects are very similar. In between the two extremes, the two equations act very similarly. I suppose some math guy could figger some way to figger just how close the two equations are. But, experience says they are close.

In short, it doesn't much matter which equation (YN or NS) is used. I prefer mine, of course, because. Just "because".

Call to action

What to do about this? To be honest, I don't know. But, the first step to recovery is a trip to the bookstore to buy a shelf full of self-help books. (While you are there, ask about my latest book, How I Recovered from My Addiction to Self-Help Books.)

Part of the problem is that Murray-Davies is such a wonderfully simple equation. You can easily solve it going forward. As it is written above, you can plug in the spectrum of the solid, the substrate and a dot area, and it will give you an estimate of the spectrum of that halftone. But (and here is the cool part) anyone who remembers some of their high school algebra can solve the Murray-Davies equation for the dot area. Plug in the three spectra (solid, substrate, and halftone) and you can solve for the apparent dot area. You can poke this into one cell of a spreadsheet even after a whole evening of experimenting with Beer's law.

This does not hold true for the YN or NS equations. :( These are both "trap door" equations. You can go one way easily, but going the other way requires a bunch more cells, maybe even a whole page. They both require an iterative approach to solving.

If only I knew a math guy who could figger this out!

Scribbling away that scratch on my car

I received an interesting comment/question on a LinkedIn thread about the Scribble pen. Answering this question provides me the opportunity to touch on a number of topics and color science, as well as the opportunity to show off just how incredibly smart I am. Since both of these are high on my list of things to do, I decided to dedicate a blog post to answering his question.

Malcolm

Malcolm Cutting

Paint & Development...looking for challenge to develop your business. Call me.

Just using imagination here and the touch up pen for the automotive smart repair scratches....now that could be quite possible.....or could it? Worth thinking about.

The short answer

No.

Requirements are tough

Malcolm is one of my oldest and dearest friends. I just met him on LinkedIn about 15 minutes ago. Feel free to draw your own conclusions about my track record with friends. From reading his LinkedIn profile, I know that he has been in the automotive coating business for multiple decades. He knows that matching paint on a car is one of the most demanding color applications, if not the most demanding.

The first issue is that on a scratch, you are trying to match two colors that are side by side. The human visual system (which includes those parts of the brain that are not devoted to memorizing every episode of Laverne and Shirley) is also very good at picking up on coherent anomalies, such as a line.

Juxtaposition accentuates color differences

The second issue is one that most of my readers are aware of. If you are successful enough in life to have access to the internet, then it is almost certain that your car is more expensive than mine. Naturally, we want our cars to look expensive to show that we are indeed more successful than some dufus who turns to blogging to feed his massive ego. Most of us express our identity through the cars we drive.

Would Guy Fieri still be cool in a Saturn?  I think not.

In short, people are picky about touching up scratches on their cars.

It has to be the same color

Naturally, when a scratch is retouched, it has to be the same color. In order to make it the same color, we need to measure the color accurately. Back in the days of Moses and Pythagoras it was done with the eyeball of someone who is both discerning and not colorblind. Ideally, this person would have been more discerning than the most demanding of customers. 

Nowadays, critical color matching is sometimes/usually/always done by expensive spectrophotometers, which are sometimes/usually/always more discerning than the most demanding of customers. I suspect if you go round and chat up the guy who does that painting at the local body shop, you'll find that he needed to go back to college to pick up a PhD in astrophysics (or some other branch of astrology) just to be able to run a spectro.

The kid who paints my bumpers 

The Scribble pen? It uses an RGBC sensor. Four channels. That's it. The standard ISO 13655, which provides the definition for a color measuring device in the graphic arts, requires that a spectrophotometer be used, and that the spectrophotometer must have at least 15 channels, and preferably 31. I ain't know arithmetic guy, buy I think 3 or 4 is less than 15.

In my own personal experience, I spent about two years working with a bunch of smart guys trying to teach an RGB camera to accurately measure color. Here is one of many papers I wrote about this sad experience:

Why do color transforms work? And here is another: Color measurement with an RGB camera. I have seen a lot of claims about this, and it burns my butt like a candle on a toilet seat. If you want discerning color measurement, you can't use RGB to measure the color.

The color must match under any lighting

In order to be happy about the matching of the color on a scratch, most car owners (I think) would want the match to be good wherever the car is. The illumination could be direct lighting from the Sun, diffuse bluer lighting from the clear sky, the incandescent lighting from one of the bulbs in my garage, the fluorescent lighting from the other bulb, and the sodium vapor lighting you see in parking structures. 

This may seem like no big deal, but those who have practiced saying the word metamerism will realize that this is not a given. It is possible for two objects to match under one lighting, but not under another. If you don't believe me, try to guess the color of a vehicle in a parking structure at night. If you want to match a color under all lighting, you will need to do spectral matching and you will need a lot of different pigments.

The RHEM indicator

The Scribble pen, as cool as it is, only has four pigments: cyan, magenta, yellow, black, and white. Or maybe that's five? I'm a math guy, not an accountant. Spectral matching is pretty much out of the question.

Hmmm... Metamerism might be a good topic for a future blog. Look for it at your favorite blog sites.

The color must match at all angles

If you were impressed by my use of the word "metamerism", I introduce another important-sounding word: goniophotometry. If you can slip those two words into casual conversation, you can pretty much bluff your way into any of those wild color science parties that you are always hearing about. Goniophotometry is the measurement of light when you change the lighting and viewing to different angles. When you reposition the lighting, or move your head around, there is a subtle or sometimes huge change in the color of an object.

The simplest example of this effect is something that is so ubiquitous that it probably goes unnoticed. In fact, I am going to guess that I might get some arguments about whether this is even a color change. I'm talking about gloss. When you view a glossy object at the gloss angle, the color of the object changes. 

This is a point where there might be some argument. One might argue that the color of the object didn't change just because of the angle of lighting and viewing. I would argue that the composition and intensity of the reflected light changes drastically at the gloss angle. 

For the purpose of our discussion, gloss is important. Would you say that a touch-up paint matches another if there is a difference in gloss? I think not! Once again, as cool as the Scribble pen is, it does not have a mechanism to adjust the gloss of the ink.

Metallic effects are another example of goniochromism (when color depends on angles of lighting and viewing). Cars often have a metallic paint. This is achieved by embedding flakes of metals within the paint, laying the paint down in thin layers to make sure that the flakes land flat, and polishing the heck out of it between layers. (This is my understanding... I am open to hearing an explanation from someone who has actually painted a car before.)

Painting a car is an exacting science

The Scribble pen, as cool as it is, does not include any metallic flakes. Nor does it include a polishing thingie. So, I am gonna guess that metallic effects might be a bit tough with this pen.

The creation of gloss and metallic effects are two areas where this pen, as cool as it is, is gonna fall short of the requirements for fixing my paint job when some mathaphobe keys my car. But I skipped over another missing feature. The Fix-The-Scratch-In-My-Car pen must also be able to measure the goniophotometric properties of the existing paint. Some cars are metallic and glossy. Some are just glossy. Other cars, particularly those left outside in Arizona, have lost some of that gloss. Some cars, such as mine, which reside in areas where they use salt on the streets to melt ice, have a glorious texture where a dull rust color is interspersed with the drab shade of pale lavender greenish orange. 

As I understand it, the big car companies use a goniospectrophotometer for quality control of their paint process. Such an instrument runs something over $100K and measures the spectra of reflected light at a zillion and a half different angles in order to make sure the color is correct from all angles. These are run by more astrophysics PhDs.

Goniospectrophotometer

Now, I could have that wrong. Maybe the big car companies use an abridged goniospectrophotometer, such as X-Rite's MA98, for quality control. Rather than a zillion and a half angles, these instruments make measurements at maybe 20 or 30 different angles. These instruments probably make more sense on the production line, since the measurement time is on the seconds, rather than minutes or hours.

The Scribble pen, as cool as it is, probably does not include even an abridged goniospectrophotometer. If it did, the price would be pretty cool, since the abridged gonios cost around $25K. 

Still more

There are a few more requirements for a Fix-The-Scratch-In-My-Car pen. Someone should probably mention that there is a difference between ink and paint. Ink is transparent. Paint is opaque. You want to paint your car with paint, since the bare metal is not all that nice looking.

Someone should also mention that the paint has to be durable. Durable enough to stay put under the effect or driving rain, beating sun, slush, and sandstorms. My wife also tells me that there are these places called car washes where these sudsy rollers come out to rub the grime off your car. The only place I am aware of like that has rollers who are wearing bikinis.

And then someone might wanna mention that it would be good if this pen could avoid scratching the car? I don't have all the details on the Scribble pen, but one possible implementation of the inking mechanism has an ink jet head spraying the back side of a ball point.

Conclusion

Malcolm, I think you got a cool idea there. I'm thinking the Scribble pen might not quite be up to the task, though. But, if we loosen up some of the constraints that came from making this device a handheld pen...

Scribble Pen, a real Color Picker Pen?

I wrote a blog post a while ago called "Color Picker Pen". This pen had a color sensor and could change the color of the ink to match. There weren't a lot of technical details available on the pen, but what I read was enough to convince me that this pen is fiction. I recently found out about another pen that's under development which just might be real. I could find nothing suspicious in their press release and emails, and it seems like the miniaturization technology might just be available today.

Meet the Scribble. Ok,.. cute name, but is this one real?  The website says that they will be having a Kickstarter campaign. I sent an email to their "info" email address, and I got a human-generated reply. So far, so good. 

The Scribble pen

The website and email provided some technical details. The pen uses an ARM 9 processor, has a rechargeable 325 maH lithium battery, 1 Gb of memory, and communicates through Bluetooth or micro USB. At the very least, this shows that they have actually through through some of the details.

Unlike the Color Picker Pen, they also demonstrate some knowledge of basic color science. The Scribblers had the foresight to include a white LED as part of the sensor. They use cyan, magenta, yellow, and black inks instead of the RGB inks in the Color Picker. And another detail shows that they have (at the very least) thought through some details here is that they also mention the use of white ink.

Why white ink?

Why do they add white ink, you may ask? My desktop printer doesn't use white. The printing press that printed my copy of the Edmund Optics Swimsuit Issue used CMYK inks, but no white. Why would this fabulous pen need white?

There are two reasons why white is used when mixing stuff to get colors. White flood-coat is often used on clear packaging to back up the standard inks. This provides opacity that is not found in the substrate or the inks. Much like writing with a red pen on black paper, the ink wouldn't show up without a base of white. But, I suspect that this isn't the reason that the engineers of the Scribble Pen added the complexity of white ink.

If I want to print a pink highlight with an ink jet printer or on a web offset press, the approach is pretty simple. Don't put much ink down. Just a dribble of magenta ink, or just a sprinkling of tiny halftone dots. But if you are searching for that same color to paint your wall, you need a different solution. You need the same volume of paint, just less color. When mixing up a can of paint, the paint mixologist first reaches for a can of white base. 

You see something similar when you can look at the Pantone fan deck, which includes the recipe for each of the thousand-plus colors of ink. Pantone 217, for example, calls for 1 part of rubine red ink and 31 parts of something called "transparent white". Transparent white is unpigmented ink, and would look transparent (not actually white) if printed by itself.

The Scribblers may actually be using ink that contains white pigment (titanium dioxide or zinc oxide, maybe). But I think it's more likely that it is more likely that they use a transparent white ink. It would be neat to write on black paper with a pen, but I would guess that it is nearly impossible to get any sort of decent opacity with the amount of ink dispensed with a pen. Think about how much "ink" that a WhiteOut dispenser spits out.

Color sensor

I was told a tiny bit about the sensor in my email from Scribble. It is an RGBC sensor. This means they collect light through a red, green, and blue filter, like practically every digital camera in the world. But they add a "C" filter, which mean "clear". That is, the fourth channel measures all the light together: red, green, and blue.

Why RGBC, you might ask? Digital cameras with this fourth channel are a relatively new development. Here is an article about the RGBC imager in Google's Moto X cell phone. The big selling point for the extra channel is that you get better sensitivity when the light is low, since the C channel sees about three times more light. 

This is a trick that the human eye plays. There are actually four light sensors in the eye. We have the equivalent of red, green, and blue sensors (the rods) that we use when the lights are bright. When the lights get low and the mood romantic, we make use of our rods, which are more sensitive to light. It's a cool design, but it does mean that we can't see color very well at night. 

Just like black and white TV, the rods can't distinguish color. The Moody Blues lamented this at the end of Nights in White Satin:

Cold hearted orb that rules the night,

Removes the colors from our sight,

Red is gray and yellow, white,

And who decides which is right,

And which is ... an illusion.

A legendary album about color perception

Which makes me wonder why they chose that sensor rather than a straight RGB sensor. I mean, they want to measure color, right? I have a few thoughts, but obviously I can't speak for the designers.

The whole "better performance at low lighting levels" thing might have been the motivator, but I'm not sure that's such a big deal. If I am taking pictures of my buddies getting drunk at the bar, I want high sensitivity to low levels of light because the lighting in the bar is low. But the Scribble pen has a white LED, so they should theoretically have enough light. If the sensor picks up a low light level, then it's likely to be a dark object. 

Then again, maybe dark objects are where the four channel can really help?

Another possibility... RGB sensors aren't good at measuring color. Yes, really. They're not. I have a whole bunch of technical papers on the topic, including this one Why Do Color Transforms Work?  Having a fourth channel might help a tiny bit, but probably not much.

Here's another thought. Years ago before computers were ubiquitous, color separation was done as an integral part of scanning. The image through a red filter would determine how much cyan was needed, the image through a green filter would determine magenta, and the image through a blue filter would determine the yellow. Black ink was determined by using no filter.

I'm thinking that might not be the reason that the Scribblengineers decided on an RBGC sensor, though. My explanation is pretty arcane, and since they have a computer in this device, they could come up with way much more better algorithms.

So, I don't know why they chose this sensor. I'm not saying it's a bad idea. It might be slightly advantageous. Or maybe it's cost effective. I dunno. 

How about the ink mixing?

One thing that puzzled me was how the ink was mixed. The email I got from Scribble was intentionally mute on how they do this part, due to patent stuff. But, it seems like a lot of mechanics to be stuffed into such a little package. I am not an expert here, but I wondered whether it is even possible.

So I did a little patent searching. I found an interesting patent, that is, if patents can be called "interesting".  I have no information about whether the good folks at Scribbler do anything like this particular patent. I have no idea of whether they know of this patent. It's from a little company called HP and is titled a "Writing instrument with user-controlled ink color". Ink jet heads from at least three inks (CMY) feed into a tiny mixing chamber, where it is transferred to the roller ball. 

Pen from US patent #6,749,355 

This patent was filed back in 2002, so evidently twelve years ago someone in HP felt that such a thing could be built. Funny thing about patents, BTW. To get a patent, it must be "reduced to practice". Contrary to what you would normally think, this isn't a requirement that you actually build and test your invention. You can "reduce an invention to practice" just by describing it. Writing the patent application in enough detail is a way of reducing the invention to practice. Go figger. So, I don't know whether this HP invention was ever built. HP did eventually abandon the patent, which usually means that they decided it wasn't worth paying the maintenance fees.

Another feature of the HP invention is that one embodiment includes a color sensor. I quote from the patent: 

The embodiment of FIG. 4 incorporates a scanner (151). This scanner (151) is preferably a three element Charge Coupled Device (CCD) array. Each of the three elements detects photons of a particular wavelength, i.e., color. 

Thus, when the scanner (151) scans a color, the three elements of the CCD array will output signals indicative of the ratio of each of the three primary colors in the scanned color. Consequently, the pen (100b) can be programmed to duplicate those ratios of the three colored inks in the cartridge (102) to reproduce the color scanned by the scanner (151). 

Consequently, the user can sample a color with the scanner (151) from any object at hand. The pen (100b) will then write in that sampled color.

This appears to describe many of the features of the Scribble pen. Uh-oh. Does that mean that the Scribble pen would infringe? That's a deceptively simple question. First, you need to look at the claims to determine infringement. Disclaimer: I am not a patent attorney, and I haven't spent the time to do a thorough analysis, but I did notice one thing in the first claim. In order to infringe this claim, the pen must have a button for every color of ink. So, it looks like the Scribble pen as described would not infringe on claim 1 of this patent. But as for claim 2, who knows? I will leave that up to someone else to decipher. 

But, to do a complete "freedom to operate", one would have to do a thorough patent search, and then do this sort of analysis for every claim in every relevant patent that turns up. In doing so, one would probably stumble upon US patent #8,292,527, which lists a fellow by the name of Kia Silverbrook as one of the inventors. I have heard this name before. As of March 26, 2014, Silverbrook has 246 times more US patents than I do. I guess that's sorta impressive. He's got more patents than God, Thomas Edison, and Shaquille O'Neal put together; 4,665 and counting.

I did not spend a lot of time digging up prior art. I would do this if I were to be submitting a patent application (just to make sure the claims were reasonable) and also to get a sense of what can be built without infringing. My meager prior art search turned up quite a few other patents that could be interesting, so I am going to take a wild guess that deciding on freedom to operate might get a bit involved.

There are a few conclusions here. First, it would appear that the technology to do the Scribble pen exists today or will exist soon. This bodes well for the Scribble folks. Second, it appears that the patentscape might perhaps be a bit rough. Without further investigation, it's hard to say whether this will be an issue for them.

Output colors

Ok... now it's time for me to vent. And scratch to scratch my head. Their press release states that the "innovative pen ... can reproduce over 16 million unique colors." Come on. Really? Where does this number come from?  

First, I'm gonna say straight out that the number did not come from a color scientist. I wrote a very entertaining blog about how many colors there are, and then another where I gave the definitive answer: 346,005. I'm sorry. There aren't 16 million colors in the universe, so you can't produce that many colors. That is, if you define "color" to mean "a unique visual sensation". The image below shows nine "unique" shades of pink. If you can see a difference between them, then perhaps you are able to sense 16 million colors.

Nine of the 16 million purported colors

Of course, I'm just being persnickety about the definition of the word "color", but still, it's a BS number. It's likely that the sensor has three outputs which are each eight bits. That would give 16 million possible inputs to the software. That's not the question though. The question is how many distinct outputs there are. How many distinct levels are there for each of the six inks?

That's the end of my rant on 16 million. Now for the head scratching part. Assuming that they computed the 16 million based on 256 levels for each of three channels R, G, and B... What happened to the C channel? Remember? The sensor is an RGBC sensor. Four channels of eight bits would give 4.3 billion colors. 

Or maybe they are talking about the total number of distinct ink mixtures that they can make? That's an interesting combinatorial problem that I don't want to get into right here.

Ok, so the 16 million number is probably marketing hype. Certainly not any sort of proof that the whole idea is bogus.

Conclusion

In the Color Picker Pen post, I came to the conclusion that the pen that was portrayed was not anything real. In the case of the Scribbler pen, I didn't find any red flags. As far as I can tell, they know what they are doing. It  isn't a final product - I mean, they are doing a Kickstarter to fund the product development. But from the limited information that has been made available, it all looks legit.

But, all the usual disclaimers hold. I am not a patent attorney, and I am not an investment counselor. For that matter, my wife would probably tell you that I can be an idiot at times. My personal liability in your investment in the Kickstarter campaign is limited to the amount of money that you paid me for the opinion. So far, I think that's zero.