Wednesday, May 30, 2018

How well do we remember color?

In a previous (and highly entertaining) blog post, I reviewed two studies that tested people on brand color recognition. The studies were not peer-reviewed, Nobel prize winning efforts, but enough effort was put into them for me to find the results suggestive. I don't mean that in a salacious way; I do mean that the experiments suggest that our recognition of brand colors is not as good as we might think it is.

The sight of Starbucks periwinkle makes me thirsty

In today's post, I will take a dive into what peer reviewed Science has to say on the matter, specifically on the topic of how accurately we remember colors.

What I didn't study

There are a lot of stones in this field, and I left a lot of them unturned. Here are some exciting topics that I skipped over...

Color and emotion - This is a big field. It would be cool to really dig into how color effects us emotionally. I would love to separate the wheat from the chaff (valid research from pontification). The paper by Yu et al. Looks like it would be fascinating. But, I didn't really look at it for this blog post.

Color and emotion and brands - Another mondo interesting study would be to look at our emotional reaction to the color of a box of cornflakes. Presumably, there is an ideal corn flake box color that will seduce the unwary buyer into filling the shopping cart. Gosh, I would love to learn all about that. I bet the paper by Rupert et al. would be a great place to start.

Color constancy - There has been all this research around the cool topic of how it is that our perception of the color of something doesn't change all that much when the illumination changes drastically. Like from the yellowish incandescent light in your living room to the blueish light outside. There is a huge difference in the spectra of the light hitting your eye, but we still see white paper as still being white. This is the kinda topic that makes people want to quit their day job and become a PhD candidate! But I'm not going to talk about it. Well... maybe I will mention it in passing.

Many roads diverged in this multicolored wood, and I am sorry that I could not travel them all. I took the one less traveled. 

Color memory

Here's what I am thinking: I have a picture in my head about what Starbucks green is. I see it as a darker shade of green that might be just a tiny bit toward the blue end. But, my memory of Starbucks periwinkle is wrong. If you don't believe me, just ask my wife. She is the world's authority on all my shortcomings, and would love to acquaint you with my multifarious imperfections. But, everyone's memory of the official Starbucks green is not quite perfect. How far off are we?

I just can't remember the name of this film

We all know that a banana is yellow, and a school bus is a slightly orange flavor of yellow, and the color of an orange is slightly weaker than true orange, but that carrots are true orange, which is why we call them carrots. These are all memory colors.

Carrots should really be called oranges

The idea of memory colors dates back to Ewald Hering in 1878. Loyal readers will remember that Hering has been mentioned in a few of my previous blog posts. Hering is the guy who developed the color opponent theory. This theory says that we can assess any color in terms of three attributes: where it fits between white and black, where it fits between red and green, and where it fits between yellow and blue. This is how color is encoded on its way to the brain, and this idea was baked into CIELAB.

Hering's three attributes of color

Hering made another contribution to color science. He said that our perception of the color of an object is effected by our memory of the color of a prototypical object. Like, if we see a banana that is kinda yellow, but not quite, we will remember it as being yellow. Remember how I said that I was gonna mention color constancy? Now is the time. Hering's theory has been used to explain color constancy. If the yellow of a banana has a slightly unusual shade, then our brain will use that fact to deduce the color of the illumination.

There is a prediction from Hering's theory of color memory that is important for the purpose of this blog. If his theory is true, then our memory will tend to bias toward the quintessential version of that color. We will remember our bananas being yellower than they really are.

David Katz reiterates Hering's theory:

... in the imagination we exaggerate colours of objects whose colours are generally distinguished only in terms of brightness, darkness or hue. If we ask a person to pick out a blue which will match the colour of the eyes of someone he knows very well, he generally selects a blue which is too saturated. If we ask a person to match a brick, he usually chooses a black which is too deep or  reds which are too highly saturated. Almost always he selects a colour which is too bright to match a bright object, one which is too dark to match a dark object, and one which is too saturated to match an object which is known to have a distinct hue.


Katz liked Hering's theory on the distortion of colors by our memory

If this theory is true, then our color memory is flawed, so our recollection of Starbuck's green is flawed. The practical message for everyone in the business of making sure that Starbucks has the correct shade of green is that the exact color of the logo doesn't really matter, since we will identify the logo by it's shape, and our brain will translate the color into the correct shade of green.

Wow. Big stuff here.

What does the research say?

Hering was a brilliant guy, and rates up there with Munsell as one of the Fathers of Color Science. But he was largely a theoretical guy. Have experimental results backed up his theories of memory colors?

An early investigation by Adams provided evidence that agreed with Hering, but it was inconclusive. 

From our investigations of the perceptions of the five natural objects grass, snow, coal, gold and blood, we may say that Hering and Katz were correct in claiming that the seeing of these objects is ordinarily affected by memory color. Although our investigation failed to give a quantitative measure of a single memory color, it was thus not unfruitful.
Adams (1923)

Five memory colors from Adams' paper

Not unfruitful? I didn't see a single fruit in their list of memory colors! It is likely that the hard results were lacking because of the lack of rigor in this paper. To be honest, the paper reads like a cheap and boring novel with pages and pages of one anecdote after another, and then a short experiment.

Bartleson provided a more rigorous test of memory colors, using colors of ten familiar objects: red brick, green grass, dry grass, blue sky, flesh, tanned flesh, broad-leafed summer foliage, evergreen trees, inland soil, beach sand. Here is their assessment:

Each memory color tended to be more characteristic of the dominant chromatic attribute of the object in question; grass was more green, bricks more red, etc. In most cases, saturation and lightness increased in memory. 

There is evidence of increased saturation in the memory colors. In most cases there are hue shifts with memory in the direction of what is probably the most impressive chromatic attribute of the object in question.
Bartleson (1960)

The grass is always greener in our memory

A bit more recently, Siple et al. did a similar test with six foods (carrot, corn, lettuce, lime, orange, and peanut), and agreed with the theme that we remember colors as being more vivid. Note that their test was the first that was literally fruitful, since they included oranges and limes.

Results indicated that, for hue and brightness, memory and preference were quite accurate for the objects tested; however, all subjects remembered and also preferred all items to be more highly saturated.
Siple and Springer, 1983

One could argue that trying to recall the color of grass is a bit problematic. After all, the color of grass varies with species. Is it Kentucky bluegrass? Fescue? Rye grass? Easter grass? The color also varies with the plenitude of rain, nutrients, and sun. And where my doggies have visited.

Three recent papers (one by Bloj et al., one by Pentz, and the final by Newhall et al.) sought to eliminate this problem of variability of the colors of real objects. Bloj asked subjects to bring along a familiar object. Even when recalling those familiar and well-defined colors from memory, the conclusion of this paper was that "Our results, on average, confirm that objects are remembered as more saturated than they are."

Somehow Johnnie graduated from Kindergarten despite his sub-par drawing skills

On to Pentz's paper. He taught a color class for several years, and recruited the members of each class for an experiment, eventually testing 283 people. The people in the class were shown a blue piece of plastic and were told that they would need to recollect the color later on. They each had a chance to hold the plaque and could look at it as long as they wanted. Later, they were shown a collection of 24 plaques which included the one they had looked at. "Only thirty percent of participants at plastics coloring seminars were able to correctly identify a color observed only an hour earlier." While the correct plaque was determined by the most participants, the two next most likely guesses had a color difference of about 10 ΔE from the correct plaque.

(How big is 10 ΔE, you may ask? Imagine a color match that is as big as you might consider acceptable for production. Then multiply that by 2 or 3.)

Newhall et al. looked at our short term memory of colors in order to eliminate the ambiguity for familiar objects. The subject was shown a color for 5 seconds, given a 5 second break, and then asked to adjust knobs to recreate that color. Here is one of the conclusions from the paper:

Significantly more purity and somewhat more luminance were required to complete the color matches by memory than were necessary for the simultaneous matches. This principal result was confirmed by the results of three supplementary experiments.
Newhall, Burnham, and Clark (1957)

Newhall et al. found that we remember colors as being higher in chroma (by 1.7 steps in Munsell chroma) and somewhat lighter. They also found no consistent change in hue between what we remember and what we see.

The typical migration pattern of a color trapped in a brain

How big is a step in chroma of 1.7? The data is all in the paper. I could type it all into a spreadsheet, convert it into CIELAB, and then compute color differences. But I could just be lazy and wave my hands. Fully saturated colors go up to maybe 15 in Munsell chroma, and maybe 100 in CIELAB C*. A step of 1.7 in chroma is roughly 10 ΔE. I dunno. I think this is kinda big for a 5 second delay.

One common theme from these experiments is that colors are remembered more vividly than they actually are. Whether or not colors are lighter in our memory and whether there is a systematic error in the hue are both up for debate.

Colligation

The ancient Greek, Ptolemy, developed a set of equations that could be used to predict the positions of the planets at any time. The equations were based on a lot of wrong assumptions, like "the Earth is in the center, and all the rest of the celestial bodies move in circles that revolve around other circles". The model worked, at least to an extent.

A millennium or so later, Copernicus decided that the Sun belonged at the center. Then Kepler came along and decided that ellipses made the whole thing simpler than the circles in circles thing, and then Newton provided the big colligation. The inverse square law of gravity was the grand unifying theory that explained the whole enchilada. If the pull of gravity goes in inverse proportion to the square of the distance between the objects, then planets will travel in Kepler's ellipses. Eureka!

By the way, colligation means "to subsume (isolated facts) under a general concept". I really love that word. It explains the essence of what I think it is to do Science: to find simple theories that explain lots of data.

We are ready for a colligation of our understanding of color memory. We have all this data from these studies. We have a generalization of how saturation changes when it gets implanted into our memory. It's a bit fuzzy what happens with hue and lightness, since the data doesn't always agree. We need an explanation that can tie it all together and explain some of the anomalies.

In this case, the colligation was fairly recent, by Bae et al. in 2015. The idea is pretty simple. We have a limited number of folders in the filing cabinet in our head. Although we can distinguish perhaps millions of colors, there are eleven basic folders where we store colors, at least in the English speaking world. The folders are labeled white, black, gray, red, orange, yellow, green, blue, pink, brown, and purple. This was the result from Berlin and Kay, and also in the groundbreaking experiment that I never got around to publishing.


Here is the grand and glorious theory of color memory distortion.....

When we want to store the color of an object in our brain, the first step is to categorize it into one of perhaps eleven archetypal colors. From there, presumably, we may make modifications to distinguish from the archetypal color (yellowish green, or dark red), and the modifications get stored along with the general category. Later when we retrieve that color from our memory, the archetypal color gets weighted a bit more than the modifier.

Evidence from Bae

Bae et al. had the participants try to remember 180 colors, equally spaced in hue, all with L* = 70 (fairly light) and C* = 38 (somewhat saturated). They saw the color for 100 msec, the color was removed for 900 ms, and then they had a chance to select the color from a ring.

The results of the paper are summarized in my drawing below. There are seven regions. Within any of the regions, for example, the blue one, people will tend to distort the hue toward the solid line which represents the archetypal example of that hue. (Four of the eleven basic colors were left out. White, black, and gray don't have a place on the hue circle. And since their L* was fairly high, they missed out on brown. They almost missed red.)

Hieroglyph found in a Mayan tomb, hitherto-for undecipherable 

This explains why the hue of a color sometimes shifted in the experiments, and sometimes not. But the color memory experiments seem to all agree on one thing. Our memory of a color is generally more saturated than the actual color. How does the eleven-folder theory of color memory explain this?

Here is a quote from Heider that can explain this:

It was quite clear, without further analysis, that the most saturated colors were the best examples of basic color names both for English speakers and for speakers of the other 10 languages represented.

When we think red, we don't think some wimpy-butt red. We think fire-engine-lipstick-Corvette-candy-apple-OMG-I'm-bleeding red. It only makes sense that most of the colors that were tested in the experiments cited above would not be the most saturated colors imaginable. Hence, our memory would tend to distort most of the colors in the experiments toward the extreme of saturation.

There are some archetypal colors that don't fit Heider's hypothesis, namely brown, gray, and pink. I would take a wild guess that these are the exceptions to the rule in the previously described experiments.

Why eleven?

Bae's research suggests that the eleven basic color names are the appropriate number. Or rather, it does not suggest that there are colors beyond the seven which qualify to be archetypal colors. But Bae's experiment, awesome as it was, only looked at 180 colors - all of which had the same L* and C*. There is quite a bit of uncharted color space.

Based on my personal experience, I would like to think that I have more than just eleven archetypal colors. I mean, I see tan, coral, olive green, and plum as distinct colors in their own right.

My candidates for induction into the Hall of Archetypal Colors

In some languages, such as Russian, Japanese, and Italian, there is a separate word for light blue which stands on its own as a distinct color. So, maybe there are twelve archetypal colors? Dimitris Mylonas (Mylonas and MacDonald, 2015) suggested that lilac and turquoise also belong on the list. In two other papers, he has named a much larger collection, including cream, lime, olive, salmon, mustard, peach, tan, and coral.

So, I don't think we can say at this time that there are exactly eleven colors that serve as archetypal colors in our memory. There could be more. It also seems quite plausible (to me) that the number is different for different people. I would think that people who deal with colors all the time (like artists, graphic designers, fashion designers, interior decorators, and the spouses of color scientists) might have developed a wider collection of focal point colors. On the other hand, it could be that the relatively small collection of focal point colors are a result of something hardwired in the brain.

Here's another interesting thought. We know that some people have perfect pitch, an uncanny knack to identify musical notes. All of these studies looked at people's color memory in the aggregate. Perhaps there were a few individuals whose superpower is to have perfect hue? I have heard more than one person make that claim. Of course, one person who made that claim also told me that he was raised from infancy by a troop of iguanas in a volcanic crater. He probably learned it from them.

All interesting stuff for further research!

Non-References

Burnham, Robert W., and Joyce Clark, A Color Memory Test, Journal of the Optical Society of America, Vol 44, No 8, Aug 1954

Rupert, Andrew Hurley, Rachel Randall, Liam O'Hara, Charles Tonkin, Julie C. Rice, Color harmonies in packaging, Color Research & Application, Volume 42, Issue 1, 28 March 2016

Yu, Luwen, Stephen Westland, Zhenhong Li, Qianqian Pan, Meong Jin Shin, Seahwa Won, The role of individual colour preferences in consumer purchase decisions, Color Research & Application, Volume 43, Issue 2, 10 October 2017

References

Adams, Grace Kinckle, An Experimental Study of Memory Color and Related Phenomena, The American Journal of Psychology, Vol. 34, No. 3 (Jul., 1923), pp. 359-407

Bae, Gi-Yuel, Maria Olkkonen, Sarah R. Allred, and Jonathan I. Flombaum, Why Some Colors Appear More Memorable Than Others: A Model Combining Categories and Particulars in Color Working Memory, Journal of Experimental Psychology: General, 2015, Vol. 144, No. 4, 744–763

Bartleson, C. J., Memory Colors of Familiar Objects, Journal of the Optical Society of America, Vol 50, No 1, Jan 1960

Berlin, B., and P. Kay, Basic color terms: their universality and evolution (Stanford, Calif.: Center for the Study of Language and Information 1969).

Katz, David, The World of Colour, Kegan, Paul, Trench, Tubner, 1935, p. 164

Heider, Eleanor Rosch, Universals in color naming and memory, Journal of Experimental Psychology, 1972, Vol. 93, No. 1, 10-20

Hering, Ewald, Outlines of a theory of light sense, Grundzüge der Lehre vom Lichtsinn 1905, translated 1964, Harvard University Press

Mylonas, Dimitris and Lindsay MacDonald, Online Colour Naming Experiment Using Munsell Samples, European Conference on Colour in Graphics, Imaging, and Vision - CGIV, June 2010

Mylonas, Dimitris, Mathew Pruver, Mehrnoosh Sadrzadeh, Lindsay MacDonald, and Lewis Griffin, The Use of English Colour Terms in Big Data, May 2015, AIC Midterm 2015

Mylonas, Dimitris and Lindsay MacDonald, Augmenting Basic Colour Terms in English, Color Research and Application, Volume41, Issue 1, February 2016

Newhall, S, M., R. W. Burnham, and Joyce R. Clark, Comparison of Successive with Simultaneous Color Matching, JOSA 47, No. 1, Jan 1957

Pentz, Anthony J., Does color memory exist?, SPE/ANTEC 1999 Proceedings (Society of Plastics Engineers Annual Technical Conference and Exhibit)

Siple, Patricia, and Robert Springer, Memory and preference for the colors of objects, Perception & Psychophysics, 1983,34 (4), 363-370

No comments:

Post a Comment