When regression goes bad

There is always a temptation, when one is using regression to fit a data set, to add a few more parameters to make the fit better. Don’t give into the temptation!

Temptation

In the case of polynomial regression, this might be the delectable temptation of adding one more degree to the polynomial. If y(x) is approximated well by a weighted sum of 1, x2, and x3, then (one could reasonably assume) adding x4 and x5 will only make the fit better, right? Technically, it can’t make the fit worse, but in most real cases, the fit to the data will be improved with every predictive term that is added.

If one is doing multiple linear regression, the same thing applies. Let’s say, for example, one is trying to determine a formula that will predict tomorrow’s temperature. It would be reasonable to use the temperature today and yesterday as predictors. It might reduce the residuals to add in today’s humidity and the wind speed. The temperature from surrounding areas might also improve the fit to historical data. The more parameters are added, the better the fit.

There is, of course, a law of diminishing returns. At some point, each additional parameter may only reduce the residual error by a miniscule amount. So, simply from the standpoint of simplicity, it makes sense to eliminate those variables that don’t cause a significant improvement.

But there is another reason to be parsimonious. This reason is more than a matter of simple convenience. Adding additional parameters has the potential of making regression go bad!

A simple regression example

Let’s say that we wanted to use regression to fit a polynomial to a Lorentzian1  function:

Equation for a Lorentzian

This function is fairly well behaved, and is illustrated in the graph below.

Plot of the Lorentzian

Oh… there’s a question in the back. “Why would someone want to do polynomial regression to approximate this? Isn’t it easy enough just to compute the formula for the Lorentzian directly?” Yes, it is. I am using this just as an example. This is a way for me to get data that is nice and smooth and where we don’t have to worry about the noise, and we can be sure what the data between the data points looks like.

The plot above shows knot points at integer locations, -5, -4, -3 … 4, 5. We will use these eleven points as the data set that that feeds the regression. In the graph below, we see what happens if we use a fourth order polynomial to approximate the Lorentzian.

Fourth-order polynomial fit to the Lorentzian

While the overall shape isn’t horrible, the fourth order polynomial shown is red, is a bit disappointing. Elsewhere it is not quite as bad, but at zero, the polynomial predicts a value of 0.655, whereas it should be 1.000.

What to do? Well, the natural response is to add more terms to the regression. Each term will bring the polynomial closer to the knot points. In fact, since there are eleven data points to go through, we know that we can find a tenth order polynomial that will fit this exactly. Clearly this is a good goal, right? Trying to drive the residual error down to zero, right? Then we will have a polynomial that best matches our Lorentzian curve, right?

I should hope that I have done enough foreshadowing to suggest to the reader that this might not be the best course of action. And I hope that the repetitive use of the italicized word “right” at the end of each question was a further clue that something is about to go wrong.

A tenth-order polynomial fit to the Lorentzian

And wrong it did go. We see that the graph of the tenth-order polynomial function did an absolutely fabulous job of approximating the Lorentzian at the knot points. It goes right through each of those points. We also see that the fit in the region from -1 to +1 the fit2  is not too bad. However, as we go further away from zero, the fit gets worse and worse as it oscillates out of control. Between 4 and 5, the polynomial traces out a curve that is just plain lousy. Linear interpolation would have been so much better. Something went bad, and it wasn’t last week’s lox in the back of the fridge.

Let’s make this example a bit more practical. Often when we are doing regression, we are not interested in predicting what happened between sample points, but rather we ultimately want to predict what will happen outside the range. Interpolation is sometimes useful, but we often want extrapolation. In the weather prediction example, I want to know the weather tomorrow so I can plan a picnic. I don’t usually care about what the temperature was an hour ago.

If we look out just outside the range in our example, things get really bad. No… not really bad. It gets really, really bad. I mean awful. I mean “Oh my God, we are going to crash into the Sun” bad. The graph below shows what happens if we extend the approximation out a little bit – just out to -8 and +8. Just a few points beyond where we have data. How bad could that be?

A tenth-order polynomial fit to the Lorentzian, extended to -8 to +8

The observant reader will note that the scale of the plot was changed just a tiny bit. The polynomial approximation at x=8 is -8669.7. I asked “how bad could it be?” Well, if this model is going to predict tomorrow’s temperature to be anything below eight thousand degrees below zero, I am going to give up my plans for a picnic and party like crazy tonight because all the beer in my fridge will be frozen tomorrow.

What went wrong?

In this example, there were two things that caused this regression to go horribly wrong. The first is that I tried to push regression farther than it should be pushed. Under no conditions should anyone use regression with that many parameters when you only have eleven data points3. People who do that should be sent away to have a picnic somewhere where it is eight thousand degrees below zero. I am going to make the bold statement that, even if you have 11,000 data points, you should be very careful about using eleven parameters in your regression.

The second thing that went wrong is that the mathematical model used to fit the data was not so good. The Lorentzian function does something that polynomials are just not good at – it has an asymptote. Polynomials just aren’t good at asymptotes. The only polynomial that comes close to having a vertical asymptote of zero is the polynomial that is identically zero everywhere. And that polynomial is just plain boring. Even if we were a bit more reasonable about the order of the polynomial that we used, we would never get a good approximation of the Lorentzian out in the tails by using polynomials.

This underscores the importance of taking the time to understand your data, to get to know its needs, and especially its aspirations for the future, in order for you to use a function that might be able to successfully approximate the true underlying function.

1. This is function goes by quite a long list of names, since it was invented independently for different reasons. Someday I will write a blog explaining this function’s fascinating history. 2. For the benefit of the astute and persnickety reader, I realize that I am using the word “fit” incorrectly. The term should be applied strictly to just how the approximating function behaves at the points used in the regression. It would have been more appropriate for me to have said “fit to the original function”. 3. Ok… My statement here is too strong. A discrete Fourier transform is a regression, although we don’t normally think about it that way. Many folks, myself included, have successfully avoided Siberian consequences using a DFT that has as many parameters as data points. (But they still have to deal with Gibb’s phenomena.) I apologize to anyone who just stuck their mukluks into their picnic basket.

Goldilocks and the three research projects

Once upon a time, I volunteered to be the chaperone for a youth service project. We traveled to the Appalachians and spent a week repairing the roof of an old tar-paper shack1. With a hoard of high-energy, hormone-and-Mountain Dew powered teens running up and down ladders, using hammers and power tools and having all-night flatulation contests, it was inevitable that one of these irresponsible youth would get injured.

And I did! Playing volleyball.

A quick trip to ER

I tore a tendon in my ring finger. As I remember, the tendon snapped when I drove a thunderous spike over the net into the gaping jaws of the awestruck opponents. Some of my jealous team mates, on the other hand, claim that I got injured tripping over my jock. They claim that I jammed my finger because it was stuck in my nose when I fell. Don’t believe them.

Actual unretouched photo

It was a very odd injury in that it did not hurt. I just couldn’t hold my finger straight. Being a role model for the group whose job was to model responsible behavior to the youth, I finished the game. And then went swimming with the group. When we pulled back into the center at 9:00, I asked another of the adults (one who had more sense than I) to accompany me to the nearest emergency room, which was a half-hour’s curvy drive through the mountains to the next town.

The emergency room was crowded. The first set of the night’s fatalities from the barroom brawls were just finding their way in. I looked around the room at the fellow whose wife was holding his finger. She was sitting across the room, scowling at him and clutching a baggie filled with ice. I saw a man who was hit hard enough that you could still read the PBR label (in reverse) on his cheek, and a woman who would clearly need a frock of professional seamstresses to put her leg back together.

I am glad I brought the other adult with me. I don’t think that the rest of the patients would have been much in the way of company as I waited. .... and waited. I believe it was about 12:30 when another guy turned to me and said, “I’m glad I wasn’t hurt real bad, or I’d be dead by now!” I smiled and agreed with him. It’s always polite to agree with someone when they say they are glad not to be dead. But on the inside, I didn’t agree with him. If he had been hurt worse, he would have been taken back to see the doctor earlier.

Triage

Medics on a battlefield perform a process called triage to prioritize the incoming. Triage comes from a French word meaning, “Honey, pick up some Spam. We’re having company for dinner.” Luckily for this essay2, the American meaning of the word triage has taken a different meaning.  What triage means is to divide something into three groups. In medicine it means that incoming wounded are separated into the following three groups:

1. Those who likely to die, despite all heroic lifesaving measures, 2. Those who are likely to live, and escape permanent disability and major disfigurement, even if the doctor is not able to work on them immediately, and 3. Those on whom immediate medical attention can have a significant impact.

The doctors concentrate their efforts on those patients in the third category until such time as they can work on the first and second groups. In this way, the scarce resource of the doctor’s time is used for the greatest good.

In the case of my emergency room visit, the person who admitted me clearly (and justifiably) placed me in the second group, so I had low priority. That is why I and the fellow who complained of the wait were serviced last.

Some health care critics claim that, in reality, the three triage groups are the following:

1. Those who are poor and uninsured, 2. Those who are who have insurance or money, but do not have symptoms of any expensive diseases, and 3. Those on whom a walletectomy is indicated.

Whether these bitterly cynical health care critics are right about the way triage is performed is irrelevant to the story-line.

Applied research management by triage2

In addition to being a really cool sounding word, triage provides a good model for prioritizing applied research. For applied research, “incoming wounded”, that is, sub-projects one is considering spending time on, are divided into these three groups:

1. Those aspects that will probably die, despite all heroic lifesaving measures, 2. Those aspects of the project which are likely to live, and escape permanent disability and major disfigurement, even if the researcher is not able to work on them immediately, and 3. Those aspects on which immediate research attention can have a significant impact.

I will focus the rest of this essay on the first of these, the terminal project.

The terminal project

It happens on occasion, that applied researchers find themselves working on an approach to a problem that just will never pan out. This in itself is part of research. After all, if the “correct” solution were easy, it would be called development! Driving down a dead-end street is only a problem if that street is also one-way. So long as one recognizes the dead-end, and turns around, there is no problem.

But we often find ourselves playing in the sandbox. (Or, more often, we find our coworker playing in the sandbox.) We see in him that “glazed over” look of one obsessed with meson transmogrification units. We hear the ubiquitous, “Just one more month, and I will have it!” We hear the unheeded cries of the prophet in the wilderness, beckoning the rest of the group to embrace the technology which will save the company from complete and utter destruction and humiliation.

Let’s face it, there are a lot of prima dons4 out there. I have worked in five companies in my fourteen-year professional career, and there has always been one where I have worked. (At least until I moved on to the next company that was foolish enough to hire me! Funny how a company’s problems went away when I left.)

Is this solely a matter of egos? I think that this may often be the case. The errant researcher is dazzled by the potential of becoming the next Watson and Crick, or Einstein. They justify their search by rationalizing that no sarcophagi have been found by digging where there are already shovel marks.

Warning signs of a terminal project

I also think that there are other routes by which one can get stuck in a one way dead end, routes that do not require admission of a character fault and years of psycho-therapy5 to solve. For those who suspect that a terminal project may be something other than ego-induced, I offer some warning signs...

How many levels deep is the project? Consider the researcher who set out to build a potato chip conveyance mechanism. The decision was made to use a magnetic levitation device to move the chips. The advantage of nearly frictionless motion is immediately obvious to all but the densest of colleagues.

After many failed attempts, the researcher realizes that the magnetic fields created by conventional means are not strong enough. The obvious solution is to build a stronger electromagnet.

After many attempts to build such a behemoth, it is realized that the basic bottleneck is the resistance in the wires in the electromagnet’s core. Obviously, a superconducting electromagnet is the solution.

After many attempts to wrap superconducting ceramics around an electromagnet’s core, it is realized that ceramics don’t bend very easily. So the search is on for a room-temperature, malleable superconductor.

I think that the line between applied and pure research has been crossed, and the researcher has completely lost sight of the original job, that of moving potato chips. If only the researcher would have noticed the nesting of sub-projects and realized that there is another way to convey potato chips! It doesn’t take a rocket scientist to see that the simplest solution is to impregnate the potato chips with iron so that they can be magnetically levitated6.

Iron fortified potato chips The Uni-disciplinarian, or ‘How Meteor Crater Company got screwed’  - Once upon a time, there was a company named Meteor Crater that built wooden boxes. They had been relying on a certain supplier of nails for as long as anybody could remember when that nail supplier had the gall to go out of business. In a panic, the purchasing people had ordered an equivalent part from another vendor, but they were having big problems on the shop floor. The CEO called an emergency meeting7 of the executives to decide what to do about this crisis.

One junior executive offered a suggestion. “I met this fellow at a conference last year. He is supposed to be the leading authority on hammer technology.” After much discussion, and few other promising ideas, Dr. Sledge was called in.

“Eenterezting...”, the doctor said as he inspected the new nail. “I haff neffer zeen zuch a nail. Vot iss dis shpiral doo-hickey zat goess from zee point arount und arount to zee head? Und vy do zey haff ziss zlot at zee ent?” He set a nail on it’s point on a piece of wood, and gently tapped it with a hammer. Seeing no response, he hit it harder, and then harder still. The wood finally split to allow the nail to enter.

“Ya, ve haff a beeg problem here. Ve clearly need a prezision hammer. Vun zat can deliffer a blow hart enoof to zet zee nail, but not zo hart as to shplit zee vood.”

Und zo Herr Doctor (oops, excuse me) And so the doctor set about to build this precision hammer. His first attempt was to use calibrated titanium weights, dropped from calibrated distances. Sledge had some initial successes and failures, but could not consistently set the curious spiraled nails. He eventually hit upon the idea of hammering under vacuum. By eliminating the air resistance, he could reduce the variability in the force of a hammer blow which was due to changes in air resistance.

This brought an incremental improvement in the yield. Sledge calculated the current yield (probability of a successful bond) at about 37%. It was then that the doctor realized the next limiting factor: fluctuations in the earth’s gravitational force8. He experimented for months with various means for propelling the hammer “head” (now resembling a bullet) into the nail.

I could go on and on about the incremental improvements which Doctor Sledge delivered. To make a long story short, Meteor Crater finally quit the crate building business and set up a tourist shop in Arizona, and Sledge wound up selling $5,000 hammers to the Pentagon. Meteor Crater got screwed because nobody thought to investigate screwdriver technology. The moral of this story is that “to a man with a hammer, the world looks like a bunch of nails.”

The golden hammer

I remember seeing a sign in an engineer’s office that compared specialists and generalists. The specialist is one who knows more and more about a narrower and narrower subject until eventually, he knows everything about nothing9. The generalist is one who gradually develops a broader and shallower understanding until he knows nothing about everything.

In my opinion, the specialists are ideal for pure research. Pure research is drilling a well. Generalists, on the other hand, are the best applied researchers. They are best able to objectively weigh all the possibilities. Applied research is clearing the topsoil from a field.

Violating the laws of physics - When I was a lad of 14, my father took me with him on a business trip. He drove to the northern part of the state to talk to an inventor who was looking for financial backing. The inventor had some kind of improvement to the Wankel engine. I had no idea what he was talking about, but it sure sounded neat to me.

He also showed us a perpetual motion machine he had been working on. There was a cast aluminum block, about 18 inches square. Inside this block was an aluminum flywheel. Both the block and the flywheel had permanent magnets and springs and gears. It was all orchestrated to bring north and south poles together just when the flywheel needed a little extra momentum. The magnets would supply the extra kick to keep the wheel going around.

The inventor gave the wheel a spin, and it whirred for a while and came to a stop. “Well, it needs just a bit of adjusting, but I’ve almost got it there.” [Well, it was almost perpetual!] I wanted to learn more about this amazing machine, but my father politely excused us. Why my father would turn down the opportunity to be involved in the development of an infinite source of energy was beyond me!

I met a man who had an idea for a new medical imaging device. It would allow a doctor to see things that you could not see with any existing modality: ultrasound, nuclear magnetic resonance, and computer-aided tomography. The only problem was, such a device would break some basic laws of physics. I tried to reason with him, but he told me, “Wait until I get the Nobel prize. Then you will see.” The student of psychology will recognize this as the manic phase of a bipolar disorder.

Basically speaking, anything having to do with perpetual motion, time travel, moving at twice the speed of light, cold fusion and changing lead into gold can be pretty safely removed from the project list for applied research.

When once around the block starts looking like a Sunday drive - The “once around the block” strategy was designed to minimize the potential for spending too much time on the terminal aspects of the project. When properly applied, each trip around the block serves as a reminder of all the other issues concerning the project. It puts a bit of structure around an otherwise unstructured process.

The “once around the block” process can also serve as an indicator of a project which has gone terminal. Researchers tend to get stuck on one of the trips around, inspecting all the wiring in the Empire State Building. When they stop making regular trips around the block, there may be a terminal project budding.

Conclusions

This essay introduced a triage technique for determining the appropriateness of a project in an applied research setting. The triage divides aspects of a project into pure research, applied research and product development. Focus was then put on one painful aspect of applied research, the terminal project. Future essays will consider how to draw the line between applied research activities and development.

Notes

[1] By the way, if you are looking for a posh vacation spot where you can sip umbrella drinks and get your toenails done, I heartily recommend volunteering for a week the Appalachia Service Project. [2] ... and the poor reader who is being forced to read this against his or her will. [3] Although the differences are often subtle, management by triage is not to be confused with ménàge a trois. The former is considerably more fun. I recommend it for any party that is getting stale. [4] “Prima donna” is the feminine version. Calling a male researcher a prima donna is improper grammar. [5] ... or three weeks on Prozac! [6] Note that there is a huge market of anemic women who love potato chips. By the way, I have submitted the patent application. I can optimize the levitation capabilities by completely doing away with the potatoes in the chip. My new Imitation Potato Flavored Iron Chips are currently being manufactured in machine shops around the world. They pay me to clean them off the floor! [7] “Meetings” are what companies call to stave off a crisis. The theory is that crises occur when things change too fast. Meetings are well know to slow things down, so they quite effectively delay the crisis. Not only that, but they also free up the people who do the real work to actually deal with the crisis! [8] Those of you with bathroom scales may have noticed this phenomenon. I myself have put much research into reversing the trend for this force to gradually increase over time. [9] Signal processing theory aficionados will recognize an allusion to the Dirac delta function.

You can tune a piano...

You can tune a piano, but you can’t tuna fish.

Or so goes the title of the REO Speedwagon album. I have always disagreed. Both parts are wrong. You can tune a fish by going up and down the scales. And, contrary to popular belief, you can’t tune a piano. Read on, and I’ll explain why you can’t.

The circle of fifths

As a first example, we consider the circle of fifths. No, I don’t mean a circle of my buddies Jack Daniels, José Cuervo, Johnny Walker and Jim Beam.

I mean the circle of fifths on the piano. From C to G is an interval known as a fifth. From a G to a D is another fifth. This eventually winds up back at C, so the whole thing forms a circle with twelve notes. I suspect that many of those who have read this far are well aware of this.

C → G → D → A → E → B → F# → D♭ → A♭ → E♭ → B♭ → F → C

Now for two facts putting these to facts to their logical conclusion will lead to a contradiction that will prove that you can’t tune a piano. First fact: The ratio of frequencies from a note to the note that is one octave higher is two. If the A below middle C is tuned to 440 Hertz (440 cycles per second), then the A above middle C is 880 Hertz.

If you count the octaves as you go through the circle of fifths, you will find that they span seven octaves. Put this together with the first fact, and you will see that the frequency of the last entry in the circle is exactly 128 times the frequency of the first C in the circle. 128. Remember that number.

Now for the second fact: The ratio of frequencies from a note to the note which is one fifth above that note (as in from C to the next G up), is 3/2. Thus, if we tune to A 440, the E just above middle C is 660 Hertz.

We’re coming up on the contradiction, so pay attention.

There are twelve steps in the circle of fifths1. If each of those steps is 3/2, then the ratio of frequencies from the C at the end of the circle to the C at the start should be (3/2)12, which is 531,441 divided by 4,096, which is about 129.746. The astute student of mathematics will note that 129.746 is not equal to 128.

The conclusion is that this set of three statements is not consistent: An octave is a ratio of two. A fifth is a ratio of three halves. The circle of fifths winds up back where it started.

A Hint from Pythagoras

As the story goes, Pythagoras happened to be walking by the blacksmith’s shop one day when he noticed the musical tones that were created by the different blacksmiths pounding on the iron with their hammers. As the story goes, Pythagoras was intrigued, and started experimenting to try to understand what caused the tones to be different. As the story goes, he found that the weight of the hammers was the important factor, and that the ratio of weights told what chord the hammers would make when they were pounded together.

He determined a very simple, yet powerful fact. Simple ratios make for consonant notes. The octave, with a ratio of two to one, is a simple ratio, and the two notes sound consonant. The fifth, with the simple ratio of three to two, is also consonant. The major third has a ratio of five to four, and this is also consonant.

This story comes to us by way of Iamblichus, who was born in the third century AD, about eight centuries after Pythagoras. The story was wrong2, but the conclusion was correct. The note that is produced when you hammer on a piece of iron depends on the piece of iron, and not the weight of the hammer. Pythagoras probably came to this conclusion by some other means, perhaps listening to the tone produced by a string held taught by differing amounts of weight, or perhaps by listening to the tone produced by a column of air of different lengths.

Defining a C major scale

Regardless of how the “simple ratios” rule was determined, it can be used to define all the notes in a C major scale. Much like early Rock and Roll, we just need three chords: the tonic, the dominant, and the subdominant, that is, C, G, and F.

Let us start by assuming that we have tuned our instrument so that middle C is equal to 240 Hz. This is a bit off, but it makes for easier calculations. With this definition of C, then G must be three halves of that (360 Hz), and E must be five fourths (300 Hz).

The G chord allows us to build from the definition of G (360 HZ) to ascertain that the fifth above that, D, is 540 Hz. This is in the next octave, so we can divide by two to bring it into this octave. Thus, the D above middle C is 270 Hz. From the G chord, we can also ascertain B, the third above the G. This makes B out to be 450 Hz.

The F below middle C is a fifth below C. That means that the frequency of F, when multiplied by three halves, gives you 240 Hz. Thus, F must be two thirds of the frequency of C, or 160 Hz. This is in the octave below middle C, so we multiply this by two to get 320 as the frequency of F above middle C.

The note A is the final note in the key of C major. This is five fourths the frequency of F, which gives us 400 Hz. Here is the full table for a C major scale, with C tuned to 240 Hz.

Middle C 240 Hz D 270 Hz E 300 Hz F 320 Hz G 360 Hz A 400 Hz B 450 Hz C 480 Hz

Trouble brewing

This was all pretty simple. Given that the circle of fifths ran into trouble, it should come as no surprise that there might be problems underlying our simple C major scale as well. The trouble can be exemplified by asking a seemingly simple question, “what is the ratio for a minor third?” The C major scale gives us three opportunities to figure this out. D to F is a minor third, as is E to G, and A to C.

The ratio from E to G is six fifths (1.200), as is the ratio from A to C. But the other example of a minor third is D to F, with a ratio of 32 / 27, or 1.185. We have two problems here. The first is that we don’t know for sure what a minor third is. The second is that sometimes a minor third has a ratio that is simple (6/5) and sometimes it is not (32/27). What went wrong?

The difficulty is that we have implicitly accepted two different notes as being the same note. First, there is the F that is a fifth down from C. Second, there is the F that is a minor third up from D. The piano forces these two different notes into the same key. The keyboard of the piano is a lie. The two different notes are only 1.25% different, but they are different notes.

All we need to do to fix this up is to add a second F to the keyboard, defining it to be a minor third above D. And of course, we need to officially define a minor third to be the simple ratio 6 / 5. And all is well with the world.

Well, at least until someone wants to play a chord that has a sharp or a flat. Or until someone wants to play anything by Cole Porter. Then Debussy comes along and asks you to play in sixteen sharps. Pretty soon, your piano has exactly 4.76 gazillion keys. Every chord is perfectly in tune. Now, If your name is Ruben, then you could probably learn to play such a monster, or at least a toy version of it. All the rest of us mere mortals have learned to live with a piano tuning that is a compromise, where each key is tuned to be a compromise between all the frequencies that it is supposed to play.

Two halves don’t make a whole

Every choir director I have ever known has played a trick on us during warm ups. Maybe it’s been on purpose, or maybe they don’t realize that it’s just a cruel joke. A joke like asking a third grader to find an integer fraction that, when squared, gives you 2. Choir directors ask you to sing a chord, and then move down by a half step and then another. Then they ask you to go up a full step. The cruel task master will then chastise the choir for going flat. Cruel trick. There is a perfectly reasonable explanation for going flat that has absolutely nothing to do with the indolence or skill of the choir.

First, let’s consider what “going down a half step” means. If I am the lead voice, and I am singing a C, then a half step down is to B. That is, a half step is from do to ti. From the chart above, we can see that going down a half step means multiplying the frequency by 15 / 16. And the next half step, you just repeat that3. So, after two half steps, the frequency is now 225 / 256 times the original frequency.

Then the choir director commands to go up by a whole step. If you are singing the root of the chord, then you are going to be going from do to re. Looking at the chart above, do to re is the interval from C to D, which means multiplying the frequency by 9 / 8. This means that the final landing point is (225 / 256) X (9 / 8) = 2025 / 2048. This is about 0.98877, just over 1% short of returning back home.

The choir director, in tyrannical ignorance, will demand that two half steps be equal to a whole step, and accuse the choir of (God forbid) going flat. But the choir is just doing what it was told.

I hope this section has caused some choir directors to have a bit more humility. Maybe some of them will even find contrition in their hearts, and will be willing to apologize for their misguided despotism.

If any of my previous choir directors feel such a need, rest assured that apologies are not necessary. I have been blessed through the years with many, many choir directors who were inspirational magicians, and I want to thank them for the glorious moments of ethereal chords.

That is why I say that you can’t tune a piano.

1)      Any alcoholic will note a certain irony in that. 2)      This of course was not the only thing that history has gotten wrong about Pythagoras. See a previous blog for another example. 3)      Crucial point here: if the choir is doing their job right, they will mentally change keys when they drop a half step. The old ti will become the new do from which to move up or down.

Flat paint is not flat

I needed to paint the inside of an ambient light exclusion device for an optical mensuration1 experiment. (To the layman, I was trying to paint the inside of a box black.) I had decided on flat black mainly because it would look cool. Flat black really gives A/B plywood an air of scientificalness.

The trouble was, this was autumn and the temperature in my garage2 was 45°. The warning label on the spray paint can advised me against painting at temperatures below 50°, but I threw caution to the wind. In single-minded devotion to the greater cause of Science, I forged ahead, hoping that I did not get caught by the Spray Paint Police.

I can look back on this today and laugh at my naiveté. I was quite frankly surprised by the results. The flat black paint came out glossy.

How can flat paint become glossy?

Naturally, my first reaction was to get out the microscope to investigate the microstructure of the surface. This is what I saw.

The image at the left is the surface that I had just painted. The one on the right was a surface that looked flat black because I had painted it at the regulation temperature. Clearly, the glossy surface on the left is smoother than the matte surface on the right. Ironically, the surface of “flat” black paint is not flat!

My “flat black” spray paint became glossy because the vehicle (the stuff that carries the pigment) evaporated much slower at 45°. This gave the surface of the paint time to smooth out.

Changing color with colorless water

Seeing these images, I immediately had a flashback to one day when I was five years old. I had found myself with a small plastic bucket filled with water and a paint brush. The young scientist in me bloomed when I painted the wall of our house with water, and found that it turned the house from a chalky light green to a richer green.

Some adult wandered by and asked me what I was doing. When I said that I was painting the house, they frantically went off to find my mom. My little ruse had fooled even them. I probably got spanked before the truth was realized. If only I had known Fresnel’s law at the time, I would have been able to explain the phenomenon to my mother before her hand came down on my precocious little butt.

Fresnel’s Law

Fresnel’s Law3 describes what happens when light goes from one medium to another, in this case from air to the surface of the paint or water. When light goes from air to a surface that is optically “harder” (having a higher index of refraction), some light enters the surface and other light reflects directly from the surface, in billiard ball fashion – with angle of incidence equaling angle of reflectance.

Fresnel’s law predicts what percentage of the light is reflected and what percentage of light enters the second medium. The law takes into account the angle of incidence, the indices of refraction of the two media, and the polarization of the incoming light. All that is fabulously interesting, but all we need to know is that somewhere between two and five percent of the photons act like little billiard balls. We call this specular reflection.

I brought my good friend Smeldon in to demonstrate. He reads his magazine4 with the light behind him, and the magazine tilted slightly away from him. Without even thinking about it, he will orient the magazine so as to avoid the specular reflection. Smeldon could orient the magazine so he sees only the gloss of the magazine, as in the drawing on the right. If he were to do this, the print on the paper would be largely washed out by the specular light.

Smeldon’s magazine was printed on a coated stock. After the paper is formed, it goes through a second process where various additives, like kaolin (a white clay) and calcium carbonate, are affixed to the surface of the paper and then the paper is polished to a glossy finish. The coating makes the surface glossy. Most magazines are printed on coated stock. Newspapers and Sunday paper inserts are printed on uncoated stock.

If Smeldon were to try this same experiment with a newspaper, he would not see the same results. In fact, he would be hard pressed to find a way to orient the newspaper in such a way as to see gloss. (The interested reader will try this before going on. Go ahead. I’ll wait.)

Smeldon is not an accomplished color scientist and applied mathematician like me (and he is not nearly as good-looking as me) but he does have a bit of a scientific bent. His conclusion is that newsprint is an exception to Fresnel’s law. It would appear that newsprint has very little surface reflection.

Holding out on the dark side

Smeldon is inquisitive and intuitive, but in this case, he is wrong. A newspaper has plenty of specular reflection; it’s just hiding in plain sight. I’ll get back to that in a bit.

Anyone who has looked at a magazine and newspaper side by side will notice that the magazine has much richer color. A solid black on newsprint looks almost gray when sitting next to a solid black of a glossy glamour magazine.

Why is this? I have heard many otherwise intelligent people attribute this to “holdout”. As the explanation goes, glossy stock will “hold out” because of its nice smooth and hard surface. That is, it will impede the progress of ink that tries to seep in. Newsprint has no protective coating, so ink will seep into the paper, and some of the cute little pigment particles will hide behind paper fibers.

I am sure this is explanation is true, but hey, flies walk on the ceiling. I am not convinced that this is the predominant effect. I submit the following image as proof. The image is of newsprint. I borrowed my wife’s clear nail polish and painted over the word “FREE”. The black ink with the polish is noticeably darker. I didn’t add pigment. This was clear nail polish. This is not explained by the “ink hiding behind the paper fibers” theory – the cute little pigment particles are still in hiding. Why did it get darker?

Newsprint with splotch of nail polish

The next picture sheds some light on the quandary by shedding light from a different direction. To take the next picture, I left the camera and the newsprint where they were and moved the light over to the specular angle. The splotch of nail polish has suddenly gotten much brighter because the specular reflection has suddenly reappeared.

Same sample, with light at specular angle

Going back to the original pair of microscope images, we saw that the “flat” black ink on the right had a rough texture. Photons hit this rough surface and reflect specularly in all sorts of directions. Contrary to Smeldon’s conclusion, newsprint still follows the magic of Fresnel’s law, it’s just that the specular reflection goes in all directions.

In the first “free estimate” image, it’s not so much that the nail polish made the black ink darker. It’s more that the specular reflection in the other parts of the image made the black ink without the polish look lighter. The nail polish focused the specular reflection so that it all bounced away from the camera.

My conclusion is that smoothness will make a surface appear darker in color. This is not because there is less total light reflected. It is because the specular reflection from a smooth surface generally bounces off in a direction where we don’t normally notice it.

Notes 1)      No, I didn’t spell that word wrong. “Mensuration” means the act of measuring. All good mensurologists know this word and use it all the time. It’s like a secret handshake. 2)      Due to certain previous incidents, I wasn’t allowed to paint in the basement like other respectable mensurologists do. I guess we can see why I don’t have a Nobel Prize. Yet. 3)      In my humble opinion, Fresnel’s Law is way cooler than Snell’s Law. By the way, Snell’s Law was discovered by Ptolemy. I don’t know who discovered Fresnel’s law, but we know from a previous blogpost that it probably wasn’t Fresnel. As for the Fresnel lens? That idea came from Georges Louis Leclerc, and not Augustin-Jean Fresnel. 4)      Magazines were a technology common in the 20th century that was similar to the iPad. While articles could be read on these devices, you could only check the status of a few celebrities, and one magazine could barely hold a few dozen articles. To their credit, they were produced from a once common renewable resource called “trees”. Ask your grampa to tell you about trees.