How many piano tuners are there in New York City? You may have been asked this interview question and been left scratching your head. Of course you aren’t expected to know the answer, the interviewer just wants to see how you think. These questions are called “Fermi Problems” — after the famous engineer who used them to estimate the strength of atomic blasts, the circumference of the Earth and the likelihood that aliens exist, amongst other things.
It works like this: 1) break the problem down into smaller parts, 2) use common sense and make assumptions 3) calculate the answer by aggregating your guesses. For example we know roughly 8 million people live in NYC. Assume 5 people per family. Maybe 1 in 10 families own a piano. They need to be tuned once per year. If one piano tuner can do 4 per day, working 200 days per year, that’s 800 tunings each. Run the numbers and you get 200 piano tuners. Checking Yelp there are 24 companies listed: assuming each employs multiple people and all tuners aren’t listed, we arrived at a reasonable guess.
When you break a problem down into smaller parts, it becomes easier to estimate: simple enough to write on the back of a napkin. Even though each estimate might be off, aggregating lots of smaller errors cancels them out. You usually arrive at approximately the right answer, within an order of magnitude (i.e. the right number of zeros) at least. That’s the power of decompositional reasoning: by breaking things into constituent parts and analyzing each part independently, you can draw powerful conclusions about the whole.
The truth is that you rarely need to know anything with exact precision. If you’re operating an industrial lab you might need several decimal places on your thermometer, but if you’re checking the temperature of your house, you just need to know within a couple of degrees. If you’re asking your friend if you should wear a coat, all you need in response is to hear “it’s hot today”: no need to ask for a precise measurement. This is important because precision comes at a cost, and it increases exponentially. Hot or not is easy to answer, getting the temperate right within 10 degrees is usually possible with a guess, whereas you need a thermometer to get it down to the degree. Resistance Temperature Detectors (RTDs) as used in industrial settings, can get you within 0.1 degrees of accuracy but cost thousands of dollars to install.
As Rory Sutherland of Ogilvy says, “In business, you don’t need to be “right.” You just need to be right enough”. We’re not looking for the universal theory of everything when doing a back of the napkin calculation, we’re just fleshing out the situation so we can make the best of it. The level of proof you require for a decision depended on the consequences of the decision. Often you can rule out entire courses of action based on simple equations.
For example if the average CPM (cost per thousand ad impressions) for Facebook ads is $10, and the average CTR (clickthrough rate) is 1%, and you normally get a CVR (conversion rate) of 5%, you can expect to pay $20 per purchase. If your product costs $40 and you make a 50% profit margin, you have a shot at breaking even. However if your product costs $2 you’re not even in the right ballpark. Unless you think you can get 10x better results than average, rule out Facebook ads and move on to another channel.
If the numbers work out and you turn on your campaign and spend $20, but you get half the clicks you were expecting, you can turn it off and reasses the economics. Because at each stage of a fermi calculation the numbers get smaller, it gets increasingly more difficult to reach statistical significance for each step lower in the funnel. You can tell if your CTR assumption was correct with $20, but it might take $400 to prove your CVR is what you expected. In this way you can cascade significance down, improving your confidence with each experiment invested. If the CTR was in range, keep spending until we prove the CVR. If that works out, let’s see how much we can spend before our assumptions break again, and the cycle continues.
| Name | Link | Type |
|---|---|---|
7 Types of Reasoning: Definitions and Examples | Reference | |
Enrico Fermi | Reference | |
Fermi Problem | Reference | |
How Many Piano Tuners Are There In New York City? | Blog | |
It Isn’t a Replication Crisis. It’s a Replication Opportunity | Article | |
Measuring Temperature Accurately: What Are the Costs? | Article | |
Yelp Piano Tuners New York | Reference |