Correlation Does NOT Equal Causation

Your variables may be related, but does one really cause the other?

Most readers have probably heard the phrase, “correlation does not equal causation.”  Recently, however, I heard someone confess that they’ve always pretended to know the significance of this phrase, but they truly didn’t know what it meant.  So, I thought that it’d be a good idea to make a post on the meaning behind “correlation ≠ causation.”

Imagine that you are the president of your own company.  You notice one day that your highly-payed employees perform much better than your lower-payed employees.  To test whether this is true, you create a database that includes employee salaries and their performance ratings.  What do you find?  There is a strong correlation between employee pay and their performance ratings.  Success!  Based on this information, you decide to improve your employees’ performance by increasing their pay.  You’re certain that this will improve their performance. . .right?

Not so fast.  While there is a correlation between pay and performance, there may not be a causal relationship between the two – or, at least, such that pay directly influences performance.  It is fully possible that increasing pay has little effect on performance.  But why is there a correlation?  Well, it is also possible that employees get raises due to their prior performance, as the organization has to provide benefits in order to keep good employees.  Because of this, an employee’s high performance may not be due to their salary, but rather their salary is due to their prior high performance.   This results in current performance and pay having a strong correlational relationship, but not a causal relationship such that pay predicts performance.  In other words, current performance and pay may be correlated because they have a common antecedent (past performance).

This is the idea behind the phrase, “correlation does not equal causation.”  Variables do not necessarily have a causal relationship just because they are correlated.  Instead, many other types of underlying relationships could exist, such as both having a common antecedent.

Still don’t quite get it?  Let’s use a different example.  Prior research has shown that ice-cream sales and murder rates are strongly correlated, but does that mean that ice cream causes people to murder each other?  Hopefully not.  Instead, it is that warm weather (i.e. the summer) causes people to (a) buy ice cream (b) and be more aggressive.  This results in both ice-cream sales and murder rates.  Once again, these two variables are correlated because they have a common antecedent – not because there is a causal relationship between the two.

Correlation does not imply causation

Hopefully you now understand why correlation does not equal causation.  If you don’t, please check out one of my favorite websites:  Spurious Correlations.  This website is a collection of very significant correlations that almost assuredly do not have a causal relationship – thereby providing repeated examples of why correlation does not equal causation.  If you do understand, beware of this fallacy in the future!  Organizations can make disastrous decisions based on falsely assuming causality.  Make sure that you are not one of these organizations!

Until next time, watch out for Statistical Bullshit!  And email me at MHoward@SouthAlabama.edu with any questions, comments, or stories.  I’d love to include your content on the website!

Bullshit Measurement

Are you measuring what you think you’re measuring? Could you be measuring something else entirely?

Accurate measurement of variables is essential for business success.  Sometimes, it’s fairly easy to record these variables – sales, revenue, profit.  Other times, it can be very very difficult.  For example, let’s say that you want to hire employees that are smart and conscientious.  How can we measure intelligence and conscientiousness?

Well, a good starting point is to develop a test or survey.  Many intelligence tests exist with varying levels of sophistication and accuracy, and you could pay to give these tests to applicants.  Many self-report surveys also exist that can measure conscientiousness, and you could pay to give these tests to applicants, too.  But what if you don’t want to use one of these existing measures?  What’s the worst that could happen?

In this post, we won’t talk about the worst that could happen, but we’ll discuss a pretty bad outcome: when your measure inadvertently gauges the wrong construct, which could result in a lawsuit.

I should also note that this example comes from an actual consulting experience that I encountered.  The names have been changed, but remember that these things actually happen in industry!


I was once hired along with a full team to review the new selection system of a trendy company.  Let’s call them X-Corp.  X-Corp wanted their selection to measure a construct that they invented: “the ideal X-Corp employee.”  They made a list of the ideal X-Corp employee characteristics.  It included the common constructs like intelligence and conscientiousness, but it also included some unorthodox constructs.  These included hip, stylish, savvy, sleek and so fourth.  X-Corp argued that the ideal employee needed to appeal to any potential customers, and therefore needed to have these characteristics; however, my team was already doubtful about the business relevance of theses constructs.

Even more concerning, X-Corp felt that their survey had to attract people to work for X-Corp.  For this reason, it couldn’t be a traditional survey.  It had to be different and exciting.  Once again, we were doubtful about how exciting a selection survey could be.

When we saw the survey to measure “the ideal X-Corp employee,” we began to worry even more.  The first question looked something like this:

Bullshit Measurement 1

What?

The text of the item read, “Using the scale, please indicate whether you are more like a sports car or a hybrid/electric car.”

…What?

Immediately, we asked X-Corp what this item was meant to measure.  Sure enough, they just said “the ideal X-Corp employee.”  We asked which subdimension, specifically, was the item meant to measure.  As they couldn’t respond, we realized that they didn’t really have an idea.  It seemed that they just put things in their survey that they thought would be a good idea without really thinking about the ramifications.

Do you think this item would help identify good employees?  Well, we first have to ask what is the “correct” answer.  According to X-Corp, the correct answer was being more like a hybrid/electric car.  So, anyone would indicated that they were more like a sports car got the item wrong.  Do you think this is fair?  More importantly, do you think those that feel more like a “hybrid/electric car” are necessarily better than those that feel more like a “sports car?”  I would guess that the answer is probably not.  There are probably many sports car people that are more intelligent, conscientious, hip, savvy, and so on when compared to hybrid/electric car people.  Thus, this item probably fails to measure “the ideal X-Corp employee.”

That item was bad, but it wasn’t the worst.  The worst was probably the following item:

Bullshit Measurement 2

Once again, what?

The text of the item read, “Using the scale, please indicate whether you are more or less like Kanye West.”

Once again…what?

X-Corp claimed that Kanye West was too narcissistic, and anyone who felt that they were like Kanye were not welcome at X-Corp.  Do you think that Kanye people are inherently worse than non-Kanye people?  Once again, I am guessing that the answer is probably not. Kanye people are probably just as good as non-Kanye people, and perhaps even better in some regards (i.e. creative, hip, etc.).  But can you think of anything else that this item might inadvertently measure?  Let’s look at the graph below, which is similar to the actual results.

Bullshit Measurement Graph

As some of you may have guessed, African Americans were much more likely to see themselves similar to Kanye than Caucasians.  This makes sense, as Kanye himself is African American.  Thus, this item partially measures the applicant’s ethnicity.

Remember when I said that those responding that they were more like Kanye were rated as worse applicants?  If this survey went live, that would mean that African Americans would automatically be penalized, thereby resulting in adverse impact.  This would almost assuredly result in a lawsuit, in which X-Corp could not justifiably defend – or, at least, have a very hard time defending that the Kanye question actually represented job performance.  This would have cost the company millions of dollars!

In the end, my team strongly recommended that the company should not use their selection survey, and should instead use a traditional survey.  The company wasn’t happy, and we were never asked to work with the company again.  But, they did guarantee that they would not use their selection system.  While it wasn’t the most satisfying result, I was happy that we were able to stop another case of Statistical Bullshit!

If you have any questions or comments about this story, feel free to contact me at MHoward@SouthAlabama.edu .  Also, feel free to contact me if you have any Statistical Bullshit stories of your own.  I’d love to include them on StatisticalBullshit.com!

Bullshit Charts

Is Statistical Bullshit possible when no numbers are involved?

Possibly the most widespread form of Statistical Bullshit is Bullshit Charts.  Charts are meant to provide clear and easy-to-read information, but Bullshit Charts are designed to mislead the reader – whether intentionally or unintentionally.  Often, these charts will alter common cues that the reader expects, hoping that the reader will not notice these subtle changes.  Through doing so, the chart is not “lying” per se, but it is certainly Statistical Bullshit!

Bullshit Charts are common in situations with little time to process all relevant information, such as during a commercial or business meeting.  And I’m sure you’ve experienced this before.  Maybe a commercial presented a chart for a split second, showing that their product is superior to others.  It may have looked reasonable, but if you could only pause the TV, you could have seen that the x- or y-axis was mislabeled.  In other words, it was indeed a misleading Bullshit Chart.

Below are some of my favorite examples of Bullshit Charts.  The Statistical Bullshit should be apparent in each of these charts, but please email me at MHoward@SouthAlabama.edu if you have any questions or comments about these charts.  Until next time, watch out for Statistical Bullshit!


1.  Need to make your argument seem more convincing?  Just give yourself a bigger slice of the pie no matter what the data shows…

Bullshit Charts 1

2.  Again, just change the distribution of the pie to help your case!  Or make up the data, as these labels and percentages seem to not make any sense at all…

Bullshit Charts 2

3.  Or, just ignore the size of the bars.

Bullshit Charts 3

4.  Does the data disprove your claim?  Just flip the chart upside down to make it seem like you’re correct!

Bullshit Charts 4

5.  Although those in the Philippines may only be ~.2 meters shorter than those in The Netherlands, you can always draw them as about 1/3rd the size to prove a point…

Bullshit Charts 5

6.  Again, you could just ignore the size of the bars altogether.

Bullshit Charts 6

7.  I’ve seen this trend catching on more recently.  Three-dimensional charts are often difficult to read.  If you want to prove a point, it is rarely a good idea to use 3-D charts.

Bullshit Charts 7

8.  Then again, some two-dimensional charts aren’t much better…

Bullshit Charts 8

9.  So, sometimes it’s just easiest to go back to giving yourself a bigger slice of the pie.

Bullshit Charts 9

10.  If all else fails, just give your chart nonsense labels and just hope for the best!

Bullshit Charts 10

Sources for these and other Bullshit Charts:

https://www.reddit.com/r/dataisugly/

https://www.reddit.com/r/shittydataisbeautiful/

https://www.reddit.com/r/badstats/