Did you know that statistically, the countries that eat the most chocolate also produce the most Nobel Laureates? Or that when iPhone sales are higher, more Americans die from falling down the stairs? Unfortunately, it’s unlikely that chocolate is the driving force behind Laureate achievements, and likewise, iPhones are unlikely to be banned on staircases anytime soon. These rather strange examples, demonstrate that statistics can be extremely misleading!
We’re surrounded by data and statistics, and it’s no different at work. Whether we’re making hiring decisions, analysing employee opinion surveys, or evaluating the success of training programmes, dealing with data is often a key step in making decisions. How well we do this has a big impact on the effectiveness of our decisions, policies and investments. So whilst we’re not suggesting that we all become statisticians, we do think it’s important to understand what data is (and isn’t!) telling us, so we can improve the decisions we make and challenge misleading claims.
In this article we explore one of most common statistical pitfalls – correlation versus causation.
A correlation shows the extent to which two variables are related. For example, you will probably find a positive correlation between seniority and salary in most organisations, because senior employees tend to get paid more. So far, so simple. But correlations can be misleading in two ways:
- They can occur by chance if there’s enough data
- People wrongly make claims about causality
If you analyse lots of data, the chances are that you will find a correlational relationship somewhere, like the example of eating chocolate and winning Nobel Laureates, it doesn’t necessarily mean the two are actually related. It’s a bit like a statistical fishing trip and the danger, which is of particular concern given the rise of ‘big data’ and HR analytics, is that acting on the basis of chance correlations can have considerable consequences for organisations.
Furthermore, whilst analysing data for correlations can indicate that two concepts might be linked, this isn’t enough to infer causality. Expanding on our salary example above, it doesn’t hold true that being paid more causes your senior position, even though the two factors are statistically related. It’s the other way round. In practice, correlations are often misused, deliberately or mistakenly, to suggest causality where there isn’t any evidence for it. If someone want to sell you a new psychometric assessment that claims to select better people, or an engagement solution that increases productivity, what sort of evidence do they have to support their claim? Have they done a before and after (longitudinal) study? If not, we cannot be sure if their intervention caused the change, whether it was coincidence or something else entirely caused it to happen.
Advice for dealing with statistics
When you’re given statistical information, ask yourself the following questions:
Do I understand what the data is telling me? Don’t be afraid to ask for clarification if something is unclear, especially if it feels like you’re being blinded by statistics (it might be on purpose!).
Who provided the statistical information? Who is making the claim and what’s in it for them? Do they have a vested interest, or are they trying to sell me something? Are they citing their own (potentially biased) research, or is it from a reputable source?
How do they know? Have they got good evidence for what they’re saying? What sort of evidence is it and what is it based on? Is it likely to be applicable to your organisation?
What’s missing? Is there a piece of data that’s conspicuously absent? Is there something more you need to know before you accept what they’re telling you? Ever read a case study telling you when something didn’t work brilliantly?!
We’ve produced a guide which aims to offer a user-friendly introduction into five of the most common statistical pitfalls and misconceptions called ‘Lies, damned lies and statistics!’ and some practical advice on avoiding these traps within your organisation.
About Future Work Centre
We’re passionate about the communication of science and the value of evidence-based practice in organisations. We help organisations make sense of their data through the provision of flexible training modules, practical user-friendly resources and consultancy. Get in touch: firstname.lastname@example.org