Data is powerful.
It is objective, it never forgets, and it is abundant. It can inform decisions. It can align teams. And it can serve as a catalyst for change.
But using data poses risks. A desire to become "data driven" can lead you to abandon soft information, give you too much confidence, and lead you astray.
Nassim Nicholas Taleb writes:
All [data] models are wrong. Some models are useful. Most models are dangerous.
Like model trains, data models are a representation of the world. They are not the world itself. And they can be flawed.
Data is attactive because it black and white. It gives us confidence that we understand the world - driving us to make bold decisions.
But we need to be careful.
A clear understanding of the difference between signal and noise is the first, and most important defense.
Signal and Noise
For any decision, problem, or objective there is relevant data and irrelevant data. Some data matters; most doesn't. We call this "signal" and "noise".
Signal = relevant data.
Noise = irrelevant data.
Simple enough.
Some data is easy to ignore: the stage of the moon’s relation to monthly revenue, for instance. But plenty of data looks like signal, when its actually noise. Separating the two is vital.
An Example
Let’s say your company hires mine to study your CRM (customer relationship management) data. You want to identify what makes salespeople effective and incentivize those activities. Your CRM would probably track things like:
No. of deals in the pipeline
When a prospect signs a contract
Each interaction between the salesperson and the prospect
When we comb through the data, we might find a strong correlation between the number of interactions with a prospect (calls, meetings, emails) and their likelihood of ultimately signing a contract.
More calls throughout the process = higher likelihood of a signature.
It makes sense. The more you talk to a prospect —> the deeper the relationship becomes —>the more questions you answer —> the more likely you are to close the deal.
So you preach the gospel of more calls, emails, and meetings to your salespeople. You reduce paperwork and streamline processes to give them more time to engage prospects. You buy them all AirPods. You sit back and watch as calendars fill up, and your Verizon bill rises.
What happens next?
The close rate doesn’t budge. It stays exactly the same. Nothing changes.
Why? Because we confused the signal with noise.
Here’s what happened: your salespeople could sense who was really interested. They poured time into the most promising prospects. They called those clients more. Those prospects had a higher close rate, but they also had a higher baseline interest in your product or service.
Golf and Deaths
The image below illustrates the point perfectly.
Golf doesn’t cause deaths (unless my 4 year old is teeing off) but you wouldn't know that from this data. Clearly, the signal we're missing here is human existence - causing both death and golf.
Work to separate Signal from Noise
Confusing signal for noise, at best wastes time. At worst, it makes things worse. It’s possible that more calls would drive your number of closes down (perhaps way down) by needlessly annoying interested customers.
But don't despair. The signal exists.
You should approach data with a critical eye, apply logic to it, and test theories cautiously. And when you find the signal, you should pour on the heat. Build score cards. Incentivize performance. Build a culture around that metric.
This is actually good news. Results come from extreme focus on a few activities (the right activities). Recognizing that most of your data is noise gives you the opportunity to focus on the signal with extreme intensity.
Ignore the noise. Focus on the signal. Know the difference.