Why People Ignore Good Data – and Trust Their Gut
We have always learnt that mathematics is a concrete field wherein we can expect right or wrong answers. Statistics is composed of deep mathematical concepts and is a way of taking significant decisions across every sphere and sector. The question then arises - is statistics an explicitly reliable field? Can we trust statistics? Answer is yes, but with a pinch of salt. Although it composed of mathematics, it has one component of mathematics which is deeply subjective in nature - probability or you can say probability distributions. This makes it a very flexible and not so fixated area. That is the reason governments, companies, agencies use statistics to drive opinions, narratives and some of those opinions are used to make very significant decisions. In reality, statistics lead to probabilities and numbers which always come with terms and conditions. There is a big asterisk attached to it.
Why is Statistics subjective?
Gelman & Hennig (2017) state that statisticians often use subjective traits to make decisions on statistical models, assumptions and sometime use legacy methodologies. Very often, the quality of statistical analysis is measured by its reproducibility, meaning can the algorithm or study be applied directly to other problem statements. If the applied algorithm is good enough, it should ideally be applicable to similar cases - however, Cambridge University Science Magazine, in one of their articles mention that statisticians often struggle with reproducing the results leading to what is known as 'Reproducibility crisis'.
One of the branch of statistics - 'Bayesian statistics' differ from the usual frequentist approach in a way it is incorporates subjectivity into the plane.
Question then arises as to why is Statistics subjective. To understand this, we need to touch upon basic fundamentals of statistics - which includes assumptions, beliefs, probability and inference. In my opinion, here are 4 reasons why statistics is subjective in nature.
- Assumptions
- Every statistical method relies on assumptions about the data. Examples include assuming a normal distribution, independent observations, or constant variance. If these assumptions aren't valid for the real-world data, results can be misleading or invalid. The choice and acknowledgment of such assumptions are at the discretion of the statistician, making the process subjective.
- Probability
- Statistical inference is grounded in probability, which is inherently about modeling uncertainty. The assignment of probabilities depends on how one frames the problem, interprets randomness, and sometimes even personal belief (especially in Bayesian statistics). This means conclusions often reflect the analyst’s chosen probabilistic model—not just the data.
- Beliefs (Prior Information)
- Especially in Bayesian statistics, prior beliefs (priors) are explicitly included. Analysts can select priors based on prior studies, expert opinion, or convenience. These priors heavily influence the results, which means that two statisticians might draw different conclusions from the same data, based on their subjective choices.
- Inference
- Drawing conclusions from data (statistical inference) is not a purely mechanical process. Choices regarding significance levels, test selection, or interpretation of confidence intervals are subjective. For instance, deciding whether a p-value of 0.049 versus 0.051 is “significant” illustrates the gray area in statistical reasoning.
Examples of subjectivity
FDA has demanded the food and beverages companies to provide statistics of the added sugars across their products. Companies have very smartly positioned their brand and claims. In one aspect, they might be true, but in reality, the picture is something else.
Truth is sometimes subjective and people hear what they want to hear
See examples below:
According to the national consumers league, skippy peanut butter made claims of reduced sugars in their product. Upon delving deeper, it was found that it just had 1 gram less sugar, but 20 grams more fat and calories per 2 table spoon of their product.
Similarly, there are loads of companies claiming "0 added sugars". Oikos Triple Zero blended Greek Yogurt is one of them - but it was found that the alternative they used (stevia leaf extract) had 120 more calories per serving.
Similarly, Kellogg's, a famous brand for breakfast cereals, claimed that their products were "hearty and healthy" and provided 20% more alertness to children. Later, it was found that the "sample size" of the study was too low to justify this number and generalize it.
One of the toothpaste companies claimed that 80% of doctors recommended the brand. It did not mention that doctors recommended multiple brands, it was just a knit picked statistic to receive an applaud. Its just like saying - I got a 3rd rank in the class. What if I only have 3 students in the class? LOL
The "95% Confidence" Misinterpretation – Avandia
In 2013, the diabetes drug Avandia was found to lower blood sugar, but meta-analyses suggested it might raise heart attack risk. Some pooled studies produced p-values just below 0.05 (statistically significant), while others landed just above it (not significant), depending on trial design, sample size, and inclusion criteria.
The issue? A 95% confidence level doesn’t mean a 95% chance the result is correct — it’s a probability statement about repeated experiments. In Avandia’s case, the maths was valid, but small shifts in assumptions flipped the conclusion from “safe” to “risky,” letting different stakeholders tell opposite yet technically correct stories.
Why do business love their gut?
Numbers and statistics are great, if used appropriately and accurately. However, in my experience as well, a lot of stakeholders I work with trust their gut in addition to the numbers. Gut feeling comes through experience. If you have a gut for numbers, you will be able to:
- Ask right questions - for example if the numbers don't make sense, you can always ask questions
- Trust, but check - a lot of times, if certain KPIs such as market share increase significantly, you will ask the team to check the numbers and find the reason
- Be vary - when statistical representations seem overwhelming, checking the underlying assumptions, and denominators to become accustomed to the reality check
Conclusion
Statistics is subjective and can be easily misinterpreted. A data analyst should question as much as possible and should make an effort to interpret numbers, graphs and statistics. That will enable a good gut feeling for the insights, and once you get that, you start to question, reason and make sense of everything. You look at statistical claims with a pinch of salt. And that is good.
References
NCL staff
https://sites.stat.columbia.edu/gelman/research/published/gelman_hennig_full_discussion.pdf
https://sites.stat.columbia.edu/gelman/research/published/gelman_hennig_full_discussion.pdf
Jake Metzger
Examples
How to use statistics with a pinch of salt?