Appreciate the limits of data
- Seduced by their seeming precision and objectivity, we can feel betrayed when the numbers fail to capture the unruliness of reality.
- Goodhart's law
- Once a useful number becomes a measure of success, it ceases to be a useful number.
- Whenever you try to force the real world to do something that can be counted, unintended consequences abound.
- It can haunt AI as well. Numbers can be at their most dangerous when they are used to control things rather than to understand them.
- We can turn it around: “We tend to get what we measure, so we should measure what we want."
- Those who do the counting have power. To simplify the world enough that it can be captured with numbers means throwing away a lot of detail. As Stone observes, in order to count, one must first decide what should be counted. Bias-in, bias-out problem can sharply limit the value of these gleaming, data-driven recommendations.
- We’ve known since the nineteen-fifties that even simple algorithms outperform human predictions. But we should be cautioned against the temptation to believe that numbers hold all the answers. Data may be a pretty decent proxy for something that really matters, but there’s a critical gap between even the best proxies and the real thing – between what we’re able to measure and what we actually care about. Nature is built on unavoidable randomness, limiting what a data-driven view of reality can offer. Our proxies can still serve as a metric of something, even if we find it hard to define what that something is.
- When polls have faltered in predicting the outcome of elections, we hear calls for more and better data. But, if more data isn’t always the answer, maybe we need instead to reassess our relationship with predictions—to accept that there are inevitable limits on what numbers can offer, and to stop expecting mathematical models on their own to carry us through times of uncertainty.
- It’s possible for two things to be true: for numbers to come up short before the nuances of reality, while also being the most powerful instrument we have when it comes to understanding that reality.
- Numbers don’t lie, except when they do. Harford is right to say that statistics can be used to illuminate the world with clarity and precision. They can help remedy our human fallibilities. What’s easy to forget is that statistics can amplify these fallibilities, too. As Stone reminds us, “To count well, we need humility to know what can’t or shouldn’t be counte
Metadata