… but plan for tidying it up!
- Data isn’t objective, and analysis isn’t structured. It’s just as creative—and just as messy—as the papers I ran away from in college.
- But we can’t define metrics informally; our instructions have to be precise. And for metrics like win rates, there are a lot of devils in the details.
- Each version is equally accurate because they are tautological: They measure precisely what they say they measure, no more and no less. Our job as analysts isn’t to do the math right so that we can figure out which answer is in the back of the book; it’s to determine which version, out of a subjective set of options, helps us best run a business.
- This choice isn’t a technical one, but a creative one, built on top of “messy” questions, like:
- “How easy is the metric to understand?”
- “Do people already have pre-existing ideas of what this number might mean?”
- “How much do we think we can learn from it?
- Often, the most useful things we find are the things we weren't looking for.
- Companies’ first analytics tools, regardless of which ones they choose, will always be disorganized, not because the tool is inherently so, but because the creative process is.
- Rather than trying to stifle or control this phase, we’re better off having a plan for how to tidy it up later, by working in a sandbox or reserving space for the polished final drafts that we’ll eventually produce.