Do data analytics right

Summary

1. Understand the purpose of data analytics and make sure it is being met
2. Always focus on data quality, without clean data any investment into reporting/analytics is wasted money
3. Hire people who understand the data and avoid thinking biases. You don’t need people who tell you everything is great, look we have more pageviews and higher conversion rate.
4. Make sure you have clear expectations and you know how you will use the outcomes of the work data people are doing
5. If the data analyst doesn’t understand the relationships between multiple areas and departments of the company — make him/her understand it. The word analytics by itself points in the direction to be able to find relationships between stuff, that’s what the person needs to see and point out!
6. The data analyst should work in close cooperation with qualitative researchers like UX, design, market research and be aware of how customers perceive the product. Going through specific customer cases with the customer support might give a new context to the data he/she is analysing.
7. Only after all the above has been done, you should let the analyst show the insights to product managers, business people and other decision makers. Showing incorrect data to decision makers does more harm than good. When the insights brought by the analyst can be trusted, make him/her part of the decision making. Seeing how the decisions are done and how the people think makes sure the analyst knows what kind of insights are valuable for future projects.

Notes

• The main purpose of data analytics is to provide insights to the decision makers so they can make the vision of the company happen. And it doesn’t matter how catchy your company’s motto is, the main goal is to make profit.
• The fact that you have automated the entire [reporting] process instead of employing people to do it regularly is great, but does this give the insights needed?
• You can lie with any metric and you can easily get fooled by it. The world is not simple. People are not simple. Decisions are not based on a single number.
• Crap in the data means crappy insights which might mean harmful decisions. Once your data is clear, you still haven’t won the fight for meaningful insights. Once your data is clear, you still haven’t won the fight for meaningful insights.
• There are countless biases in our brains which might lead to incorrect interpretation of the data and thus again to shitty insights. To name a few:
  • Simpson’s paradox: a trend appears in several different groups of data but disappears or reverses when these groups are combined; more detail also here
  • Survivorship bias: concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility
  • Availability bias: a mental shortcut that relies on immediate examples that come to a given person's mind when evaluating a specific topic, concept, method or decision.
  • Confirmation bias: tendency to search for, interpret, favor, and recall information in a way that confirms or supports one's prior beliefs or values
• Questions the company should ask first:
  • Is the expected outcome clear to the decision maker?
  • Do the decision makers know what and why they need from a data analyst?
  • Do they understand what the newly hired machine learning expert should bring to the table?
  • Is the company’s product even ready for including some data driven personalisation of the product?
  • Do people in marketing know how to use the customer segmentation / personas done by k-means clustering?
  • Are the digital product designers aware of what the behaviour of the users on the website means and how they should re-design the product?
  • Are the decision makers aware of how the data insights should be reflected to beat up the competition?
• The data analyst must put his/her work into a larger context. They must realise he/she is seeing only the quantitative part of the pictur – link the raw quantitative world into what the customer feels when using the product/service. It is not about how many times we log XY customer’s action. It is about understanding why and when log XY is triggered by a customer.