Digital Analytics 201: Why Raw Numbers Don't Tell the Whole Story

Recital Hall
Digital Analytics 201: Why Raw Numbers Don't Tell the Whole Story
In this session, I'll talk about some of the basics of data science for the web. I want to share some of my experience and lessons I have learned on the front lines of working with customer-facing applications for [a large B2C company].
The raw numbers we get from digital analytics tools like Google Analytics, KissMetrics, Adobe Analytics, Adwords, etc. are often treated as “The Truth” in many organisations. But they only tell us part of the story. When a metric is up by a few points, how do we know it's "improved"? How do we know it “matters”? Do we declare victory for our new marketing because we see a few hundred more pageviews? Or should we be looking for our next content strategy?
When you come to this session, be prepared to think about statistics, but don't worry if you've never seen this kind of math before, I did all the calculator stuff for you already.
By the end of it, I want to:
  • give you a basic understanding of statistical significance and standard deviation and why they are important
  • show you how to think about measurement and defining "success" or "failure"
  • show you split testing and multivariate testing, explain how they're different, and why they're important to understanding your outcomes
Business and Open Source