How we use actionable metrics at Skillshare

At Skillshare, we believe that metrics are important to get a pulse of how our business is doing and figure out what to do next. Essentially that means, we should know what’s working and what’s not, and thus, what areas of our business to focus on next.

This can be more macro, such as identifying gaps in our customer lifecycle, or more micro, such as figuring out the optimal call-to-action and experience in a conversion flow. Before we work on any feature/solution at Skillshare, we always make sure we’ve identified what problem we’re solving, and what the measure of success is. After the feature/solution is deployed, then it’s all about learning — did our solution solve the problem and by how much — and iterating to continuously improve the solution.

But not all metrics are created equal.

In fact, sometimes they can simply be extraneous data, or worse, more of a distraction to the team.

Vanity metrics (as coined by Eric Ries) are those that are fun to look at, but provide little-to-no value in actually helping you understand what areas of your business are working / not-working, and what actions you should take to improve.

For instance, at Skillshare we don’t worry about the number of users, visitors, or pageviews that we get. Those numbers can fluctuate day-to-day, and regardless if it’s showing an upward or downward trend, it doesn’t really help us decide what to do next. We do use gross ticket sales (GTS) as a nice-to-have pulse of where our revenue numbers sit, but we recognize that it’s not going to tell us any great insight either.

So, what metrics should you pay attention to?

The litmus test we use at Skillshare is essentially: does this metric help us make a decision on what to do next? If you’re looking at a metric, and first go “Great!” (or “Ouch!”), but then go “OK so what?” and can’t figure out what to do next, then it’s probably a Vanity metric.

When thinking about what metrics to follow, I like to think of a user conversion funnel:

1. Acquisition (e.g. registered user)
2. Activation (e.g. first student / teacher experience)
3. Retention (e.g. repeat student / teacher)
4. Super users

This is similar to Dave McMclure’s AARRR framework, but shifted further down the funnel a bit for our own purposes (and recognizing that students generate revenue from first activation).

In the current stage of our business, we especially pay attention to activation and retention. For example, what percentage of users attend or teach a class (activation)? You can then add in other factors, such as demographic info. Another example: out of the teachers and students we have in our community, what proportion of them are coming back to teach and attend more classes (retention)? This metric is, in a way, a measure of the quality of our product and of the in-class experience.

Other examples of actionable metrics may include:

• Loyalty (time since last visit)
• Utilization rate of assets
• User lifetime value (LTV)
• User cost-per-acquisition (CPA)

Focus and simplicity are important.

We keep our team focused on the biggest opportunities that will bring the highest ROI in moving the needle. In other words, we only use metrics to help us find a scalable and repeatable model for success, as opposed to trying to optimize a model we’ve already validated. This also implies that we don’t spend countless hours buried in numbers nor dedicating a large amount of resources to build robust analytics dashboards.

In fact, as a young startup, metrics only account for perhaps 20% of our decision-making, with gut instincts and experience making up the remainder. But as our user-base and engagement grows, I’m a firm believer that metrics should play a much larger role, being the primary input for 80% of our decisions.

Closing Remarks

Actionable metrics are important inputs to reveal gaps in your product and opportunities for your business. Without them, you’re most likely making decisions and “improvements” to your business based on a lot of assumptions without any knowledge of whether or not your specific efforts are paying off.

Just try to stay away from being distracted by vanity metrics, and keep things simple and focused.

In a future article, I’ll talk in more detail of how we use A/B Testing and Cohort Analysis as part of our analytics arsenal. In the mean time, what are some ways you / your company uses analytics, and what have you found to be the most helpful?