Don’t Let Metrics Debt Kill Your Business

Caid Christiansen• May, 19 2015

Under-investment in metrics can lead to long-lasting negative effects on your organization.

That’s the thought of Jamie Quint, Managing Partner at Quint Growth Partners, a “full-service growth consultancy.” He calls under-investment in metrics “metrics debt,” a term that we can get behind.

As Quint notes, ad hoc analyses are extremely slow or impossible, a poorly implemented metrics solution is often worse than no metrics solution at all, and limited metrics can prevent product managers and executives from asking important questions.

So what makes a good metrics system? For Quint, a good system has the following characteristics:

• User behavior is tracked on a per-session basis and what exactly led to each session can easily be identified.
• All key events that a user can do on each page in your site/app are tracked and all user-level meta-properties on those events are properly attributed.
• Pre-login behavior is tied to post-login behavior and behavior is tied between devices as much as possible.
• Events are coherently named, and you have a list with a textual description of what each event represents.
• Metrics development keeps up with product development.
• All metrics data is recorded into a raw data-store (e.g. Amazon Redshift) from which you can easily run ad-hoc analyses or verify 3rd party analytics reporting.

And we’d tend to agree. Quint’s description of metrics debt, and subsequent outline of what makes a good metrics system at an organization, is one of those rare ideas we come across that we like so much we feel inclined to write about it.

The thing is, strategy and execution are only part of the story. Measurement, evaluation, and refinement are the other part, and the first part isn’t much good without the second. While sure, it might be good to invest in a marketing initiative and see a bump in sales without closely tracking the connection, it’s even better to be able to understand why that bump happened and how you might go about replicating it.

An organization can only do that if they have a strong foundation not only in strategy, but also in metrics. Metrics debt is real. We’ve seen it hurt a lot of organizations.

Is metrics debt killing your business?