The Problem with Vanity Metrics

Vanity metrics are measurements that look impressive but do not connect to business outcomes. Page views, registered users, app downloads, and social media followers are classic vanity metrics — they can all increase while the business is declining. The antidote to vanity metrics is a metrics framework that traces the causal chain from user behavior to business outcomes. If your business model is subscription SaaS, the chain looks like this: acquisition (users find and sign up for the product) → activation (users experience the core value) → retention (users continue to use the product) → revenue (users pay for continued access) → referral (users recommend the product to others). Every metric in your framework should map to one of these stages and have a clear causal relationship to business outcomes.

The AARRR Metrics Framework

The AARRR framework (Acquisition, Activation, Retention, Revenue, Referral), developed by Dave McClure, provides a practical structure for product metrics.

Choosing Your North Star Metric

The North Star Metric (NSM) is the single metric that best captures the core value your product delivers to users. It is the metric that, if it grows, indicates that the product is delivering more value and the business will grow as a result. Examples: Spotify's NSM is time spent listening (more listening = more value delivered = more retention and revenue). Airbnb's NSM is nights booked (more bookings = more value for hosts and guests = more revenue). Slack's NSM is messages sent (more messages = more team communication = more retention). Your NSM should be: a leading indicator of business outcomes (it predicts revenue and retention), measurable in real-time, actionable (your team can take specific actions to improve it), and not a vanity metric (it should be hard to game).

Building a Metrics Hierarchy

The North Star Metric sits at the top of a metrics hierarchy. Below it are input metrics — the specific user behaviors that drive the NSM. Below those are diagnostic metrics — measurements that help you understand why input metrics are moving. This hierarchy enables the team to move from 'our NSM is declining' to 'our Day 7 retention is declining' to 'users who do not complete the onboarding checklist have 40% lower Day 7 retention' to 'we should improve the onboarding checklist completion rate.' Without this hierarchy, teams know something is wrong but cannot identify what to fix.

Instrumentation and Data Quality

A metrics framework is only as good as the data behind it. Instrumentation — the process of adding tracking code to your product to capture user behavior — is a technical investment that pays dividends in product decision quality. Best practices for product instrumentation: use a consistent event naming convention (e.g., object_action: 'user_signed_up', 'feature_activated'), track events at the right granularity (not so granular that data is overwhelming, not so coarse that it is uninformative), validate instrumentation regularly (check that events are firing correctly and completely), and establish data governance (who can add tracking? how are events documented? how is data quality monitored?). Poor data quality is the most common reason product teams cannot answer basic questions about user behavior.