Measurements

Being data-driven is an incredibly lofty goal. Many companies want to get to the point where the data leads them precisely where they need to go.

Here’s the problem:

The data you choose is relevant to track, by definition, will influence what decisions then bubble up.

Here’s a simple example:

Let’s assume you are a software agency for mid-sized markets. You may decide to track Support Ticket Time, or the time that lapses between a ticket being filed, and the client receiving a notification that the ticket is being processed.

The underlying assumption is that the shorter the time, the happier clients will be.

So, with that now being tracked and deemed important, you set out to improve the metric:

1. You create a custom ticketing software that strongly notifies the relevant team members when an issue comes in.
2. You associate ticket response times with bonuses to the team.

What are the second-order effects?

• Your team is tired and frustrated from being woke in the middle of the night
• Team members are now only deploying once per day in the morning and never on Friday’s. Issues that come in then can be addressed the rest of the day for an easier way to get “fast response times”.
• Some team members may realize how to game the system - introduce known, easy to fix bugs. They then jump immediately on the issue and bring not only the average response, but average time to fix time down. This earns recognition, promotion, and bonuses.

Don’t be data-driven. Be data-informed.

Most data points are not the goal - they are a proxy for the true thing you are trying to measure:

• customer satisfaction
• quality
• team engagement
• growth rate

The data helps you to identify potential issues. Work on identifying the real problem underlying those issues, and fix that. Don’t just focus on the metric.

After all, you can measure only 3% of what matters.