Quantify
There are metrics to be found at all stages of the DevOps pipeline. It’s important to know which of these metrics is going to be useful to you by reviewing your existing processes. In order to know if your DevOps practices are having a positive impact, you need a good starting point to measure against.
From a business perspective, you should know how often you’re deploying to production. You should know how many of the deployments have resulted in outages or bugs with a measurable impact on the user base.
You should know the average time it takes your team to recover from outages with DevOps training. You should understand, at-a-glance, what your up-time is, and if you’re meeting any SLAs that you may be bound to. There are plenty of additional business level metrics worth tracking, though that’ll be something each company typically clarifies for themselves and their teams.
The technical side of DevOps will value different metrics. Knowing how long your CI process takes is important. The average response time of your REST services or the number of concurrent users at any given time represents useful data that may change the way developers solve for specific problems.
Knowing how the code is performing on the servers allows your engineers to quantify the impact code changes have on performance. This dovetails into understanding how your production servers are performing, and if you are over or under provisioned. Your operations team should have all the metrics they need to ensure that they are running the most elastic and secure infrastructure possible.
This tenet of “quantify everything” is a bit nebulous, because the volume of data is massive and growing at all different levels. Knowing what to track is crucial to any successful DevOps plan. In fact, if you’re new to DevOps, here are a few key performance indicators you should track, to get you started:
- Frequency of deployments
- The frequency of failed deployments
- Mean time to recovery (MTTR)
- Mean time to discovery (MTTD)
- Lead time
- Uptime
- Customer complaint volume
- Service performance
Start by capturing as much info about your current process as possible. Once you feel you have a good handle on your current metrics, you’ll have something to measure your DevOps efforts against.
The journey of a thousand miles begins with the first step. Once you have a serious understanding of your metrics, the DevOps philosophy becomes increasingly clear. There is a major art opening at a downtown museum and attendance is 5% of projections. Something is wrong. Imagine a Google Satellite image on your screen. What you see looks like a gray slab. You zoom out and find a building. Zoom out further, and city block becomes apparent. Further and you realize the bridge over a river is blocked by an accident and traffic is stalled. You’ve identified the problem and understand which parties must cooperate for an effective solution. The police should work with fire in clearing the accident and caring for any potentially injured parties. The city road teams must inspect the bridge for further damage, in case structural damages occurred. The art opening should expand hours or re-schedule. The DevOps philosophy supports this type of analysis, discovery, collaborative action.
