BLOG

DORA metrics – Dos & Don’ts for achieving engineering excellence

Peter Drucker’s famous quote, “If you can’t measure it, you can’t manage it,” provides clear guidance to any organization seeking improvements in its processes. Today, every software development organization seeking higher agility, efficiency, stability, and reliability in its operations measures a wide range of metrics. The metrics hold valuable insights for improving the health and performance of software development practices. However, many times tracking the developer productivity metrics can cause undesired results. For instance, tracking the lines of code often leads to a dramatic fall in the quality of the code. Similarly, tracking the number of bug fixes can force quality testing teams to log every minor and trivial bug, affecting the shipping of code. At the same time, organizations also realize that not all metrics are created equal, and some can also mislead or provide a false sense of comfort.

DORA’s (DevOps Research and Assessment) research program recognized these problems. It came up with its set of metrics that are now considered the holy grail for measuring the success of software development. Each year, DORA surveys thousands of organizations and rates their performance on these four key metrics, viz. deployment frequency, mean lead time for changes, change failure rate, and mean time to restore. The survey reports provide a sense of DevOps trends across the globe and help teams compare themselves to their peers, identify areas for improvement, and take steps to become elite performers.

While the metrics are beneficial and can help teams achieve continuous improvements, it is possible to misuse DORA metrics inadvertently. The most common challenge with metrics is that they are often misunderstood and end up becoming the goal, which is an anti-pattern.

Using DORA Metrics Correctly

Organizations seeking to improve performance on DORA metrics need to eventually improve their project management and CI/CD processes. Organizations can improve their DORA metric scores by following the CI/CD best practices:

breaking down a project into smaller tasks and batches and implementing trunk-based development,

reducing the number of manual handoffs with automated testing and deployments, and
increasing the integration across CI/CD tools for continuous compliance, governance, and release management.

Again, DORA metrics are only lagging indicators and organizations need to tack additional leading indicators for predictability and proactive response. For example, improvements in quality can be achieved by monitoring test automation metrics such as test pass rate, test code coverage, and more. These metrics can help teams identify issues with their testing and code review processes earlier in the cycle and take remedial actions.

Similarly, teams must track their system health and performance continuously, improve tool integration, and automate incident response playbooks to achieve better efficiencies in hybrid and multi-cloud operations. Minimizing the noise from redundant alerts and notifications and providing better context, prioritization, and traceability into issues helps teams respond better. As cloud operations are becoming increasingly intertwined with modern DevOps and infrastructure as code (IAC) practices, SRE metrics such as error budgets, service level objectives (SLOs), errors, availability, and latency can also help organizations make progress toward engineering excellence.

DORA Metrics – Common Pitfalls

It is important to realize that DORA metrics implementation alone cannot guarantee better business outcomes. Many teams track DORA metrics due to the fear of missing out without understanding how to translate DORA metrics for their business. For example, metrics like lead time and deployment frequency may indicate the speed at which work items travel through a pipeline and get released to production; however, businesses are more interested in tracking the number of new features they can release every month/quarter. Visibility into chunks of work getting released at a certain frequency doesn’t answer if the teams have built what the business and customers want.

Again, overdependence on the DORA metrics can be a pitfall. While these metrics are markers of velocity and stability, they don’t offer any insights into how to improve. Organizations fail to act on these metrics as they lack the means to drill down or correlate them with other key indicators like pull request size, review time, code churn, and more.

Lastly, it is too wishful to expect positive changes without onboarding the teams on the ground. Organizations shouldn’t expect developers to be excited about tracking their DORA dashboards. Failing to optimize developer workflows and expecting the metrics to induce continuous change isn’t practical. Businesses need to stop considering DORA or any other standard metric framework (Flow, SPACE, etc.) as the be-all-end-all of engineering excellence and need to focus on a more accurate picture with increased customization, context, and traceability.

Choosing the Right Metrics for Engineering Excellence

As businesses don’t operate in a static world, they need to frequently adjust the target values and thresholds of their metrics to adapt to the changing environment. New frameworks like SPACE can provide better visibility into developer productivity. However, organizations must carefully select and customize the metrics to meet their business objectives.

Gathr helps organizations create their DevOps monitoring dashboards with custom metrics and visualizations. It is possible to integrate new tools in Gathr within minutes, use Excel-like formulas to change definitions, and leverage out-of-the-box apps and templates to automate developer workflows. You can explore our DevOps solutions and request a demo to experience the power of Gathr.