Balanced metrics: The key to success in DevSecOps transformation


Two workers sit at a table with a laptop and a notebook.

When measuring the success of large-scale transformations—particularly in the technology space—it’s natural to look at hard metrics, such as cycle time, mean time to recovery (MTTR), and so on. In IT, for example, hard metrics are what we do all day long.

But within any organization, change is ultimately personal. In my experience, relying exclusively on hard numbers often leads you to ignore the human side of transformation, and sometimes even action the wrong things. That’s why we use both hard and soft metrics to guide our DevSecOps transformation at ServiceNow.

Measuring progress in 2 ways

In our transformation journey, we look at more than 20 data points, with equal weight given to soft numbers and hard numbers:

  • Hard metrics are quantifiable, such as the number of releases flowing through the pipeline, the frequency of deployment, and the number of teams using the pipelines.

  • Soft metrics are qualitative, subjective values, including Net Promoter Score (NPS) and Adoption Readiness surveys of our program leads, that assesses the engineering community’s satisfaction with the transformation. We often follow these surveys with retrospectives where engineers can share their thoughts on changes from a people (culture), process, and technology perspective.

To help us transform insight into action, we use a combination of both metric types to consider the human side of a project via satisfaction and feedback surveys. Including soft metrics brings team members into the transformation process, giving them the chance to voice their views and influence change.

Teams that show great self-reported metrics, such as satisfaction scores, and hard metrics, such as deployment frequency, are the ones that make steady progress and find new ways to improve week over week.

On the flip side, teams that rely only on the rigor of hard metrics and overlook user satisfaction often hit a plateau quite quickly. This results in frustration—pushing hard but not reaping rewards.

I must admit that although I love to see high satisfaction scores, I welcome the low ones because they yield a goldmine of feedback that will help us improve how we work.

"We use both hard and soft metrics to guide our DevSecOps transformation." Manshu Verma, Senior Program Manager, IT Program Management, ServiceNow


Gaining insights from divergence

Hard and soft metrics often track with one another; good progress against hard metrics often results in high satisfaction. Where it gets interesting is when the two metrics diverge.

For example, one of our teams’ high satisfaction scores showed they were very happy with their new process. Yet, they were used to their previous release schedule and saw no great need to increase their deployment frequency. By considering both metrics, I knew we needed to create encouraging conditions for them to deploy more frequently and experience the benefits themselves.

At first blush, it may seem like low satisfaction scores are purely a symptom of cultural challenges, but a deeper look can reveal whether it’s the process, technology, or culture that needs fine-tuning.

As an example, one of our teams had low satisfaction scores—not because they resisted change, but because their unique setup wasn’t supported by our technology. They listed their complaints, and the DevSecOps tooling team enabled features to accommodate them. As a result, the releases going through the pipeline and satisfaction increased not only for this team, but also for other teams that had this problem but not as severely. (Read more about using metrics to gain visibility and achieve desired outcomes.)

Transformation is a journey

Transformation is an ongoing process, from crawl to walk to run to sprint. Because of that, attitudes and metrics are going to change. Be sure to take the pulse of your community periodically—on both hard and soft metrics—to assess your progress and make any necessary course corrections to achieve your goals.

Find out more about how ServiceNow uses its own products.

© 2021 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.

Topics

  • Image of magic quadrant leader report
    IT Management
    ServiceNow is again a Magic Quadrant Leader in Enterprise Agile Planning Tools
    ServiceNow has been recognized as a Leader in the Gartner Magic Quadrant for Enterprise Agile Planning Tools for the second year in a row. Find out why.
  • 3 women developers
    Application Development
    How 3 women are shaking up the developer community
    Three women from different backgrounds, inclinations, and industries are shaking up the developer community. Find out how and why they become developers.
  • ESG: solar farm
    Cybersecurity and risk
    Survey: ESG is a business imperative
    Businesses across the globe are under pressure to address environmental, social, and governance (ESG) issues. Learn ESG benefits and how to reach ESG goals.

Trends & Research

  • ServiceNow is a Leader in the Forrester Wave: Digital Process Automation, Q4 2021.
    AI and Automation
    ServiceNow recognized as a Leader in Digital Process Automation
  • ServiceNow is a Leader in the Forrester Wave: Enterprise Service Management, Q4 2021.
    Employee Experience
    ServiceNow named a Leader in Enterprise Service Management
  • Supporting hybrid work: A woman looks at a phone while writing in a notebook.
    Employee Experience
    ServiceNow a Strategic Leader in the 2021 Fosway 9-Grid™ for Cloud HR

Year