Balanced metrics: The key to success in DevSecOps transformation


Two workers sit at a table with a laptop and a notebook.

When measuring the success of large-scale transformations—particularly in the technology space—it’s natural to look at hard metrics, such as cycle time, mean time to recovery (MTTR), and so on. In IT, for example, hard metrics are what we do all day long.

But within any organization, change is ultimately personal. In my experience, relying exclusively on hard numbers often leads you to ignore the human side of transformation, and sometimes even action the wrong things. That’s why we use both hard and soft metrics to guide our DevSecOps transformation at ServiceNow.

Measuring progress in 2 ways

In our transformation journey, we look at more than 20 data points, with equal weight given to soft numbers and hard numbers:

  • Hard metrics are quantifiable, such as the number of releases flowing through the pipeline, the frequency of deployment, and the number of teams using the pipelines.

  • Soft metrics are qualitative, subjective values, including Net Promoter Score (NPS) and Adoption Readiness surveys of our program leads, that assesses the engineering community’s satisfaction with the transformation. We often follow these surveys with retrospectives where engineers can share their thoughts on changes from a people (culture), process, and technology perspective.

To help us transform insight into action, we use a combination of both metric types to consider the human side of a project via satisfaction and feedback surveys. Including soft metrics brings team members into the transformation process, giving them the chance to voice their views and influence change.

Teams that show great self-reported metrics, such as satisfaction scores, and hard metrics, such as deployment frequency, are the ones that make steady progress and find new ways to improve week over week.

On the flip side, teams that rely only on the rigor of hard metrics and overlook user satisfaction often hit a plateau quite quickly. This results in frustration—pushing hard but not reaping rewards.

I must admit that although I love to see high satisfaction scores, I welcome the low ones because they yield a goldmine of feedback that will help us improve how we work.

"We use both hard and soft metrics to guide our DevSecOps transformation." Manshu Verma, Senior Program Manager, IT Program Management, ServiceNow


Gaining insights from divergence

Hard and soft metrics often track with one another; good progress against hard metrics often results in high satisfaction. Where it gets interesting is when the two metrics diverge.

For example, one of our teams’ high satisfaction scores showed they were very happy with their new process. Yet, they were used to their previous release schedule and saw no great need to increase their deployment frequency. By considering both metrics, I knew we needed to create encouraging conditions for them to deploy more frequently and experience the benefits themselves.

At first blush, it may seem like low satisfaction scores are purely a symptom of cultural challenges, but a deeper look can reveal whether it’s the process, technology, or culture that needs fine-tuning.

As an example, one of our teams had low satisfaction scores—not because they resisted change, but because their unique setup wasn’t supported by our technology. They listed their complaints, and the DevSecOps tooling team enabled features to accommodate them. As a result, the releases going through the pipeline and satisfaction increased not only for this team, but also for other teams that had this problem but not as severely.

Transformation is a journey

Transformation is an ongoing process, from crawl to walk to run to sprint. Because of that, attitudes and metrics are going to change. Be sure to take the pulse of your community periodically—on both hard and soft metrics—to assess your progress and make any necessary course corrections to achieve your goals.

Learn more about ServiceNow DevOps and Value Stream Management.

© 2021 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.

Topics

  • Project portfolio management: 2 employees looking at a whiteboard
    IT Management
    Why project portfolio management matters for digital transformation
    Project portfolio management (PPM) can help teams improve project visibility, target the same goals, make better decisions, and achieve project success.
  • Veterans Day - November 11. Service shapes ServiceNow. Our veterans, the world over, make us stronger.
    Culture
    Honoring veterans at ServiceNow
    One consistent theme in conversations with veterans at ServiceNow is their dedication to serving others and an attitude of selflessness. Learn more.
  • Agile project management: 3 employees discuss a project while posting sticky notes.
    IT Management
    The payoffs of Agile project management
    A key factor to success in digital transformation is using the right delivery methodology. Increasingly, the answer is Agile. Find out why in this blog.

Trends & Research

  • Supporting hybrid work: A woman looks at a phone while writing in a notebook.
    Employee Experience
    ServiceNow a Strategic Leader in the 2021 Fosway 9-Grid™ for Cloud HR
  • Business resilience: Two employees with face masks work at computers in an office.
    Cybersecurity and risk
    The fruits of business resilience
  • ServiceNow named a Leader in the 2021 Gartner Magic Quadrant for Enterprise Low-Code Application Platforms
    Application Development
    ServiceNow recognized a Leader in 2021 Gartner® Magic Quadrant™ for Low-Code Application Platforms

Year