Task templates and action alerts are
powerful tools for automating IT incident response in ServiceNow,
especially for organizations that lack a sophisticated Configuration
Management Database (CMDB). Using the two in tandem enables filtering
of incoming events, prescribes steps for determining severity, and
helps determine how incident response should be assigned.
But as Louis Oliver, senior technical consultant at Mary Kay,
explained in a breakout session at Knowledge 2019, these advanced
automation tools have long had hidden drawbacks. With the ServiceNow
Kingston release, task templates often required rigid sets of rules to
define sequences of steps taken by various assignment groups. The more
steps in sequence and the more groups of users potentially impacted,
the greater the chance alerts could become unwieldy to manage and resolve.
The rigidity of task templates also made them brittle. If some
aspect of a template needed to be revised—for example, if a personnel
change placed a new person in charge of a particular service—changing
the filters for that instance would change them for every instance.
As a result, Oliver’s team would frequently get bogged down in
manual triage before incidents could be assigned—a process that
occasionally went on for weeks. “It breaks the whole thing,” said
Oliver. Somebody would call us and say, ‘Hey, I got a ticket that
shouldn't go to my team. Why did we get this?’ We were getting those
calls all the time.”
With the London release came the solution Oliver was waiting for: a
new alert management tool and another tool to manage
subflows—workflows that operate beneath a
parent workflow. Like parent workflows, subflows are flexible and
reusable, but they handle more specific routines. For Oliver,
implementing these features prevented alerts from triggering
exponentially-expanding branches of task templates. They also created
a clearer view of the specific workflow tied to each alert.
The upgrade paid off dramatically. “On average, we used to work
through seven or more task templates and order action rules for each
alert management rule per element,” said Oliver. Mary Kay has since
slashed average incident resolution times from more than seven days to
less than 24 hours. In many cases, Oliver’s team can resolve incidents
in less than an hour.
“Before, it might take us two to three weeks to figure out who to
assign an incident to make sure we were going to the right place, and
not accidentally breaking something else,” Oliver said. “Now it can
take as little as 15 minutes.”
Oliver advised companies still running the Kingston release to
upgrade to London before attempting to create subflows on their own,
as these subflows won’t be editable after making the upgrade.
“If you haven't gone to London yet and you're heavily leveraging
alert action rules and task templates, be prepared,” said Oliver.
“You'll have to change the way you do it.”