The two most relevant software delivery process performance metrics are:

• Stability: delivery process stability is directly related with the quality of the output

• Throughput: delivery process throughput is directly related to the efficiency with which the output is produced

Although not strictly a delivery process performance metric, but a business one, Cost of Delay is directly related with the delivery process and it is an essential metric. Cost of Delay is out of scope of this study though.

Delivery Process Performance Metrics

Each of the two delivery process performance metrics is a composition of two variables:

Open

Stability

• Change Failure Rate (CFR): ratio between the number of changes that failed to produce a successful output and total number of changes introduced in the process

• Failure Recovery Time (FRT): time that it takes to bring back the process to deliver a successful output after a failed one

Throughput

• Lead Time: how long it takes for the code to flow through the

  process from the beginning until the end

• Frequency: how often changes lead to a successful output when the code change flows through the process

  • Instead of frequency, we will use its inverse, time interval, which is easier to measure and manipulate, given that its expressed in units of time.

  • Time Interval is the amount of time that elapses between two consecutive completed code flows across the delivery process.

In terms of units, three out of four variables are expressed as time unis and the other one (CFR) is expressed in percentages.

Delivery Process Performance metrics vs DORA metrics

Please do not mistake these two generic process performance metrics with the DORA metrics. Although the name of the metrics are the same, the variables that define them are not.

DORA considers a delivery process that does not end with the code in a “releasable state”, as defined in Continuous Delivery, but one where the value stream goes all the way down to the deployment.

This approach, common among cloud based environments, was incorporated to DORA once Google acquire it, after Accelerate was published. It does not apply to most open source projects, operating system or embedded ones, where the organization in control of the delivery does not control the deployment environment in most cases.

Open source projects are part of complex supply chains. Such concept es essential in automotive too.

Delivery Process Performance Metrics: iteration 2 of the AGL Delivery Process model

One of the reasons why the simplest metrics chosen to characterize the delivery process performance are Throughput and Stability is because they can be applied to the whole process but to specific stages as well. Throughput and Stability applied to the iteration 2 of AGL’s Delivery Process model turn into:

Open

To differentiate if we are talking about a specific metric measured across the entire Delivery Process or across a specific stage of it, we will use the stage name on the metrics and/or variable, like shown in the image below.

Open

Why there is no stability metric associated to the commit stage?

Stability measures changes that become a source of a failure and how long it takes for the system to recover from it. A commit that fails is discarded, so it does not make it into master/main. It is a binary result which provides to Stability little sense as metric. Check at the Reads section the book Measuring Continuous Delivery for more details.