www.propelo.ai Open in urlscan Pro
2606:4700:20::681a:74f  Public Scan

Form analysis
0 forms found in the DOM

Text Content

 * Why Propelo
   * Data Sources
   * Enterprise Readiness
   * Security and Privacy
   * Pricing
 * Use Cases
   * DORA Metrics
   * Business Alignment
   * Planning
   * Engineering Execution
 * Engineering Excellence
 * Resources
 * Blog
 * Company
   * Contact
   * Careers

Free Trial


--------------------------------------------------------------------------------

Blog Layout

--------------------------------------------------------------------------------


THE DORA DEVOPS METRICS – EVERYTHING YOU NEED TO KNOW

Priya Kothari • Mar 09, 2022

How quickly can your team deliver a new project?

Why can’t you take on new work?


How often are we deploying?

How quickly are we able to recover from Production Issues?




These are all questions that Engineering Leaders have to answer to the
Management, as well as internally. Engineering organizations are increasingly
expected to deliver software more frequently and with better quality. To meet
this demand, software teams are embracing Agile development methodologies and
DevOps to varying degrees. As they try to get more efficient in their agile
practices, software delivery teams need software metrics to measure their
development speed and stability. 




The complexity of distributed teams, outsourced projects and remote work makes
it even more critical to have the right metrics defined to measure the overall
DevOps performance.

DORA METRICS: THE 4 KEY INDICATORS OF ELITE DEVOPS PERFORMANCE


There are 4 key metrics that many organizations use to measure their delivery
performance. These metrics were suggested by the DevOps Research & Assessment
(DORA) research program, and hence are known as DORA metrics.  The 4 DORA
Metrics are:

Software Delivery Lead Time

Software Delivery Lead Time (also known as Lead Time for Changes) is the amount
of time it takes a commit to get into production - from the time the software
development team starts working on a task to the time the customer gets the
feature. This includes the time spent in build, test, integration,
implementation and deployment, and does not include the time spent in design,
prioritization or backlog




Deployment Frequency

Deployment Frequency is the number of times code or software is deployed to
production or “shipped”. This metric helps organizations determine and set their
delivery cadence.




Mean Time To Restore (MTTR) 

MTTR is the time it takes to restore a failure in production. A failure can be
an unplanned outage or a service failure.




Change Failure Percentage

This is the percentage of deployment causing a failure in production. It is the
measure of the number of times “a hotfix, a rollback, a fix-forward, or a patch”
is required after a software deployment or a service change.




Figure: Lead Time Broken Down Into Various Stages

HOW TO MEASURE DEVOPS SUCCESS: WHY DORA METRICS ARE IMPORTANT


The four DORA metrics provide a great baseline to measure the tempo, rhythm and
responsiveness of an engineering organization.

A.  Development Velocity

The first two metrics are a measure of software delivery performance “tempo”,
also known as Development Velocity. It is important for organizations to
understand their development velocity. 


Lead Time

Lead Time helps organizations understand how quickly they can deliver software.
It gives you a sense of the efficiency of the development teams. With shorter
lead times, you can deploy to production in smaller deployments and more often.
This enables faster feedback on what is getting built and allows for quicker
course correction. Conversely, longer lead times signify bottlenecks in the
development process.



Lead Time is a great metric to track, especially if looking at the trend over
time. It shows whether there are any issues, and if things are getting better or
worse. And at the end of the day, how quickly you are able to respond to the
business. However, what is more important is to get further breakdown of the
different stages. 




 * Are there any bottlenecks? What stage are they in? 
 * Are PRs taking long to Review? If so, why? 
 * What is the Merge Time? 




These are all good questions to ask. With a combination of data from the Project
Management, SCM and the CI-CD and Deployment systems, this breakdown can be
achieved.

Deployment Frequency

Deployment frequency helps organizations set their release schedules. Many SaaS
organizations chose to deploy builds frequently - some even on a daily basis.
Smaller, more frequent builds help reduce risk. However, not every organization
will want to or need to deploy very quickly or frequently. SaaS organizations
may need to deploy very frequently. On the other hand, for certain business
applications, deployment frequency of once or twice a year might be sufficient -
their customers may not be happy with frequent changes. 


Figure: Trend - Lead Time to Change over time

Figure: Trend - Count of deployed work items over time

Software teams should evaluate the needs of their business and ensure that the
velocity of their development process (the Lead Time and Deployment Frequency)
matches their business need. The principles of Lean and Agile can still be
applied by delivering software in small batches rather than delivering as large
monoliths.


B.  Software Quality

Software development organizations often put a lot of emphasis on improving the
Development Velocity. However, perhaps even more important is the Quality of
their output. 


Change Failure Percentage

Change Failure rate can give organizations a sense of how frequently they are
shipping out code that causes issues. Ideally, the Change Failure Percentage
should be as low as possible indicating good quality code.


Mean Time To Resolution (MTTR)

Change Failure rate can give organizations a sense of how frequently they are
shipping out code that causes issues. Ideally, the Change Failure Percentage
should be as low as possible indicating good quality code.


Figure: Trend - Total Defects versus Jobs completed

Figure: Trend - Resolution Time And Number of Tickets

The above 2 metrics measure the reliability and stability of the software that
is delivered. Together, these are a good indicator of the quality of the
development output.




As teams grow, it is critical to find a balance between how much and how often
to deploy vs how stable the product is. A higher development velocity is
important but should not come at the expense of quality or the stability of the
delivered software.

WHAT IS A DORA REPORT?




Engineering leaders need to be able to evaluate the performance of their
organizations on an ongoing basis. The DORA report is a great way to start
getting some initial insight into the development velocity and software quality.
With these metrics, you can start to see if there are any bottlenecks in the
development process, and the quality of their output.




It is however important to not view the numbers as absolute. Being able to track
DORA metrics on a regular basis helps you see the trends - this could be a
better indicator of issues.


Figure: The Four DORA Metrics

GETTING STARTED WITH DORA METRICS




Accurately measuring DORA metrics can be a challenge for most organizations.
Much of the data that is needed to calculate these DevOps metrics lies in
various systems across the DevOps toolchain - project management, SCM, CI/CD,
service desk, issue tracking, and other systems. The data must be parsed, broken
into spreadsheets, and then correlated to get the right DORA metrics.




As an example, consider an organization that uses Jira for their planning,
GitHub for SCM, Jenkins for CI/CD, ServiceNow for service desk, PagerDuty for
production monitoring, and various other tools for testing, security, etc. This
is a very common scenario in many organizations where they select different
tools that meet their needs for different purposes.




To get a metric such as Lead Time, you need to correlate the data from Jira with
data from GitHub and Jenkins, so you can accurately understand how much time was
required from the start of the task till it made it to Production.




Similarly, for accurate MTTR, you need to correlate data from PagerDuty back to
ServiceNow and then to Jira. This can be quite a challenge.


BEYOND DORA - OTHER AGILE DEVOPS METRICS




In addition to the 4 DORA metrics, there are several other metrics that can help
engineering teams determine their efficiency, productivity and their
bottlenecks. Some of these are:


PM Hygiene

 * Requirements Clarity
 * Sprint Distribution
 * Details of User Stories 
 * Prioritization
 * Acceptance Criteria & Test Cases




Sprint Hygiene

 * Story Point Calculation
 * Estimation Accuracy
 * Story Point Mix
 * Dependencies & Interactions
 * Distribution of workload between teams




PR Lifecycle

 * Time to First Commit,
 * Time to Review,
 * Time to merge,
 * Number of reviewers
 * Items stuck in various stages




CI-CD Effectiveness

 * Number of failed builds
 * Number of failed deployments
 * Number of rollbacks
 * Number of hotfixes




Incident Management

 * Number of Hops
 * Bounce backs
 * Time spent on resolution




Code Hotspots, Technical Debt, Security Considerations and a lot more.






ACCELERATING DEVOPS WITH DORA METRICS AND PROPELO




Propelo can help organizations solve this challenge. Propelo integrates with
over 40+ DevOps tools, and provides 150+ out-of-the-box software metrics and
insights into the performance of engineering organizations.




Propelo automatically correlates data across various systems and provides
accurate Lead Time information. It provides a detailed breakdown of time spent
in each stage. Users can drill-down into each stage and check which activity or
step within the stage takes the most time to complete. Is the PR stuck in review
or merge? Is the build taking too long? Or is the build waiting a long time for
deployment? With Propelo, you can analyze bottlenecks in the delivery phase of
the Lead Time and bubble those up for better visibility.




Propelo can accommodate various types of deployment models and technologies.
Whether you are pushing a package or container or turning on feature flags
Propelo can measure your Deployment Frequency.




Propelo lets you take a scalpel approach to precisely measure certain types of
failures across all your teams and gives you a standardized view of MTTR across
all your teams.




With Propelo, customers can define the meaning of a "failure" whether that is
captured in PagerDuty or Jira, and also control the meaning of a "hotfix" or
"rollback" or "fix-forward" whether it is in the SCM(GitHub/Gitlab) or CI/CD
(Jenkins). This allows for accurate calculation of Change Failure Percentage.


SUMMARY




Metrics are the foundation to understanding the efficiency and effectiveness of
your engineering organization. With the right software metrics, you can make
data-driven decisions and demonstrate alignment with the business towards
customer-centric outcomes.




DORA metrics provide a good foundation to start measuring development velocity
and software quality. Tracking DORA metrics regularly helps you see trends and
point out problem areas. However, DORA metrics can be hard to obtain since data
resides in different tools deployed across the DevOps toolchain. You need to
correlate data from various sources such as GitHub, Jira, Jenkins, PagerDuty,
etc, which can be difficult, time consuming and frustrating.




Propelo provides out-of-the-box software metrics and insights into the
performance of engineering organizations. It aggregates & correlates data from
over 40 tools across the DevOps toolchain, and centralizes it for easy
consumption and reporting. 




With Propelo, you get in-depth understanding of the factors that affect your
teams, so you can prioritize and resolve the issues that have the most impact on
your business.




To learn more about how Propelo can help with your path to Engineering
Excellence, visit www.propelo.ai or contact us at sales@propelo.ai


Newer Post > < Older Post


--------------------------------------------------------------------------------


PROPELO.AI - THOUGHT LEADERSHIP AND INSIGHTFUL CONTENT


PROPELO FAR AND AWAY THE LEADER IN ENTERPRISE DEVOPS ENVIRONMENTS, G2 REVIEWS
SHOW

By Mahesh Kumar • 01 Sep, 2022
G2 does not publish an enterprise market guide for software development
analytics. The reason may surprise you, but we knew that all along. Propelo is
the only vendor to satisfy the minimum required reviews in the enterprise
segment. And G2 needs a minimum number of vendors in that segment before
publishing a market guide. Why wait until other vendors catch up? Read our
reviews on G2 and let's examine why Propelo exhibits such strength in the
enterprise.


WHY CHOOSING THE RIGHT MEASUREMENT FREQUENCY IS CRUCIAL

By Mahesh Kumar • 31 Jul, 2022
For engineering leaders, it is important to measure metrics both in the short
term and medium term. Both metrics have their uses and ensure that metrics can
be acted upon in a deliberate manner and the outcomes are impactful to the
business.


DATA-LED ACTIONABILITY FOR VP OF ENGINEERING

By Mahesh Kumar • 31 Jul, 2022
Data and Insights alone are not sufficeint for VP of Engineering. They must take
actions driven by the data. What are the categories of actions, what metrics
drive those actions, how often to review data? These questions are answered in
this blog.
Show More


--------------------------------------------------------------------------------

 * Why Propelo
   * Data Sources
   * Enterprise Readiness
   * Security and Privacy
   * Pricing
 * Use Cases
   * DORA Metrics
   * Business Alignment
   * Planning
   * Engineering Execution
 * Engineering Excellence
 * Resources
 * Blog
 * Company
   * Contact
   * Careers

SALES@PROPELO.AI 

700 S BERNARDO AVE, STE 103 

SUNNYVALE, CA 94087


Free Trial





Share by: