Skip to main content

Check out Port for yourself 

Track DORA metrics

This guide is designed to help you implement and track DevOps Research and Assessment (DORA) metrics within your organization in Port.

DORA Metrics are a set of key performance indicators that measure the effectiveness and efficiency of your software development and delivery process. By tracking these metrics, you can identify areas for improvement and ensure that your team is delivering high-quality software efficiently. This guide will cover the four key metrics: deployment frequency, lead time, change failure rate, and mean time to recovery.

Prerequisites​

  • Complete the Port onboarding process.
  • Access to a repository (GitHub, GitLab, or Azure Repos) that is connected to Port via the onboarding process.
  • While this guide demonstrates implementations using GitHub, GitLab, and Azure Repos, other Git providers can be used as well.
  • Optional for advanced strategies: If you're using workflows or pipelines, ensure they are configured for deployment tracking by following the relevant setup guides, such as CI/CD integrations or your platform-specific tools.

Tracking deployments​

In this section, we will cover how to track your team's deployments. Deployments refer to releasing new or updated code into various environments, such as Production, Staging, or Testing. Tracking deployments helps you understand how efficiently your team ships features and monitors release stability.

Deployments contribute to three key DORA metrics:

  • Deployment Frequency: How often changes are deployed to production or other environments.
  • Change Failure Rate: The percentage of deployments that fail and require intervention, rollback, or generate issues.
  • Lead Time for Changes: The time it takes from code commit to deployment into production.

To track the necessary data for these metrics, we will create a Deployment Blueprint with properties that capture the essential information for each deployment.

Data model setup​

  1. Navigate to your Port Builder page.
  2. Click the + Blueprint button to create a new blueprint.
  3. Name it Deployment and add the schema below:
Deployment blueprint (click to expand)
{
  "identifier": "deployment",
  "title": "Deployment",
  "icon": "Rocket",
  "schema": {
    "properties": {
      "createdAt": {
        "title": "Deployment Time",
        "type": "string",
        "format": "date-time",
        "description": "The timestamp when the deployment was triggered."
      },
      "environment": {
        "title": "Environment",
        "type": "string",
        "enum": [
          "Production",
          "Staging",
          "Testing"
        ],
        "description": "The environment where the deployment occurred."
      },
      "deploymentStatus": {
        "title": "Deployment Status",
        "type": "string",
        "enum": [
          "Success",
          "Failed"
        ],
        "description": "Indicates whether the deployment was successful or failed."
      }
    },
    "required": []
  },
  "mirrorProperties": {
    "leadTimeHours": {
      "title": "Lead Time (Hours)",
      "path": "pullRequest.leadTimeHours"
    }
  },
  "calculationProperties": {},
  "aggregationProperties": {},
  "relations": {
    "service": {
      "title": "Service",
      "target": "service",
      "required": false,
      "many": false
    },
    "pullRequest": {
      "title": "Pull Request",
      "target": "githubPullRequest",
      "required": false,
      "many": false
    }
  }
}
Missing Lead Time

If you do not have the lead time configured, you can follow the example guides below to map resources for your Git provider's respective blueprint:

  • Open the Pull Request Blueprint: GitHub
  • Open the Merge Request Blueprint: GitLab
  • Open the Pull Request Blueprint: Azure DevOps

Tracking strategies​

Below are the main ways you can track deployments directly within Port:

One of the ways to track deployments is by monitoring when pull requests (PRs)/ merge request (MRs) are merged into a branch, typically the main/master branch. This is the recommended approach for tracking deployments and calculating lead time.

The lead time for these merges is calculated as the difference between when the PR/MR was created and when it was merged.

Below is a demonstration of how deployment tracking can be implemented using the PR merge strategy.

Example:

  • When a PR is merged, a deployment entity is created in Port to represent the deployment that took place.
  • The lead time for that PR is calculated and added to the deployment as part of the blueprint.

Here is how you can implement this:

  1. Add Pull Request blueprint, sample can be found here.

  2. Add the configuration below to the data sources page in your Port portal, and select your GitHub integration:

Deployment config (click to expand)
 - kind: pull-request
   selector:
    query: .base.ref == 'main' and .state == 'closed'  # Track PRs merged into the main branch
   port:
     entity:
       mappings:
         identifier: .head.repo.name + '-' + (.id|tostring)
         title: .head.repo.name + " Deployment"
         blueprint: '"deployment"'
         properties:
          environment: '"Production"'  # Hardcoded value
          createdAt: .merged_at
          deploymentStatus: '"Success"' # Hardcoded value
       relations:
         pullRequest: .head.repo.name + (.id|tostring)
         service: .head.repo.name
Hardcoded values

The value for deploymentStatus is hardcoded to Success to treat all deployments as successful, and the environment is hardcoded to Production for the main branch in this example. You can modify these values based on your requirements.


Tracking incidents​

Incidents are essential for tracking key DORA metrics, including Change Failure Rate (CFR) and Mean Time to Recovery (MTTR). Effective incident tracking reveals insights into how frequently deployments fail and how quickly teams can resolve issues. This section outlines how to:

  • Use incidents to calculate CFR and MTTR.
  • Link incidents to services to track the impact of failures.
  • Aggregate metrics across incidents for better monitoring.

Data model setup​

Ensure that your PagerDuty incident blueprint is properly configured to map incidents to the correct services(gitHub). This includes defining the appropriate properties and relations for incidents.Follow this PagerDuty Incident Blueprint Setup Link to implement.

Add incident resolution time and recovery time properties

To add incident resolution time and recovery time properties to the PagerDuty incident blueprint, follow these steps:

  1. Navigate to your Port Builder page.

  2. Select the PagerDuty Incident blueprint.

  3. Click on the {...} button in the top right corner, and choose {...}Edit JSON.

  4. Add the following properties to the blueprint:

    Additional properties for PagerDuty Incident Blueprint (click to expand)
      "resolvedAt": {
        "title": "Incident Resolution Time",
        "type": "string",
        "format": "date-time",
        "description": "The timestamp when the incident was resolved"
      },
      "recoveryTime": {
        "title": "Time to Recovery",
        "type": "number",
        "description": "The time (in minutes) between the incident being triggered and resolved"
      }

Add incident resolution time and recovery time mapping config

To add incident resolution time and recovery time mapping config to the PagerDuty data source, follow these steps:

  1. Navigate to your Port Data Sources page.

  2. Select the PagerDuty data source.

  3. Add the following mapping to the incident mapping section:

    Incident mapping config for resolvedAt and recoveryTime (click to expand)
        resolvedAt: .resolved_at
        recoveryTime: >-
          (.created_at as $createdAt | .resolved_at as $resolvedAt |
          if $resolvedAt == null then null else 
          ( ($resolvedAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) -
            ($createdAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) ) / 60 end) # Time in minutes and divide by 3600 if you want it calculated in hours

Syncing incidents with PagerDuty and other tools​

To sync incidents from PagerDuty, follow the steps in the PagerDuty guide. The guide provides detailed steps for setting up integrations to track incidents related to deployments.

For other incident management tools, follow these respective guides:

Relating Incident to services​

The relation between the "PagerDuty incident" blueprint and the "service" blueprint is automatically created when you install the PagerDuty integration.
This ensures that each PagerDuty incident is related to the relevant service in Port.

Metrics​

We will now aggregate the DORA metrics at the service level, allowing us to track metrics such as Deployment Frequency, Change Lead Time, Change Failure Rate (CFR), and Mean Time to Recovery (MTTR) for each service.

Metrics Aggregation

If you want to track these metrics at higher levels, such as team or domain, makes sure the appropriate team or domain blueprints exist, and that they have relationships defined with the service blueprint. Then, you can apply the aggregation properties for these higher hierarchies, similar to how we are doing for the service blueprint below.

Aggregation​

The metrics in this guide are aggregated monthly. However, you can easily switch the timeframes to weekly, hourly, etc., based on your requirements.

Adding Aggregation to Blueprints

Before proceeding, follow these steps to add the aggregation and calculation properties to the Service Blueprint:

  1. Go to the Builder in your Port portal.
  2. Locate and select your Service blueprint.
  3. Click the {...} button in the top right corner, and choose {...}Edit JSON.
  4. Insert the respective aggregation or calculation properties under the aggregationProperties or calculationProperties section in the Service blueprint's JSON schema.
  5. Save your changes to apply the new aggregation configuration.

Add this aggregation property to calculate deployment frequency:

Deployment Frequency (click to expand)

"deployment_frequency": {
   "title": "Monthly Deployment Frequency",
   "icon": "DeploymentIcon",
   "type": "number",
   "target": "deployment",
   "query": {
      "combinator": "and",
      "rules": [
         {
            "property": "deploymentStatus",
            "operator": "=",
            "value": "Success"
         }
      ]
   },
   "calculationSpec": {
      "func": "average",
      "averageOf": "month",
      "measureTimeBy": "$createdAt",
      "calculationBy": "entities"
   }
}

Aggregation Data Availability

At this point, you can already visit each service to view the aggregated DORA metrics. However, note that the aggregation data will only be calculated based on newly ingested data moving forward. click here for more details on aggregation properties.

Visualization​

Port's dashboards allow you to create custom widgets to track metrics and monitor your team's performance over time.

Dashboard setup​

  1. Go to your software catalog.
  2. Click on the + New button in the left sidebar.
  3. Select New dashboard.
  4. Name the dashboard (e.g., DORA Metrics), choose an icon if desired, and click Create.

This will create a new empty dashboard. Let's get ready-to-add widgets

Adding widgets​

Setup Deployment Frequency Widget

  1. Click + Widget and select Number Chart.

  2. Title: Deployment Frequency - Monthly, (add the rocket icon).

  3. Select Display single property and choose Service as the Blueprint.

  4. Select an Entity and choose Monthly Deployment Frequency as the Property.

  5. Click Save.

Setup MTTR Widget

  1. Click + Widget and select Number Chart.

  2. Title: MTTR – Monthly Average (Hour), (add the pagerduty icon).

  3. Select Display single property and choose Service as the Blueprint.

  4. Select an Entity and choose Mean Time to Recovery as the Property.

  5. Click Save.

Setup Change Lead Time Widget

  1. Click + Widget and select Number Chart.

  2. Title: Lead Time for Changes (Hour), (add LineChart icon).

  3. Select Aggregate by property and choose Service as the Blueprint.

  4. Select average as the function and choose Hour for Average of and createdAt as the Measure Time By.

  5. Add custom Unit for Unit of Measurement (Hours).

  6. Click Save.

Change Lead Time Over Time

  1. Click + Widget and select Line Chart.

  2. Title: Change Lead Time Over Time, (add the LineChart icon).

  3. Choose Service as the Blueprint.

  4. Select an Entity and choose Lead time for change as the Property.

  5. Set Time Interval to Month and Time Range to In the past 365 days.

  6. Click Save.

Deployments Frequency Over Time

  1. Click + Widget and select Line Chart.

  2. Title: Deployments Frequency Over Time (add the rocket icon).

  3. Select Service as the Blueprint.

  4. Select Monthly Deployment Frequency as the Property.

  5. Set Time Interval to Month and Time Range to In the past 365 days.

  6. Click Save.

Metric widget groupings

It would be visually cleaner and more informative to group related widgets, such as the Line Chart and Number Chart widgets, side by side for easier comparison. You can replicate more examples by checking our dora metrics dashboard on the demo environment.

Congrats 🎉 You have successfully set up DORA metrics tracking in your portal 🔥