Skip to main content

Check out Port for yourself ➜ 

Set up DORA scorecards

Scorecards let you define maturity levels for your services and teams, giving engineering leaders a clear view of which areas meet delivery standards and which need attention.

This guide builds on the Track DORA metrics guide, which sets up the underlying aggregation and calculation properties. Here, we will create scorecards that automatically grade each service and team against DORA State of DevOps performance tiers for deployment frequency, lead time for changes, change failure rate (CFR), and mean time to recovery (MTTR).

Each metric gets its own scorecard, evaluated at both the service and team level, using progressive maturity levels: Bronze (Medium), Silver (High), and Gold (Elite).

Common use cases​

  • Set clear, measurable delivery and reliability expectations across engineering teams.
  • Identify services that are falling behind on deployment velocity, lead time, or incident response.
  • Compare team delivery maturity at a glance and drive improvement initiatives.
  • Gate deployments or trigger alerts when services drop below a minimum scorecard level.

Steps to set up DORA scorecards​

  1. Review prerequisites: ensure the DORA metrics data model is in place with the required aggregation and calculation properties on your service and team blueprints.
  2. Understand the thresholds: review the DORA performance tiers and how they map to Port scorecard levels.
  3. Create service-level scorecards: add a scorecard for each DORA metric on the service blueprint, grading individual services.
  4. Create team-level scorecards: add the same scorecards on the team blueprint, with deployment frequency normalized per service for fair comparison.
  5. Visualize scorecard results: build a dashboard with pie charts and tables to monitor DORA maturity across the organization.
Service and team level

This guide creates scorecards at two levels:

  • Service level: grades each individual service against DORA benchmarks, giving service owners direct visibility into their delivery performance.
  • Team level: grades teams using the same thresholds, with deployment frequency normalized by the number of services the team owns (so teams of different sizes are compared fairly).

Prerequisites​

This guide assumes the following:

  • You have a Port account and have completed the onboarding process.
  • You have completed the Track DORA metrics guide, which configures the required aggregation and calculation properties on the service and _team blueprints.
  • For CFR and MTTR scorecards, you have an incident integration (e.g., PagerDuty) connected as described in the DORA metrics guide.

Maturity levels​

Port scorecards use four fixed maturity levels. An entity must pass all rules at a given level (and all levels below it) to achieve that level.

LevelDORA TierDescription
BasicLowEntity exists but meets no scorecard criteria
BronzeMediumFoundational: deploys monthly, lead time under a month
SilverHighGood practices: deploys weekly, lead time under a week
GoldEliteExcellence: deploys daily, lead time under a day
Scorecard levels vs. tier labels

Port scorecards use fixed level names (Basic, Bronze, Silver, Gold). The DORA tier calculation properties on your entities use the standard DORA labels (Low, Medium, High, Elite). Both represent the same thresholds as shown in the table above.

DORA thresholds​

Each scorecard uses thresholds aligned with the DORA State of DevOps benchmarks:

MetricBronze (Medium)Silver (High)Gold (Elite)
Deployment Frequency>= 0.25/week (monthly)>= 1/week>= 7/week (daily)
Lead Time for Changes<= 720 hours (1 month)<= 168 hours (1 week)<= 24 hours (1 day)
Change Failure Rate<= 30%<= 20%<= 5%
MTTR<= 43,200 min (1 month)<= 1,440 min (1 day)<= 60 min (1 hour)
MTTR Bronze maps to DORA Low tier

For deployment frequency, lead time, and CFR, the Bronze threshold aligns with the DORA "Medium" tier. For MTTR however, the Bronze threshold of 1 month is more lenient than the DORA "Medium" boundary (1 day – 1 week), so it maps to the "Low" tier instead. This ensures services with infrequent incidents still receive a baseline score rather than being marked as Basic.

Create the scorecards​

Each DORA metric is a separate scorecard so you can see individual metric performance as distinct badges on each entity. Select the level you want to configure:

Deployment frequency

Blueprint: Service

#RuleLevelPropertyCondition
1Deploys at least monthlyBronzedeployment_frequency>= 0.25
2Deploys at least weeklySilverdeployment_frequency>= 1
3Deploys at least dailyGolddeployment_frequency>= 7

Create the deployment frequency scorecard

  1. Go to the Builder page of your portal.

  2. Search for the Service blueprint and select it.

  3. Click on the Scorecards tab.

  4. Click + New Scorecard.

  5. Click on the {...} Edit JSON button in the top right corner.

  6. Paste the following JSON configuration:

    Service Deployment Frequency scorecard (click to expand)
    {
      "identifier": "dora_deploy_freq",
      "title": "Deployment Frequency",
      "levels": [
        {
          "color": "paleBlue",
          "title": "Basic"
        },
        {
          "color": "bronze",
          "title": "Bronze"
        },
        {
          "color": "silver",
          "title": "Silver"
        },
        {
          "color": "gold",
          "title": "Gold"
        }
      ],
      "rules": [
        {
          "identifier": "svc_df_medium",
          "title": "Deploys at least monthly (>= 0.25/week)",
          "description": "DORA Medium tier — service deploys to production at least once per month.",
          "level": "Bronze",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": ">=",
                "property": "deployment_frequency",
                "value": 0.25
              }
            ]
          }
        },
        {
          "identifier": "svc_df_high",
          "title": "Deploys at least weekly (>= 1/week)",
          "description": "DORA High tier — service deploys to production at least once per week.",
          "level": "Silver",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": ">=",
                "property": "deployment_frequency",
                "value": 1
              }
            ]
          }
        },
        {
          "identifier": "svc_df_elite",
          "title": "Deploys at least daily (>= 7/week)",
          "description": "DORA Elite tier — service deploys to production at least once per day.",
          "level": "Gold",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": ">=",
                "property": "deployment_frequency",
                "value": 7
              }
            ]
          }
        }
      ]
    }

  7. Click Save to create the scorecard.

Lead time for changes

Blueprint: Service

#RuleLevelPropertyCondition
1Lead time under 1 monthBronzelead_time_for_changes<= 720 hours
2Lead time under 1 weekSilverlead_time_for_changes<= 168 hours
3Lead time under 1 dayGoldlead_time_for_changes<= 24 hours

Create the lead time for changes scorecard

  1. Go to the Builder page of your portal.

  2. Search for the Service blueprint and select it.

  3. Click on the Scorecards tab.

  4. Click + New Scorecard.

  5. Click on the {...} Edit JSON button in the top right corner. Edit JSON`.

  6. Paste the following JSON configuration:

    Service Lead Time for Changes scorecard (click to expand)
    {
      "identifier": "dora_lead_time",
      "title": "Lead Time for Changes",
      "levels": [
        {
          "color": "paleBlue",
          "title": "Basic"
        },
        {
          "color": "bronze",
          "title": "Bronze"
        },
        {
          "color": "silver",
          "title": "Silver"
        },
        {
          "color": "gold",
          "title": "Gold"
        }
      ],
      "rules": [
        {
          "identifier": "svc_lt_medium",
          "title": "Lead time under 1 month (< 720h)",
          "description": "DORA Medium tier — average lead time from PR creation to deployment is under one month.",
          "level": "Bronze",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "lead_time_for_changes",
                "value": 720
              }
            ]
          }
        },
        {
          "identifier": "svc_lt_high",
          "title": "Lead time under 1 week (< 168h)",
          "description": "DORA High tier — average lead time is under one week.",
          "level": "Silver",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "lead_time_for_changes",
                "value": 168
              }
            ]
          }
        },
        {
          "identifier": "svc_lt_elite",
          "title": "Lead time under 1 day (< 24h)",
          "description": "DORA Elite tier — average lead time is under one day.",
          "level": "Gold",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "lead_time_for_changes",
                "value": 24
              }
            ]
          }
        }
      ]
    }

  7. Click Save to create the scorecard.

Change failure rate (CFR)

Requires incident integration

This scorecard requires an incident integration (e.g., PagerDuty) to be configured. See the Track DORA metrics guide for setup instructions.

Blueprint: Service

#RuleLevelConditions
1CFR <= 30% and has deploymentsBronzechange_failure_rate <= 30 AND total_deployments > 0
2CFR <= 20% and has deploymentsSilverchange_failure_rate <= 20 AND total_deployments > 0
3CFR <= 5% and has deploymentsGoldchange_failure_rate <= 5 AND total_deployments > 0
Why require total_deployments > 0?

Without the total_deployments > 0 condition, services with no deployments would automatically achieve Gold (since 0 incidents / 0 deployments = no failures). The extra condition ensures only services with real deployment activity are evaluated.

Create the change failure rate scorecard

  1. Go to the Builder page of your portal.

  2. Search for the Service blueprint and select it.

  3. Click on the Scorecards tab.

  4. Click + New Scorecard.

  5. Click on the {...} Edit JSON button in the top right corner. Edit JSON`.

  6. Paste the following JSON configuration:

    Service Change Failure Rate scorecard (click to expand)
    {
      "identifier": "dora_cfr",
      "title": "Change Failure Rate",
      "levels": [
        {
          "color": "paleBlue",
          "title": "Basic"
        },
        {
          "color": "bronze",
          "title": "Bronze"
        },
        {
          "color": "silver",
          "title": "Silver"
        },
        {
          "color": "gold",
          "title": "Gold"
        }
      ],
      "rules": [
        {
          "identifier": "svc_cfr_medium",
          "title": "CFR <= 30% (requires deployments)",
          "description": "DORA Medium tier — fewer than 30% of deployments cause incidents.",
          "level": "Bronze",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "change_failure_rate",
                "value": 30
              },
              {
                "operator": ">",
                "property": "total_deployments",
                "value": 0
              }
            ]
          }
        },
        {
          "identifier": "svc_cfr_high",
          "title": "CFR <= 20% (requires deployments)",
          "description": "DORA High tier — fewer than 20% of deployments cause incidents.",
          "level": "Silver",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "change_failure_rate",
                "value": 20
              },
              {
                "operator": ">",
                "property": "total_deployments",
                "value": 0
              }
            ]
          }
        },
        {
          "identifier": "svc_cfr_elite",
          "title": "CFR <= 5% (requires deployments)",
          "description": "DORA Elite tier — fewer than 5% of deployments cause incidents.",
          "level": "Gold",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "change_failure_rate",
                "value": 5
              },
              {
                "operator": ">",
                "property": "total_deployments",
                "value": 0
              }
            ]
          }
        }
      ]
    }

  7. Click Save to create the scorecard.

Mean time to recovery (MTTR)

Requires incident integration

This scorecard requires an incident integration (e.g., PagerDuty) to be configured. See the Track DORA metrics guide for setup instructions.

Blueprint: Service

#RuleLevelConditions
1MTTR under 1 month and has incidentsBronzemean_time_to_recovery <= 43200 AND total_incidents > 0
2MTTR under 1 day and has incidentsSilvermean_time_to_recovery <= 1440 AND total_incidents > 0
3MTTR under 1 hour and has incidentsGoldmean_time_to_recovery <= 60 AND total_incidents > 0
Why require total_incidents > 0?

Without the total_incidents > 0 condition, services with no incidents would automatically achieve Gold (since there's nothing to recover from). The extra condition ensures only services with real incident activity are evaluated.

Create the MTTR scorecard

  1. Go to the Builder page of your portal.

  2. Search for the Service blueprint and select it.

  3. Click on the Scorecards tab.

  4. Click + New Scorecard.

  5. Click on the {...} Edit JSON button in the top right corner. Edit JSON`.

  6. Paste the following JSON configuration:

    Service MTTR scorecard (click to expand)
    {
      "identifier": "dora_mttr",
      "title": "MTTR",
      "levels": [
        {
          "color": "paleBlue",
          "title": "Basic"
        },
        {
          "color": "bronze",
          "title": "Bronze"
        },
        {
          "color": "silver",
          "title": "Silver"
        },
        {
          "color": "gold",
          "title": "Gold"
        }
      ],
      "rules": [
        {
          "identifier": "svc_mttr_low",
          "title": "MTTR under 1 month (< 43200 min, requires incidents)",
          "description": "DORA Low tier — incidents are resolved within one month on average.",
          "level": "Bronze",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "mean_time_to_recovery",
                "value": 43200
              },
              {
                "operator": ">",
                "property": "total_incidents",
                "value": 0
              }
            ]
          }
        },
        {
          "identifier": "svc_mttr_high",
          "title": "MTTR under 1 day (< 1440 min, requires incidents)",
          "description": "DORA High tier — incidents are resolved within one day on average.",
          "level": "Silver",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "mean_time_to_recovery",
                "value": 1440
              },
              {
                "operator": ">",
                "property": "total_incidents",
                "value": 0
              }
            ]
          }
        },
        {
          "identifier": "svc_mttr_elite",
          "title": "MTTR under 1 hour (< 60 min, requires incidents)",
          "description": "DORA Elite tier — incidents are resolved within one hour on average.",
          "level": "Gold",
          "query": {
            "combinator": "and",
            "conditions": [
              {
                "operator": "<=",
                "property": "mean_time_to_recovery",
                "value": 60
              },
              {
                "operator": ">",
                "property": "total_incidents",
                "value": 0
              }
            ]
          }
        }
      ]
    }

  7. Click Save to create the scorecard.

What you should see: After creating all four service-level scorecards, each service entity will display four scorecard badges, one for each DORA metric. Services with sufficient data will show Bronze, Silver, or Gold based on their performance. Services without deployment or incident data will show Basic.

Visualize scorecard results​

Once the scorecards are in place, create a dashboard to monitor DORA maturity across services and teams.

Create the dashboard​

  1. Navigate to your software catalog.
  2. Click on the + button in the left sidebar.
  3. Select New folder.
  4. Name the folder Engineering Intelligence and click Create. If the folder already exists, skip this step.
  5. Inside the Engineering Intelligence folder, click + New again.
  6. Select New dashboard.
  7. Name the dashboard DORA Scorecards and click Create.

Add widgets​

You can populate the dashboard using either an API script or by manually creating each widget through the UI.

The fastest way to set up the dashboard is by using Port's API to create all widgets at once.

Get your Port API token

  1. In your Port portal, click on your profile picture in the top right corner.

  2. Select Credentials.

  3. Click Generate API token.

  4. Copy the generated token and store it as an environment variable:

    export PORT_ACCESS_TOKEN="YOUR_GENERATED_TOKEN"
EU region

If your portal is hosted in the EU region, replace api.port.io with api.port-eu.io in the dashboard creation command below.

Create the dashboard with widgets

Save the following JSON to a file named dora_scorecards_dashboard.json:

Dashboard JSON payload (click to expand)
{
  "identifier": "dora_scorecards",
  "title": "DORA Scorecards",
  "icon": "Star",
  "type": "dashboard",
  "parent": "engineering_intelligence",
  "widgets": [
    {
      "id": "doraScorecardsDashboardWidget",
      "type": "dashboard-widget",
      "layout": [
        {
          "height": 400,
          "columns": [
            {"id": "svcDeployFreqDistribution", "size": 3},
            {"id": "svcLeadTimeDistribution", "size": 3},
            {"id": "svcCfrDistribution", "size": 3},
            {"id": "svcMttrDistribution", "size": 3}
          ]
        },
        {
          "height": 400,
          "columns": [
            {"id": "teamDeployFreqDistribution", "size": 3},
            {"id": "teamLeadTimeDistribution", "size": 3},
            {"id": "teamCfrDistribution", "size": 3},
            {"id": "teamMttrDistribution", "size": 3}
          ]
        },
        {
          "height": 400,
          "columns": [
            {"id": "serviceScorecardsTable", "size": 12}
          ]
        },
        {
          "height": 400,
          "columns": [
            {"id": "teamScorecardsTable", "size": 12}
          ]
        }
      ],
      "widgets": [
        {
          "id": "svcDeployFreqDistribution",
          "type": "entities-pie-chart",
          "title": "Service — Deployment Frequency",
          "icon": "Pie",
          "description": "Distribution of deployment frequency scorecard levels across services",
          "blueprint": "service",
          "property": "scorecard#dora_deploy_freq",
          "dataset": {
            "combinator": "and",
            "rules": []
          }
        },
        {
          "id": "svcLeadTimeDistribution",
          "type": "entities-pie-chart",
          "title": "Service — Lead Time",
          "icon": "Pie",
          "description": "Distribution of lead time scorecard levels across services",
          "blueprint": "service",
          "property": "scorecard#dora_lead_time",
          "dataset": {
            "combinator": "and",
            "rules": []
          }
        },
        {
          "id": "svcCfrDistribution",
          "type": "entities-pie-chart",
          "title": "Service — CFR",
          "icon": "Pie",
          "description": "Distribution of change failure rate scorecard levels across services",
          "blueprint": "service",
          "property": "scorecard#dora_cfr",
          "dataset": {
            "combinator": "and",
            "rules": []
          }
        },
        {
          "id": "svcMttrDistribution",
          "type": "entities-pie-chart",
          "title": "Service — MTTR",
          "icon": "Pie",
          "description": "Distribution of MTTR scorecard levels across services",
          "blueprint": "service",
          "property": "scorecard#dora_mttr",
          "dataset": {
            "combinator": "and",
            "rules": []
          }
        },
        {
          "id": "teamDeployFreqDistribution",
          "type": "entities-pie-chart",
          "title": "Team — Deployment Frequency",
          "icon": "Pie",
          "description": "Distribution of deployment frequency scorecard levels across teams",
          "blueprint": "_team",
          "property": "scorecard#team_dora_deploy_freq",
          "dataset": {
            "combinator": "and",
            "rules": [
              {"property": "type", "operator": "=", "value": "team"}
            ]
          }
        },
        {
          "id": "teamLeadTimeDistribution",
          "type": "entities-pie-chart",
          "title": "Team — Lead Time",
          "icon": "Pie",
          "description": "Distribution of lead time scorecard levels across teams",
          "blueprint": "_team",
          "property": "scorecard#team_dora_lead_time",
          "dataset": {
            "combinator": "and",
            "rules": [
              {"property": "type", "operator": "=", "value": "team"}
            ]
          }
        },
        {
          "id": "teamCfrDistribution",
          "type": "entities-pie-chart",
          "title": "Team — CFR",
          "icon": "Pie",
          "description": "Distribution of change failure rate scorecard levels across teams",
          "blueprint": "_team",
          "property": "scorecard#team_dora_cfr",
          "dataset": {
            "combinator": "and",
            "rules": [
              {"property": "type", "operator": "=", "value": "team"}
            ]
          }
        },
        {
          "id": "teamMttrDistribution",
          "type": "entities-pie-chart",
          "title": "Team — MTTR",
          "icon": "Pie",
          "description": "Distribution of MTTR scorecard levels across teams",
          "blueprint": "_team",
          "property": "scorecard#team_dora_mttr",
          "dataset": {
            "combinator": "and",
            "rules": [
              {"property": "type", "operator": "=", "value": "team"}
            ]
          }
        },
        {
          "id": "serviceScorecardsTable",
          "type": "table-entities-explorer",
          "displayMode": "widget",
          "title": "Service — DORA Scorecard Levels",
          "icon": "Table",
          "description": "DORA scorecard levels per service",
          "blueprint": "service",
          "dataset": {"combinator": "and", "rules": []},
          "excludedFields": [],
          "blueprintConfig": {
            "service": {
              "groupSettings": {"groupBy": ["team"]},
              "propertiesSettings": {
                "order": ["$title", "team", "scorecard#dora_deploy_freq", "deploy_freq_tier", "scorecard#dora_lead_time", "lead_time_tier", "scorecard#dora_cfr", "cfr_tier", "scorecard#dora_mttr", "mttr_tier"],
                "shown": ["$title", "team", "scorecard#dora_deploy_freq", "deploy_freq_tier", "scorecard#dora_lead_time", "lead_time_tier", "scorecard#dora_cfr", "cfr_tier", "scorecard#dora_mttr", "mttr_tier"]
              },
              "filterSettings": {"filterBy": {"combinator": "and", "rules": []}},
              "sortSettings": {"sortBy": [{"property": "scorecard#dora_deploy_freq", "order": "desc"}]}
            }
          }
        },
        {
          "id": "teamScorecardsTable",
          "type": "table-entities-explorer",
          "displayMode": "widget",
          "title": "Team — DORA Scorecard Levels",
          "icon": "Table",
          "description": "DORA scorecard levels per team",
          "blueprint": "_team",
          "dataset": {
            "combinator": "and",
            "rules": [
              {"property": "type", "operator": "=", "value": "team"}
            ]
          },
          "excludedFields": [],
          "blueprintConfig": {
            "_team": {
              "groupSettings": {"groupBy": []},
              "propertiesSettings": {
                "order": ["$title", "scorecard#team_dora_deploy_freq", "deploy_freq_tier", "scorecard#team_dora_lead_time", "lead_time_tier", "scorecard#team_dora_cfr", "cfr_tier", "scorecard#team_dora_mttr", "mttr_tier"],
                "shown": ["$title", "scorecard#team_dora_deploy_freq", "deploy_freq_tier", "scorecard#team_dora_lead_time", "lead_time_tier", "scorecard#team_dora_cfr", "cfr_tier", "scorecard#team_dora_mttr", "mttr_tier"]
              },
              "filterSettings": {"filterBy": {"combinator": "and", "rules": []}},
              "sortSettings": {"sortBy": [{"property": "scorecard#team_dora_deploy_freq", "order": "desc"}]}
            }
          }
        }
      ]
    }
  ]
}

Then run the following command to create the dashboard with all widgets:

curl -s -X POST "https://api.port.io/v1/pages" \
  -H "Authorization: Bearer $PORT_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d @dora_scorecards_dashboard.json | python3 -m json.tool
Engineering Intelligence folder

The script assumes an engineering_intelligence folder already exists in your catalog. If you haven't created it yet, follow steps 1-4 in the create the dashboard section first.

What you should see: The completed DORA Scorecards dashboard displays pie charts showing the distribution of scorecard levels across services and teams, followed by table widgets listing each entity with its DORA scorecard grades and tier labels side by side.

Design decisions​

Per-metric scorecards vs. combined scorecard​

Unlike the Delivery Performance scorecard which combines multiple PR metrics into a single scorecard, DORA scorecards are split into four separate scorecards (one per metric). This is intentional:

  • Independent evaluation: A service can be Elite at deployment frequency but Low at lead time. A combined scorecard would mask this by averaging across metrics.
  • Targeted improvement: Teams can focus on the specific metric that needs attention without being penalized by other metrics.
  • Conditional metrics: CFR and MTTR require an incident integration. Separate scorecards allow you to start with deployment frequency and lead time, then add CFR and MTTR when incidents are configured.

Team-level normalization​

Team deployment frequency uses a per-service normalized metric (deployment_frequency_per_service). This divides the team's total deployment frequency by the number of services it owns, so a team with 10 services deploying 70 times per week is evaluated the same as a team with 1 service deploying 7 times per week both achieve Gold.

Lead time, CFR, and MTTR thresholds are unchanged at team level since they already represent averages that scale naturally.

Guard conditions on CFR and MTTR​

CFR rules require total_deployments > 0 and MTTR rules require total_incidents > 0. Without these conditions, entities with no data would trivially pass all rules and achieve Gold which would be misleading. The guard conditions ensure only entities with real activity are evaluated.

Threshold sources​

Thresholds are aligned with the DORA State of DevOps performance tiers as published by Google Cloud's DevOps Research and Assessment program.

Next steps​

  • Add automations: Use Port automations to send Slack notifications when a service drops below Bronze on any DORA metric, or create Jira tickets for teams with consistently Low deployment frequency.
  • Customize thresholds: Adjust the numeric values in the scorecard rules to match your organization's delivery standards. For example, if daily deployments aren't realistic for your team, lower the Gold threshold from 7/week to 5/week.
  • Combine with delivery performance scorecards: Use both DORA and delivery performance scorecards together for a comprehensive view of engineering maturity.