Set up pipeline reliability scorecards

Implement with AI

Send this guide to your coding agent.

Prerequisite: Install Port MCP

Open in ClaudeOpen in CodexOpen in Cursor

Open plan mode if your tool supports it; otherwise present the plan below filled in and wait for my approval. Implement this Port guide in my org via MCP:

https://docs.port.io/guides/all/setup-pipeline-reliability-scorecard

Read the raw markdown version at https://docs.port.io/guides/all/setup-pipeline-reliability-scorecard.md - it contains every tab and code block without page markup.

Goal: get the guide's core flow working end-to-end in my org; adapting it to fit my existing setup takes priority over matching the guide 1:1.

Plan:
1. Confirm MCP is connected, in the right org, with sufficient permissions.
2. If the guide offers alternative implementation paths (tabs), pick the one matching my installed integrations and tools, confirm it with me, and implement only that path.
3. Diff the guide's data model (blueprints, properties, relations, workflows, actions, agents, automations, integrations, webhook data sources, secrets) against mine.
4. Propose adaptations for gaps, reusing existing blueprints/relations over guide-named duplicates.
5. Flag what needs a UI click, credential, or secret from me, testing MCP capability empirically before ruling anything out. If the guide has a "Set up via API" section, use it for anything MCP can't do before treating a step as UI-only.
6. Stop on any blocker and give me options. Approving this plan authorizes the writes it lists; pause only for writes beyond what's listed.

Build:
- Extend blueprint schema additively when upserting; don't remove or overwrite existing properties, and treat type conflicts as a blocker, not an auto-fix.
- Never print secret values into the chat or logs; ask me to set them in Port, or write them via the secrets API without echoing them back.
- List any mock data in the plan, minimal and labeled mock; once approved, seed it without re-asking, and tell me what you seeded.
- For anything the guide writes downstream (e.g. a webhook target), use a real entity, not a mock.
- For pages/widgets, use the real page identifier from the app URL, not a guessed slug.
- When you hit a UI step confirmed (not assumed) unsupported via MCP and not covered by the guide's API sections, pause, give exact clicks, then resume via MCP.
- Validate and give links after each meaningful step (only a tool-returned URL, no guessed paths); don't proceed if the last run wasn't a success.

Done:
- Run the guide's "Let's test it" steps where possible (e.g. execute a workflow test run) and confirm the expected output exists in Port.
- Summarize adaptations, seeded data, what was mocked or skipped, remaining UI steps, and how to verify.

Included by default

If you signed up for Port on or after May 1, 2026, these scorecards are already set up in Port. You can follow this guide to customize the thresholds and rules to match your organization's standards.

Scorecards let you define maturity levels for your services and teams, giving engineering leaders a clear view of which areas meet reliability standards and which need attention. This guide walks you through setting up Pipeline Reliability scorecards using data from GitHub, GitLab, or Azure DevOps integrations, at different levels of the organization:

• Service: Workflow failure rates and reliability trends for individual services.
• Team: Same metrics aggregated across all services owned by each team.
• Group: Executive-level reliability health for business units.
• Organization: Top-level reliability view across the entire org.

Each scorecard evaluates entities against progressive maturity levels: Bronze, Silver, and Gold.

Common use cases

• Set clear, measurable reliability expectations across engineering teams.
• Identify services with persistently high CI/CD failure rates before they impact delivery.
• Spot reliability trends early to catch pipelines that are degrading before they become critical.
• Give developers a self-service view of their pipeline health against organization-wide standards.

Prerequisites

This guide assumes the following:

• You have a Port account and have completed the onboarding process.
• You have completed the Create foundational Engineering Intelligence data model guide, which provisions the core Organization, Team, and Service blueprints (with team ownership on services) for GitHub, GitLab, and Azure DevOps.
• You have completed the Measure pipeline reliability guide, which configures the required pipeline failure rate properties on the Service, Team, and Organization blueprints.

GitHub (Ocean)

• You have installed the GitHub (Ocean) integration. The prerequisite guide sets up github_workflow_runs_30d, github_monthly_workflow_failure_rate, and github_failure_rate_trend on the service blueprint.

GitLab

• You have installed the GitLab integration. The prerequisite guide sets up gitlab_pipeline_jobs_30d, gitlab_monthly_pipeline_job_failure_rate, and gitlab_pipeline_job_failure_rate_trend on the service blueprint.

Azure DevOps

• You have installed the Azure DevOps integration. The prerequisite guide sets up ado_pipeline_builds_30d, ado_monthly_pipeline_build_failure_rate, and ado_pipeline_build_failure_rate_trend on the service blueprint as aggregation and calculation properties sourced from the azureDevopsBuild blueprint.

Branch and time scope

The example mappings used for reliability metrics often scope data to main, master, or production branches and the last 90 days so the scorecards focus on production-relevant CI/CD health. Exact defaults can vary by provider and by your current mapping.

To change the tracked branches or time window, go to Data Sources, open your GitHub, GitLab, or Azure DevOps integration, select the Mapping tab, and locate the reliability entity resource (githubWorkflowRun, gitlabJob, or azureDevopsBuild). Edit the selector.query field.

For example, this GitHub selector also includes develop:

selector:
  query: >-
    (.head_branch | IN("main", "master", "production", "develop")) and
    ((.created_at | fromdateiso8601) > (now - 7776000))

Add or remove branch names in the IN(...) list as needed. The 7776000 value is 90 days in seconds, change it to 2592000 for 30 days.

Maturity levels

Port scorecards use four fixed maturity levels. An entity must pass all rules at a given level (and all levels below it) to achieve that level.

Level	Progression	Description
Basic	0 (default)	Entity exists but meets no scorecard criteria
Bronze	1	Foundational hygiene - CI/CD is running and failure rate is below 30%
Silver	2	Good practices - failure rate under 15% and not trending worse
Gold	3	Excellence - failure rate under 5%

Create the scorecards

Select the level you want to configure:

Service

Blueprint: Service

GitHub (Ocean)

#	Rule	Level	Property	Condition
1	Has workflow runs this month	Bronze	github_workflow_runs_30d	> 0
2	Workflow failure rate < 30%	Bronze	github_monthly_workflow_failure_rate	< 30
3	Workflow failure rate < 15%	Silver	github_monthly_workflow_failure_rate	< 15
4	Failure rate not degrading	Silver	github_failure_rate_trend	!= "Degrading"
5	Workflow failure rate < 5%	Gold	github_monthly_workflow_failure_rate	< 5

Create the reliability health scorecard on the service blueprint

1. Go to the Builder page in Port.

2. Search for the Service blueprint and select it.

3. Click on the Scorecards tab.

4. If the reliability_health scorecard does not exist, click + New Scorecard. If it already exists, open it instead.

5. Click on the {...} Edit JSON button in the top right corner.

6. Paste the following JSON configuration:

   Service Reliability Health scorecard - GitHub (click to expand)

   {
     "identifier": "reliability_health",
     "title": "Reliability Health",
     "levels": [
       {
         "color": "paleBlue",
         "title": "Basic"
       },
       {
         "color": "bronze",
         "title": "Bronze"
       },
       {
         "color": "silver",
         "title": "Silver"
       },
       {
         "color": "gold",
         "title": "Gold"
       }
     ],
     "rules": [
       {
         "identifier": "github_has_workflow_runs",
         "title": "Has workflow runs this month",
         "description": "Service has at least one CI/CD workflow run in the last 30 days, confirming pipelines are active.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": ">",
               "property": "github_workflow_runs_30d",
               "value": 0
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_30",
         "title": "Workflow failure rate < 30%",
         "description": "Monthly CI/CD workflow failure rate is below 30%.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "github_monthly_workflow_failure_rate",
               "value": 30
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_15",
         "title": "Workflow failure rate < 15%",
         "description": "Monthly CI/CD workflow failure rate is below 15%.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "github_monthly_workflow_failure_rate",
               "value": 15
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_not_degrading",
         "title": "Failure rate not degrading",
         "description": "Weekly failure rate is not worsening compared to the monthly average.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "!=",
               "property": "github_failure_rate_trend",
               "value": "Degrading"
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_5",
         "title": "Workflow failure rate < 5%",
         "description": "Monthly CI/CD workflow failure rate is below 5%.",
         "level": "Gold",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "github_monthly_workflow_failure_rate",
               "value": 5
             }
           ]
         }
       }
     ]
   }

7. Click Save to create the scorecard.

Property availability

github_workflow_runs_30d, github_failed_workflow_runs_30d, github_monthly_workflow_failure_rate, and github_failure_rate_trend are aggregation and calculation properties configured by the reliability metrics setup for the GitHub integration. Newly ingested entities may show Basic until the integration completes its first sync. Once sync finishes, scorecard levels are recomputed automatically.

GitLab

#	Rule	Level	Property	Condition
1	Has pipeline jobs this month	Bronze	gitlab_pipeline_jobs_30d	> 0
2	Pipeline job failure rate < 30%	Bronze	gitlab_monthly_pipeline_job_failure_rate	< 30
3	Pipeline job failure rate < 15%	Silver	gitlab_monthly_pipeline_job_failure_rate	< 15
4	Failure rate not degrading	Silver	gitlab_pipeline_job_failure_rate_trend	!= "Degrading"
5	Pipeline job failure rate < 5%	Gold	gitlab_monthly_pipeline_job_failure_rate	< 5

Add GitLab rules to the reliability health scorecard on the service blueprint

1. Go to the Builder page in Port.

2. Search for the Service blueprint and select it.

3. Click on the Scorecards tab.

4. Open the existing reliability_health scorecard.

5. Click on the {...} Edit JSON button in the top right corner.

6. Append the following rules to the rules array:

   GitLab rules to append to the reliability_health scorecard (click to expand)

   [
     {
       "identifier": "gitlab_has_pipeline_jobs",
       "title": "Has pipeline jobs this month",
       "description": "Service has at least one CI/CD pipeline job in the last 30 days, confirming pipelines are active.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": ">",
             "property": "gitlab_pipeline_jobs_30d",
             "value": 0
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_30",
       "title": "Pipeline job failure rate < 30%",
       "description": "Monthly CI/CD pipeline job failure rate is below 30%.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "gitlab_monthly_pipeline_job_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_15",
       "title": "Pipeline job failure rate < 15%",
       "description": "Monthly CI/CD pipeline job failure rate is below 15%.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "gitlab_monthly_pipeline_job_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_not_degrading",
       "title": "Failure rate not degrading",
       "description": "Weekly pipeline job failure rate is not worsening compared to the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "gitlab_pipeline_job_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_5",
       "title": "Pipeline job failure rate < 5%",
       "description": "Monthly CI/CD pipeline job failure rate is below 5%.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "gitlab_monthly_pipeline_job_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

7. Click Save to update the scorecard.

Property availability

gitlab_pipeline_jobs_30d, gitlab_monthly_pipeline_job_failure_rate, and gitlab_pipeline_job_failure_rate_trend are aggregation and calculation properties configured by the reliability metrics setup for the GitLab integration. Newly ingested entities may show Basic until the integration completes its first sync. Once sync finishes, scorecard levels are recomputed automatically.

Azure DevOps

#	Rule	Level	Property	Condition
1	Has pipeline builds this month	Bronze	ado_pipeline_builds_30d	> 0
2	Build failure rate < 30%	Bronze	ado_monthly_pipeline_build_failure_rate	< 30
3	Build failure rate < 15%	Silver	ado_monthly_pipeline_build_failure_rate	< 15
4	Build failure rate not degrading	Silver	ado_pipeline_build_failure_rate_trend	!= "Degrading"
5	Build failure rate < 5%	Gold	ado_monthly_pipeline_build_failure_rate	< 5

Add Azure DevOps rules to the reliability health scorecard on the service blueprint

1. Go to the Builder page in Port.

2. Search for the Service blueprint and select it.

3. Click on the Scorecards tab.

4. Open the existing reliability_health scorecard.

5. Click on the {...} Edit JSON button in the top right corner.

6. Append the following rules to the rules array:

   Azure DevOps rules to append to the reliability_health scorecard (click to expand)

   [
     {
       "identifier": "ado_has_pipeline_builds",
       "title": "Has pipeline builds this month",
       "description": "Service has at least one Azure DevOps build in the last 30 days, confirming CI/CD pipelines are active.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": ">",
             "property": "ado_pipeline_builds_30d",
             "value": 0
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_30",
       "title": "Build failure rate < 30%",
       "description": "Less than 30% of pipeline builds failed in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "ado_monthly_pipeline_build_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_15",
       "title": "Build failure rate < 15%",
       "description": "Less than 15% of pipeline builds failed in the last 30 days.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "ado_monthly_pipeline_build_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_not_degrading",
       "title": "Build failure rate not degrading",
       "description": "Weekly build failure rate is not higher than the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "ado_pipeline_build_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_5",
       "title": "Build failure rate < 5%",
       "description": "Less than 5% of pipeline builds failed in the last 30 days - elite reliability.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "ado_monthly_pipeline_build_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

7. Click Save to update the scorecard.

Property availability

ado_pipeline_builds_30d, ado_failed_pipeline_builds_30d, ado_monthly_pipeline_build_failure_rate, and ado_pipeline_build_failure_rate_trend are aggregation and calculation properties configured by the reliability metrics setup for the Azure DevOps integration. They are sourced from the azureDevopsBuild blueprint. Newly ingested entities may show Basic until the integration completes its first sync. Once sync finishes, scorecard levels are recomputed automatically.

Team

Blueprint: Team

Same thresholds as service-level, but metrics are aggregated across all services owned by the team.

GitHub (Ocean)

#	Rule	Level	Property	Condition
1	Has workflow runs this month	Bronze	github_workflow_runs_30d	> 0
2	Workflow failure rate < 30%	Bronze	monthly_workflow_failure_rate	< 30
3	Workflow failure rate < 15%	Silver	monthly_workflow_failure_rate	< 15
4	Workflow failure rate not degrading	Silver	failure_rate_trend	!= "Degrading"
5	Workflow failure rate < 5%	Gold	monthly_workflow_failure_rate	< 5

GitLab

#	Rule	Level	Property	Condition
1	Has pipeline jobs this month	Bronze	gitlab_pipeline_jobs_30d	> 0
2	Pipeline job failure rate < 30%	Bronze	monthly_pipeline_job_failure_rate	< 30
3	Pipeline job failure rate < 15%	Silver	monthly_pipeline_job_failure_rate	< 15
4	Failure rate not degrading	Silver	pipeline_job_failure_rate_trend	!= "Degrading"
5	Pipeline job failure rate < 5%	Gold	monthly_pipeline_job_failure_rate	< 5

Azure DevOps

#	Rule	Level	Property	Condition
1	Has pipeline builds this month	Bronze	ado_pipeline_builds_30d	> 0
2	Build failure rate < 30%	Bronze	monthly_pipeline_build_failure_rate	< 30
3	Build failure rate < 15%	Silver	monthly_pipeline_build_failure_rate	< 15
4	Build failure rate not degrading	Silver	pipeline_build_failure_rate_trend	!= "Degrading"
5	Build failure rate < 5%	Gold	monthly_pipeline_build_failure_rate	< 5

Configure the reliability health scorecard on the team blueprint

Property naming at team level

Team-level calculation properties use no provider prefix for the aggregated failure rate metrics. The same concept has a different name depending on the level:

• Service-level (stored directly from integration): github_monthly_workflow_failure_rate, gitlab_monthly_pipeline_job_failure_rate, ado_monthly_pipeline_build_failure_rate
• Team-level (aggregated across services): monthly_workflow_failure_rate, monthly_pipeline_job_failure_rate, monthly_pipeline_build_failure_rate

This applies to trend and run-count properties as well. The count property github_workflow_runs_30d and gitlab_pipeline_jobs_30d retain their provider prefix at the team level since they are aggregation properties sourced directly from the integration.

1. Go to the Builder page.
2. Search for the Team blueprint and select it.
3. Click on the Scorecards tab.
4. Open the existing team_reliability_health scorecard.
5. Click on the {...} Edit JSON button in the top right corner.

GitHub (Ocean)

6. Paste the following full scorecard JSON:

   Team Reliability Health scorecard - GitHub (click to expand)

   {
     "identifier": "team_reliability_health",
     "title": "Reliability Health",
     "filter": {
       "combinator": "and",
       "conditions": [
         {
           "operator": "=",
           "property": "type",
           "value": "team"
         }
       ]
     },
     "levels": [
       {
         "color": "paleBlue",
         "title": "Basic"
       },
       {
         "color": "bronze",
         "title": "Bronze"
       },
       {
         "color": "silver",
         "title": "Silver"
       },
       {
         "color": "gold",
         "title": "Gold"
       }
     ],
     "rules": [
       {
         "identifier": "github_has_workflow_runs",
         "title": "Has workflow runs this month",
         "description": "Team has at least one CI/CD workflow run across its services in the last 30 days.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": ">",
               "property": "github_workflow_runs_30d",
               "value": 0
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_30",
         "title": "Workflow failure rate < 30%",
         "description": "Team's aggregate monthly CI/CD failure rate is below 30%.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 30
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_15",
         "title": "Workflow failure rate < 15%",
         "description": "Team's aggregate monthly CI/CD failure rate is below 15%.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 15
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_not_degrading",
         "title": "Workflow failure rate not degrading",
         "description": "Team's weekly failure rate is not worsening compared to the monthly average.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "!=",
               "property": "failure_rate_trend",
               "value": "Degrading"
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_5",
         "title": "Workflow failure rate < 5%",
         "description": "Team's aggregate monthly CI/CD failure rate is below 5%, indicating elite performance.",
         "level": "Gold",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 5
             }
           ]
         }
       }
     ]
   }

GitLab

6. Append the following rules to the rules array:

   GitLab rules to append to the team_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "gitlab_has_pipeline_jobs",
       "title": "Has pipeline jobs this month",
       "description": "Team has at least one CI/CD pipeline job across its services in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": ">",
             "property": "gitlab_pipeline_jobs_30d",
             "value": 0
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_30",
       "title": "Pipeline job failure rate < 30%",
       "description": "Team's aggregate monthly pipeline job failure rate is below 30%.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_15",
       "title": "Pipeline job failure rate < 15%",
       "description": "Team's aggregate monthly pipeline job failure rate is below 15%.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_not_degrading",
       "title": "Failure rate not degrading",
       "description": "Team's weekly failure rate is not worsening compared to the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_job_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_5",
       "title": "Pipeline job failure rate < 5%",
       "description": "Team's aggregate monthly pipeline job failure rate is below 5%, indicating elite performance.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

Azure DevOps

6. Append the following rules to the rules array:

   Azure DevOps rules to append to the team_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "ado_has_pipeline_builds",
       "title": "Has pipeline builds this month",
       "description": "Team has at least one Azure DevOps build across its services in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": ">",
             "property": "ado_pipeline_builds_30d",
             "value": 0
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_30",
       "title": "Build failure rate < 30%",
       "description": "Less than 30% of pipeline builds failed across team services in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_15",
       "title": "Build failure rate < 15%",
       "description": "Less than 15% of pipeline builds failed across team services.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_not_degrading",
       "title": "Build failure rate not degrading",
       "description": "Team's weekly build failure rate is not higher than the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_build_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_5",
       "title": "Build failure rate < 5%",
       "description": "Less than 5% of pipeline builds failed across team services - elite reliability.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

7. Click Save to update the scorecard.

Group

Blueprint: Team (filtered to type = "group")

Executive-level reliability health for business units. The has_pipeline_builds rule is omitted at this level since groups aggregate across many teams and always have some pipeline activity.

GitHub (Ocean)

#	Rule	Level	Property	Condition
1	Workflow failure rate < 30%	Bronze	monthly_workflow_failure_rate	< 30
2	Workflow failure rate < 15%	Silver	monthly_workflow_failure_rate	< 15
3	Workflow failure rate not degrading	Silver	failure_rate_trend	!= "Degrading"
4	Workflow failure rate < 5%	Gold	monthly_workflow_failure_rate	< 5

GitLab

#	Rule	Level	Property	Condition
1	Pipeline job failure rate < 30%	Bronze	monthly_pipeline_job_failure_rate	< 30
2	Pipeline job failure rate < 15%	Silver	monthly_pipeline_job_failure_rate	< 15
3	Failure rate not degrading	Silver	pipeline_job_failure_rate_trend	!= "Degrading"
4	Pipeline job failure rate < 5%	Gold	monthly_pipeline_job_failure_rate	< 5

Azure DevOps

#	Rule	Level	Property	Condition
1	Build failure rate < 30%	Bronze	monthly_pipeline_build_failure_rate	< 30
2	Build failure rate < 15%	Silver	monthly_pipeline_build_failure_rate	< 15
3	Build failure rate not degrading	Silver	pipeline_build_failure_rate_trend	!= "Degrading"
4	Build failure rate < 5%	Gold	monthly_pipeline_build_failure_rate	< 5

Configure the reliability health scorecard for groups

1. Go to the Builder page.
2. Search for the Team blueprint and select it.
3. Click on the Scorecards tab.
4. Open the existing group_reliability_health scorecard.
5. Click on the {...} Edit JSON button in the top right corner.

GitHub (Ocean)

6. Paste the following full scorecard JSON:

   Group Reliability Health scorecard - GitHub (click to expand)

   {
     "identifier": "group_reliability_health",
     "title": "Reliability Health (Group)",
     "filter": {
       "combinator": "and",
       "conditions": [
         {
           "operator": "=",
           "property": "type",
           "value": "group"
         }
       ]
     },
     "levels": [
       {
         "color": "paleBlue",
         "title": "Basic"
       },
       {
         "color": "bronze",
         "title": "Bronze"
       },
       {
         "color": "silver",
         "title": "Silver"
       },
       {
         "color": "gold",
         "title": "Gold"
       }
     ],
     "rules": [
       {
         "identifier": "github_failure_rate_under_30",
         "title": "Workflow failure rate < 30%",
         "description": "Group-wide aggregate monthly CI/CD failure rate is below 30%.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 30
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_15",
         "title": "Workflow failure rate < 15%",
         "description": "Group-wide aggregate monthly CI/CD failure rate is below 15%.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 15
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_not_degrading",
         "title": "Workflow failure rate not degrading",
         "description": "Group's weekly failure rate is not worsening compared to the monthly average.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "!=",
               "property": "failure_rate_trend",
               "value": "Degrading"
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_5",
         "title": "Workflow failure rate < 5%",
         "description": "Group-wide aggregate monthly CI/CD failure rate is below 5%, indicating elite performance.",
         "level": "Gold",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 5
             }
           ]
         }
       }
     ]
   }

GitLab

6. Append the following rules to the rules array:

   GitLab rules to append to the group_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "gitlab_failure_rate_under_30",
       "title": "Pipeline job failure rate < 30%",
       "description": "Group-wide aggregate monthly pipeline job failure rate is below 30%.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_15",
       "title": "Pipeline job failure rate < 15%",
       "description": "Group-wide aggregate monthly pipeline job failure rate is below 15%.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_not_degrading",
       "title": "Failure rate not degrading",
       "description": "Group's weekly pipeline job failure rate is not worsening compared to the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_job_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_5",
       "title": "Pipeline job failure rate < 5%",
       "description": "Group-wide aggregate monthly pipeline job failure rate is below 5%, indicating elite performance.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

Azure DevOps

6. Append the following rules to the rules array:

   Azure DevOps rules to append to the group_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "ado_failure_rate_under_30",
       "title": "Build failure rate < 30%",
       "description": "Less than 30% of pipeline builds failed across the group's services in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_15",
       "title": "Build failure rate < 15%",
       "description": "Less than 15% of pipeline builds failed across the group.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_not_degrading",
       "title": "Build failure rate not degrading",
       "description": "Group's weekly build failure rate is not higher than the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_build_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_5",
       "title": "Build failure rate < 5%",
       "description": "Less than 5% of pipeline builds failed across the group - elite reliability.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

7. Click Save to update the scorecard.

Organization

Blueprint: Organization

Executive-level reliability health for the entire organization. Same failure rate thresholds as group level.

GitHub (Ocean)

#	Rule	Level	Property	Condition
1	Workflow failure rate < 30%	Bronze	monthly_workflow_failure_rate	< 30
2	Workflow failure rate < 15%	Silver	monthly_workflow_failure_rate	< 15
3	Workflow failure rate not degrading	Silver	failure_rate_trend	!= "Degrading"
4	Workflow failure rate < 5%	Gold	monthly_workflow_failure_rate	< 5

GitLab

#	Rule	Level	Property	Condition
1	Pipeline job failure rate < 30%	Bronze	monthly_pipeline_job_failure_rate	< 30
2	Pipeline job failure rate < 15%	Silver	monthly_pipeline_job_failure_rate	< 15
3	Failure rate not degrading	Silver	pipeline_job_failure_rate_trend	!= "Degrading"
4	Pipeline job failure rate < 5%	Gold	monthly_pipeline_job_failure_rate	< 5

Azure DevOps

#	Rule	Level	Property	Condition
1	Build failure rate < 30%	Bronze	monthly_pipeline_build_failure_rate	< 30
2	Build failure rate < 15%	Silver	monthly_pipeline_build_failure_rate	< 15
3	Build failure rate not degrading	Silver	pipeline_build_failure_rate_trend	!= "Degrading"
4	Build failure rate < 5%	Gold	monthly_pipeline_build_failure_rate	< 5

Configure the reliability health scorecard on the organization blueprint

1. Go to the Builder page.
2. Search for the Organization blueprint and select it.
3. Click on the Scorecards tab.
4. Open the existing org_reliability_health scorecard.
5. Click on the {...} button in the top right corner, and choose Edit JSON.

GitHub (Ocean)

6. Paste the following full scorecard JSON:

   Organization Reliability Health scorecard - GitHub (click to expand)

   {
     "identifier": "org_reliability_health",
     "title": "Reliability Health",
     "levels": [
       {
         "color": "paleBlue",
         "title": "Basic"
       },
       {
         "color": "bronze",
         "title": "Bronze"
       },
       {
         "color": "silver",
         "title": "Silver"
       },
       {
         "color": "gold",
         "title": "Gold"
       }
     ],
     "rules": [
       {
         "identifier": "github_failure_rate_under_30",
         "title": "Workflow failure rate < 30%",
         "description": "Organization-wide aggregate monthly CI/CD failure rate is below 30%.",
         "level": "Bronze",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 30
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_15",
         "title": "Workflow failure rate < 15%",
         "description": "Organization-wide aggregate monthly CI/CD failure rate is below 15%.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 15
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_not_degrading",
         "title": "Workflow failure rate not degrading",
         "description": "Organization's weekly failure rate is not worsening compared to the monthly average.",
         "level": "Silver",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "!=",
               "property": "failure_rate_trend",
               "value": "Degrading"
             }
           ]
         }
       },
       {
         "identifier": "github_failure_rate_under_5",
         "title": "Workflow failure rate < 5%",
         "description": "Organization-wide aggregate monthly CI/CD failure rate is below 5%, indicating elite performance.",
         "level": "Gold",
         "query": {
           "combinator": "and",
           "conditions": [
             {
               "operator": "<",
               "property": "monthly_workflow_failure_rate",
               "value": 5
             }
           ]
         }
       }
     ]
   }

GitLab

6. Append the following rules to the rules array:

   GitLab rules to append to the org_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "gitlab_failure_rate_under_30",
       "title": "Pipeline job failure rate < 30%",
       "description": "Organization-wide aggregate monthly pipeline job failure rate is below 30%.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_15",
       "title": "Pipeline job failure rate < 15%",
       "description": "Organization-wide aggregate monthly pipeline job failure rate is below 15%.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_not_degrading",
       "title": "Failure rate not degrading",
       "description": "Organization's weekly pipeline job failure rate is not worsening compared to the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_job_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "gitlab_failure_rate_under_5",
       "title": "Pipeline job failure rate < 5%",
       "description": "Organization-wide aggregate monthly pipeline job failure rate is below 5%, indicating elite performance.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_job_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

Azure DevOps

6. Append the following rules to the rules array:

   Azure DevOps rules to append to the org_reliability_health scorecard (click to expand)

   [
     {
       "identifier": "ado_failure_rate_under_30",
       "title": "Build failure rate < 30%",
       "description": "Less than 30% of pipeline builds failed across the organization in the last 30 days.",
       "level": "Bronze",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 30
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_15",
       "title": "Build failure rate < 15%",
       "description": "Less than 15% of pipeline builds failed across the organization.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 15
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_not_degrading",
       "title": "Build failure rate not degrading",
       "description": "Organization's weekly build failure rate is not higher than the monthly average.",
       "level": "Silver",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "!=",
             "property": "pipeline_build_failure_rate_trend",
             "value": "Degrading"
           }
         ]
       }
     },
     {
       "identifier": "ado_failure_rate_under_5",
       "title": "Build failure rate < 5%",
       "description": "Less than 5% of pipeline builds failed across the organization - elite reliability.",
       "level": "Gold",
       "query": {
         "combinator": "and",
         "conditions": [
           {
             "operator": "<",
             "property": "monthly_pipeline_build_failure_rate",
             "value": 5
           }
         ]
       }
     }
   ]

7. Click Save to update the scorecard.

---

Visualize scorecard results

Once the scorecards are in place, create a dashboard to monitor pipeline reliability maturity across services and teams.

Create the dashboard

1. Navigate to your context lake.
2. Click on the + button in the left sidebar.
3. Select New folder.
4. Name the folder Engineering Intelligence and click Create.
5. Inside the Engineering Intelligence folder, click + again.
6. Select New dashboard.
7. Name the dashboard Pipeline Reliability Scorecards and click Create.

Add widgets

You can populate the dashboard using either an API script or by manually creating each widget through the UI.

API script

The fastest way to set up the dashboard is by using Port's API to create all widgets at once.

Get your Port API token

1. In your Port, click on your profile picture in the top right corner.

2. Select Credentials.

3. Click Generate API token.

4. Copy the generated token and store it as an environment variable:

   export PORT_ACCESS_TOKEN="YOUR_GENERATED_TOKEN"

   EU region

   If Port is hosted in the EU region, replace api.port.io with api.port-eu.io in the dashboard creation command below.

Create the dashboard with widgets

Provider-specific property names

The service table in the dashboard JSON below includes columns for GitHub (github_*), GitLab (gitlab_*), and Azure DevOps (ado_*). Only the columns matching the properties that exist on your service blueprint will display data. The team table uses provider-agnostic column names that work across GitHub, GitLab, and Azure DevOps.

Save the following JSON to a file named reliability_scorecards_dashboard.json:

Dashboard JSON payload (click to expand)

{
  "identifier": "reliability_scorecards",
  "title": "Pipeline Reliability Scorecards",
  "icon": "Alert",
  "type": "dashboard",
  "parent": "engineering_intelligence",
  "widgets": [
    {
      "id": "reliabilityScorecardsDashboardWidget",
      "type": "dashboard-widget",
      "layout": [
        {
          "height": 400,
          "columns": [
            {"id": "serviceLevelDistribution", "size": 4},
            {"id": "teamLevelDistribution", "size": 4},
            {"id": "servicesBelowBronze", "size": 4}
          ]
        },
        {
          "height": 400,
          "columns": [
            {"id": "serviceReliabilityTable", "size": 12}
          ]
        },
        {
          "height": 400,
          "columns": [
            {"id": "teamReliabilityTable", "size": 12}
          ]
        }
      ],
      "widgets": [
        {
          "id": "serviceLevelDistribution",
          "type": "entities-pie-chart",
          "title": "Service Reliability Levels",
          "icon": "Pie",
          "description": "Distribution of reliability scorecard levels across services",
          "blueprint": "service",
          "property": "scorecard#reliability_health",
          "dataset": {
            "combinator": "and",
            "rules": []
          }
        },
        {
          "id": "teamLevelDistribution",
          "type": "entities-pie-chart",
          "title": "Team Reliability Levels",
          "icon": "Pie",
          "description": "Distribution of reliability scorecard levels across teams",
          "blueprint": "_team",
          "property": "scorecard#team_reliability_health",
          "dataset": {
            "combinator": "and",
            "rules": [
              {
                "operator": "=",
                "property": "type",
                "value": "team"
              }
            ]
          }
        },
        {
          "id": "servicesBelowBronze",
          "type": "entities-number-chart",
          "title": "Services Below Bronze",
          "icon": "Alert",
          "description": "Number of services that have not reached Bronze reliability level",
          "blueprint": "service",
          "chartType": "countEntities",
          "calculationBy": "entities",
          "func": "count",
          "unit": "none",
          "dataset": {
            "combinator": "and",
            "rules": [
              {
                "operator": "=",
                "property": "scorecard#reliability_health",
                "value": "Basic"
              }
            ]
          }
        },
        {
          "id": "serviceReliabilityTable",
          "type": "table-entities-explorer",
          "displayMode": "widget",
          "title": "Service Reliability Scorecard Overview",
          "icon": "Table",
          "description": "Services with their reliability scorecard level and failure rate metrics",
          "blueprint": "service",
          "dataset": {"combinator": "and", "rules": []},
          "excludedFields": [],
          "blueprintConfig": {
            "service": {
              "groupSettings": {"groupBy": ["$team"]},
              "propertiesSettings": {
                "order": ["$title", "$team", "scorecard-property#reliability_health", "github_monthly_workflow_failure_rate", "github_weekly_workflow_failure_rate", "github_failure_rate_trend", "github_workflow_runs_30d", "github_failed_workflow_runs_30d", "gitlab_monthly_pipeline_job_failure_rate", "gitlab_weekly_pipeline_job_failure_rate", "gitlab_pipeline_job_failure_rate_trend", "gitlab_pipeline_jobs_30d", "gitlab_failed_pipeline_jobs_30d", "ado_monthly_pipeline_build_failure_rate", "ado_weekly_pipeline_build_failure_rate", "ado_pipeline_build_failure_rate_trend", "ado_pipeline_builds_30d", "ado_failed_pipeline_builds_30d"],
                "shown": ["$title", "$team", "scorecard-property#reliability_health", "github_monthly_workflow_failure_rate", "github_weekly_workflow_failure_rate", "github_failure_rate_trend", "github_workflow_runs_30d", "github_failed_workflow_runs_30d", "gitlab_monthly_pipeline_job_failure_rate", "gitlab_weekly_pipeline_job_failure_rate", "gitlab_pipeline_job_failure_rate_trend", "gitlab_pipeline_jobs_30d", "gitlab_failed_pipeline_jobs_30d", "ado_monthly_pipeline_build_failure_rate", "ado_weekly_pipeline_build_failure_rate", "ado_pipeline_build_failure_rate_trend", "ado_pipeline_builds_30d", "ado_failed_pipeline_builds_30d"]
              },
              "filterSettings": {"filterBy": {"combinator": "and", "rules": []}},
              "sortSettings": {"sortBy": []}
            }
          }
        },
        {
          "id": "teamReliabilityTable",
          "type": "table-entities-explorer",
          "displayMode": "widget",
          "title": "Team Reliability Scorecard Overview",
          "icon": "Table",
          "description": "Teams with their reliability scorecard level and failure rate metrics",
          "blueprint": "_team",
          "dataset": {
            "combinator": "and",
            "rules": [
              {
                "operator": "=",
                "property": "type",
                "value": "team"
              }
            ]
          },
          "excludedFields": [],
          "blueprintConfig": {
            "_team": {
              "groupSettings": {},
              "propertiesSettings": {
                "order": ["$title", "scorecard-property#team_reliability_health", "monthly_workflow_failure_rate", "weekly_workflow_failure_rate", "failure_rate_trend", "monthly_pipeline_job_failure_rate", "weekly_pipeline_job_failure_rate", "pipeline_job_failure_rate_trend", "gitlab_pipeline_jobs_30d", "gitlab_failed_pipeline_jobs_30d", "github_workflow_runs_30d", "github_failed_workflow_runs_30d", "monthly_pipeline_build_failure_rate", "pipeline_build_failure_rate_trend", "ado_pipeline_builds_30d", "ado_failed_pipeline_builds_30d"],
                "shown": ["$title", "scorecard-property#team_reliability_health", "monthly_workflow_failure_rate", "weekly_workflow_failure_rate", "failure_rate_trend", "monthly_pipeline_job_failure_rate", "weekly_pipeline_job_failure_rate", "pipeline_job_failure_rate_trend", "gitlab_pipeline_jobs_30d", "gitlab_failed_pipeline_jobs_30d", "github_workflow_runs_30d", "github_failed_workflow_runs_30d", "monthly_pipeline_build_failure_rate", "pipeline_build_failure_rate_trend", "ado_pipeline_builds_30d", "ado_failed_pipeline_builds_30d"]
              },
              "filterSettings": {"filterBy": {"combinator": "and", "rules": []}},
              "sortSettings": {"sortBy": []}
            }
          }
        }
      ]
    }
  ]
}

Then run the following command to create the dashboard with all widgets:

curl -s -X POST "https://api.port.io/v1/pages" \
  -H "Authorization: Bearer $PORT_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d @reliability_scorecards_dashboard.json | python3 -m json.tool

Engineering Intelligence folder

The script assumes a folder with the identifier engineering_intelligence already exists in your catalog. If you haven't created it yet, follow steps 1-4 in the Create the dashboard section first. To find the exact identifier of an existing folder, open it in your catalog and check the URL - the identifier appears as the last path segment (e.g., app.port.io/organization/catalog/my_folder_id). Replace engineering_intelligence in the JSON with your actual folder identifier if it differs.

Manual setup

Scorecard level distribution

Service reliability levels pie chart (click to expand)

1. Click + Widget and select Pie Chart.
2. Title: Service Reliability Levels.
3. Description: Distribution of reliability scorecard levels across services.
4. Choose Service as the Blueprint.
5. Select Reliability Health from the Breakdown by property dropdown (this is the scorecard#reliability_health property).
6. Click Save.

Team reliability levels pie chart (click to expand)

1. Click + Widget and select Pie Chart.
2. Title: Team Reliability Levels.
3. Description: Distribution of reliability scorecard levels across teams.
4. Choose Team as the Blueprint.
5. Select Reliability Health from the Breakdown by property dropdown (this is the scorecard#team_reliability_health property).
6. Add a filter: type = "team".
7. Click Save.

Attention metrics

Services below bronze number chart (click to expand)

1. Click + Widget and select Number Chart.

2. Title: Services Below Bronze.

3. Description: Number of services that have not reached Bronze reliability level.

4. Select Count Entities Chart type and choose Service as the Blueprint.

5. Add this JSON to the Additional filters editor:

   {
     "combinator": "and",
     "rules": [
       {
         "operator": "=",
         "property": "scorecard#reliability_health",
         "value": "Basic"
       }
     ]
   }

6. Click Save.

Scorecard tables

Service reliability scorecard overview table (click to expand)

1. Click + Widget and select Table.
2. Title: Service Reliability Scorecard Overview.
3. Description: Services with their reliability scorecard level and failure rate metrics.
4. Choose Service as the Blueprint.
5. Group by Team.
6. Configure the displayed columns to show: title, team, reliability level, monthly failure rate, weekly failure rate, failure rate trend, monthly workflow runs, and monthly failed runs.
7. Click Save.

Team reliability scorecard overview table (click to expand)

1. Click + Widget and select Table.
2. Title: Team Reliability Scorecard Overview.
3. Description: Teams with their reliability scorecard level and failure rate metrics.
4. Choose Team as the Blueprint.
5. Add a filter: type = "team".
6. Configure the displayed columns to show: title, reliability level, monthly failure rate, weekly failure rate, failure rate trend, monthly workflow runs, and monthly failed runs.
7. Click Save.

Next steps

• Add automations: Use Port automations to notify teams via Slack when their service drops below Bronze, or create Jira tickets for services with persistently high failure rates.
• Customize thresholds: The failure rate thresholds (30%, 15%, 5%) are starting points. Adjust them to match your organization's baselines.
• Combine with Delivery Performance: Stack this scorecard alongside the Delivery Performance scorecard to get both reliability and velocity signals in one dashboard.

Related guides

• Measure pipeline reliability : The prerequisite guide that sets up pipeline ingestion and failure rate properties.
• Set up Delivery Performance scorecards : Complement pipeline reliability with PR throughput and cycle time scorecards.
• Set up a Service Production Readiness scorecard : Combine with repository hygiene and ownership standards.