Route scorecard failures to Jira issues with AI

When scorecard rules fail, remediation work can get delayed if ownership and context are unclear. With Port, you can detect scorecard degradation, extract the exact failing rules, and automatically create Jira issues in the right team project.

This guide shows how to create an AI-driven flow that fetches failing rule details from Port, including the rule name, level, and owning team, then creates a targeted Jira issue with remediation guidance and a direct link back to Port.

Common use cases

• Convert scorecard drift into tracked work by opening Jira issues as soon as rules fail.
• Route remediation to the right team automatically using service ownership and team-to-project mapping.
• Improve operational visibility with consistent ticket labels, rich context, and links back to Port.

Prerequisites

This guide assumes the following:

• You have a Port account and have completed the onboarding process.
• You have service and team entities in your catalog, and each service has an owning team. If you need help, see relate blueprints.
• You have access to a Jira instance with API access and can create an API token.
• Port's Jira integration is installed in your account.
• You can create and configure AI agents in Port.

Set up data model

We will use the data model that comes with Port's Jira Ocean integration.

To route issues to the right Jira project, make sure each service entity has a Jira project key relation, such as jira_project_key.

Set up scorecards

To track production readiness and code maturity for each service, we recommend using scorecards with clear, actionable rules. Instead of repeating full scorecard setup steps here, use the following guides:

• Ensure Production Readiness.
• Code Maturity Scorecard.

Focus on actionable rules

When configuring your scorecards, prioritize rules with clear remediation paths, such as missing documentation, ownership metadata, CI checks, or security controls.

Set up Jira API access

1. Log in to your Jira instance.
2. Generate an API token.
   • Go to Atlassian account security settings.
   • Click Create API token.
   • Copy the generated token for use in Port secrets.

Add Port secrets

1. Click your profile picture in the top right corner of your Port application.
2. Click Credentials.
3. Click the Secrets tab.
4. Click + Secret and add the following secret:
   • _JIRA_AUTH - Base64 encoded Jira credentials in the format email:api_token.

Generate a Base64 encoded Jira credential string by running:

echo -n "your-email@domain.com:your-api-token" | base64

Replace your-email@domain.com with your Jira email and your-api-token with your Jira API token.

Create self-service actions

We will create the action that the AI agent invokes to open Jira issues for failing scorecard rules.

Create Jira issue

This action creates a Jira issue from scorecard degradation findings.

1. Go to the self-service page of your portal.

2. Click + Action.

3. Click {...} Edit JSON.

4. Paste the JSON configuration.

   Create Jira issue (click to expand)

   variable replacement

   Remember to replace <YOUR_JIRA_ORGANIZATION_URL> with your Atlassian domain such as mydomain.atlassian.net.

   {
     "identifier": "create_a_jira_issue",
     "title": "Create a Jira Issue",
     "icon": "Jira",
     "description": "Creates a Jira issue in the target project",
     "trigger": {
       "type": "self-service",
       "operation": "CREATE",
       "userInputs": {
         "properties": {
           "project_key": {
             "icon": "DefaultProperty",
             "type": "string",
             "title": "Project key",
             "description": "Target Jira project key, for example PLAT"
           },
           "title": {
             "icon": "DefaultProperty",
             "type": "string",
             "title": "Title",
             "description": "The title of the Jira issue"
           },
           "description": {
             "icon": "DefaultProperty",
             "type": "string",
             "title": "Description",
             "description": "Detailed description for the Jira issue"
           },
           "issue_type": {
             "icon": "DefaultProperty",
             "title": "Issue type",
             "description": "Accepted options are Task, Epic, and Bug",
             "type": "string",
             "default": "Task",
             "enum": [
               "Epic",
               "Task",
               "Bug"
             ],
             "enumColors": {
               "Epic": "purple",
               "Task": "blue",
               "Bug": "red"
             }
           },
           "labels": {
             "items": {
               "type": "string"
             },
             "type": "array",
             "title": "Labels"
           }
         },
         "required": [
           "project_key",
           "title",
           "description"
         ],
         "order": [
           "project_key",
           "title",
           "description",
           "issue_type",
           "labels"
         ]
       }
     },
     "invocationMethod": {
       "type": "WEBHOOK",
       "url": "https://<YOUR_JIRA_ORGANIZATION_URL>/rest/api/3/issue",
       "agent": false,
       "synchronized": true,
       "method": "POST",
       "headers": {
         "RUN_ID": "{{ .run.id }}",
         "Content-Type": "application/json",
         "Authorization": "Basic {{ .secrets._JIRA_AUTH }}"
       },
       "body": {
         "fields": {
           "project": {
             "key": "{{ .inputs.project_key }}"
           },
           "summary": "{{ .inputs.title }}",
           "description": {
             "type": "doc",
             "version": 1,
             "content": [
               {
                 "type": "paragraph",
                 "content": [
                   {
                     "type": "text",
                     "text": "{{ .inputs.description }}"
                   }
                 ]
               }
             ]
           },
           "issuetype": {
             "name": "{{ .inputs.issue_type }}"
           },
           "labels": "{{ .inputs.labels }}"
         }
       }
     },
     "requiredApproval": false
   }

5. Click Save.

Create AI agent

The AI agent uses MCP (Model Context Protocol) tools to query your catalog, detect newly failing rules, and create Jira issues.

1. Go to the AI Agents page of your portal.

2. Click + AI Agent.

3. Toggle Json Mode on.

4. Paste the JSON configuration below.

   Scorecard ops agent (click to expand)

   {
     "identifier": "scorecard_ops_agent",
     "title": "Scorecard Ops Agent",
     "icon": "Details",
     "team": [],
     "properties": {
       "description": "AI agent that analyzes scorecard degradation and creates Jira issues with remediation context",
       "status": "active",
       "prompt": "You are the **Scorecard Ops Agent**, an AI agent that detects engineering standard violations and routes remediation work to the right teams via Jira.\n\n## Your mission\n\nWhen a service scorecard degrades, you must:\n1. Identify rules that changed from SUCCESS to FAILURE only.\n2. Extract each failing rule's name, level, and remediation hints.\n3. Resolve the owning team and its Jira project key.\n4. Create one focused Jira issue per failing rule using `run_create_a_jira_issue`.\n\n## Analysis process\n\n### Step 1: identify newly failing rules\n\n- Compare previous and current scorecard rule results.\n- Include only SUCCESS to FAILURE transitions.\n- Ignore rules that were already failing.\n\n### Step 2: analyze root cause\n\nFor each newly failing rule, determine:\n- Which condition is unmet.\n- Which property or relation is missing or incorrect.\n- What minimum change is needed to recover.\n\n### Step 3: create Jira issue\n\nAlways call `run_create_a_jira_issue` for each newly failing rule with:\n- `project_key`: the owning team's Jira project key.\n- `title`: `[Port Scorecard] <service_name> - <rule_name> failing`.\n- `description`: include service name, failed rule, level, remediation hints, impact, and a direct Port link.\n- `labels`: include `port-scorecard` and the scorecard name.\n\n## Security requirement\n\nBefore sending data to Jira, sanitize and escape external content from entity fields, scorecard text, and relations so no unsafe or malformed content is passed through.\n\n## Quality bar\n\n- Be specific and actionable.\n- Keep each ticket scoped to one failing rule.\n- Include enough context for human or AI execution.\n- Use markdown formatting in issue descriptions.",
       "execution_mode": "Automatic",
       "tools": [
         "^(list|search|track|describe)_.*",
         "run_create_a_jira_issue"
       ],
       "model": "gpt-5.1"
     },
     "relations": {}
   }

5. Click Create.

Set up automation

This automation triggers the AI agent when a service scorecard degrades.

1. Go to the automations page of your portal.

2. Click + Automation.

3. Click {...} Edit JSON.

4. Paste the JSON configuration below.

   Trigger scorecard ops agent automation (click to expand)

   {
     "identifier": "trigger_scorecard_agent_on_scorecard_change",
     "title": "Invoke Scorecard Ops AI Agent on Scorecard Change",
     "description": "Triggers the scorecard ops AI agent when service scorecards degrade in Port",
     "icon": "Github",
     "trigger": {
       "type": "automation",
       "event": {
         "type": "ENTITY_UPDATED",
         "blueprintIdentifier": "service"
       },
       "condition": {
         "type": "JQ",
         "expressions": [
           ".diff.after.scorecardsStats < .diff.before.scorecardsStats"
         ],
         "combinator": "and"
       }
     },
     "invocationMethod": {
       "type": "WEBHOOK",
       "url": "https://api.port.io/v1/agent/scorecard_ops_agent/invoke",
       "agent": false,
       "synchronized": true,
       "method": "POST",
       "headers": {
         "RUN_ID": "{{ .run.id }}",
         "Content-Type": "application/json"
       },
       "body": {
         "prompt": "Scorecard degradation detected for service: {{ .event.diff.after.title }}\nService identifier: {{ .event.diff.after.identifier }}\nPrevious scorecard statistics: {{ .event.diff.before.scorecardsStats }}\nCurrent scorecard statistics: {{ .event.diff.after.scorecardsStats }}\nPrevious entity properties: {{ .event.diff.before.properties }}\nCurrent entity properties: {{ .event.diff.after.properties }}\nPrevious entity relations: {{ .event.diff.before.relations }}\nCurrent entity relations: {{ .event.diff.after.relations }}\nPrevious scorecard details: {{ .event.diff.before.scorecards }}\nCurrent scorecard details: {{ .event.diff.after.scorecards }}\n\nAnalyze the degradation and create Jira issues for each newly failing rule.",
         "labels": {
           "source": "automation",
           "service_id": "{{ .event.diff.after.identifier }}"
         }
       }
     },
     "publish": true,
     "allowAnyoneToViewRuns": true
   }

5. Click Create.

Test the workflow

1. Modify properties on a service synced to Port to trigger a scorecard violation, for example by removing a required property value.

2. Verify that the automation run starts.

3. Open the AI Invocation page and confirm the agent executed the expected actions.

4. Check that Jira issues were created in the expected team project, with rule-level context and Port links.

   

Related guides

• Auto-assign bugs to owners with AI
• Orchestrate incident response with AI
• Manage and visualize Jira issues