> For the complete documentation index, see llms.txt.
Skip to main content

Check out Port for yourself ➜ 

Set up the Pull Request Enricher with Port workflows

Overview​

This guide walks you through setting up a "Pull Request Enricher" using Port workflows and an AI agent.
By the end of this guide, your developers will receive automated, contextual comments on their pull requests to help streamline the review process.

This guide uses a single event-triggered workflow. When a pull request is opened, the workflow invokes an AI agent to analyze it and then dispatches a GitHub Actions workflow to post the agent's response as a comment.

PR Enricher with AI workflow

Common use cases​

  • Automatically comment on new pull requests with assessments based on PR metadata.
  • Highlight potential areas of concern.
  • Provide context from related Jira tickets directly in the PR comments.
  • Suggest relevant reviewers based on code changes.

Prerequisites​

This guide assumes you have:

Open Beta

Port workflows are currently in open beta and available to all users. Workflows may undergo changes without prior notice.

How it works​

The workflow chains three nodes:

  1. An event trigger that fires when a githubPullRequest entity is created with status set to open.
  2. An AI node that invokes the PR Review Assistant agent. The workflow pauses until the AI invocation completes, then exposes the agent's response as an output.
  3. A GitHub dispatch node that triggers a GitHub Actions workflow to post the agent's response as a PR comment.

Set up data model​

First, we will create the Pull Request Enricher AI agent in Port.
We will need to configure:

  • The data sources it will use to analyze pull requests.
  • The agent configuration that defines its capabilities.

Configure data source access​

For this guide, we will be using GitHub as our data source to provide comprehensive pull request analysis capabilities. Make sure you have Port's GitHub Ocean integration installed and configured.

Additional data sources

While this guide uses GitHub, you can enhance the agent's capabilities by adding other data sources like:

  • PagerDuty for checking active incidents.
  • Your software catalog services for broader context.

Create the agent configuration​

  1. Go to the AI Agents page of your portal.

  2. Click on + AI Agent.

  3. Toggle Json mode on.

  4. Copy and paste the following JSON schema:

    Pull Request Enricher agent configuration (Click to expand)
    {
      "identifier": "pr_assistant_ai_agent",
      "title": "PR Review Assistant",
      "description": "AI agent that analyzes pull requests and provides helpful context for reviewers",
      "icon": "Pipeline",
      "properties": {
        "description": "Comment on open PRs with additional context to assist developers",
        "status": "active",
        "tools": [
           "^(list|search|describe|track)_.*"
         ],
        "prompt": "Analyze pull request details and provide a structured review in the following format:\n\nPR Description:\n Summarize the PR description and key properties in one sentence \n\nKey Points:\n{{ List up to 3 key insights about the changes }}\n\nObservations:\nProvide up to 3 observations using 🟢 for low risk, 🟡 for medium risk, and 🔴 for high risk\n\nRecommendations:\nList up to 3 optional recommendations for improvement. Provide this in markdown format",
        "execution_mode": "Automatic"
      }
    }

  5. Click on Create to save the agent.

Build the workflow​

Now let's create the workflow that ties the trigger, the AI agent, and the comment dispatch together.

The workflow uses an INTEGRATION_ACTION node to dispatch the GitHub Actions workflow. With the GitHub Ocean integration, you do not need to add a GitHub token to Port secrets. The integration handles workflow dispatch authentication.

To build the workflow:

  1. Go to the Workflows page of your portal.

  2. Click on the + Workflow button in the top-right corner.

  3. Click on the Skip to editor button.

  4. Copy and paste the workflow JSON below into the editor to replace the example workflow:

    PR Enricher workflow JSON (Click to expand)
    {
      "identifier": "pr_enricher",
      "title": "PR Enricher",
      "icon": "Pipeline",
      "description": "Analyze newly opened pull requests with an AI agent and post the result as a PR comment",
      "nodes": [
        {
          "identifier": "trigger",
          "title": "On PR opened",
          "icon": "Github",
          "description": "Trigger when a pull request is opened",
          "config": {
            "type": "EVENT_TRIGGER",
            "event": {
              "type": "ENTITY_CREATED",
              "blueprintIdentifier": "githubPullRequest"
            },
            "condition": {
              "type": "JQ",
              "expressions": [
                ".diff.after.properties.status == \"open\""
              ],
              "combinator": "and"
            }
          },
          "variables": {}
        },
        {
          "identifier": "enrich_pr",
          "title": "Analyze PR with AI agent",
          "icon": "AI",
          "description": "Invoke the PR Review Assistant agent",
          "config": {
            "type": "AI_AGENT",
            "agentIdentifier": "pr_assistant_ai_agent",
            "userPrompt": "Analyze pull request with entity identifier '{{ .outputs.trigger.diff.after.identifier }}'"
          },
          "variables": {}
        },
        {
          "identifier": "post_pr_comment",
          "title": "Post PR comment",
          "icon": "Github",
          "description": "Dispatch the GitHub Actions workflow that posts the agent response as a PR comment",
          "config": {
            "type": "INTEGRATION_ACTION",
            "installationId": "<YOUR_GITHUB_OCEAN_INTEGRATION_ID>",
            "integrationProvider": "github-ocean",
            "integrationInvocationType": "dispatch_workflow",
            "integrationActionExecutionProperties": {
              "org": "<YOUR_GITHUB_ORG>",
              "repo": "<YOUR_GITHUB_REPO>",
              "workflow": "post-pr-comment.yml",
              "workflowInputs": {
                "comment": "{{ .outputs.enrich_pr.response }}",
                "pr": "{{ .outputs.trigger.diff.after.properties.prNumber | tostring }}",
                "repo": "{{ (.outputs.trigger.diff.after.properties.link | tostring | split(\"/\"))[3] + \"/\" + ((.outputs.trigger.diff.after.properties.link | tostring | split(\"/\"))[4]) }}"
              },
              "reportWorkflowStatus": true
            }
          },
          "variables": {}
        }
      ],
      "connections": [
        {
          "sourceIdentifier": "trigger",
          "targetIdentifier": "enrich_pr"
        },
        {
          "sourceIdentifier": "enrich_pr",
          "targetIdentifier": "post_pr_comment"
        }
      ]
    }
    Replace placeholders

    Make sure to replace YOUR_GITHUB_OCEAN_INTEGRATION_ID, YOUR_GITHUB_ORG, and YOUR_GITHUB_REPO with your GitHub Ocean integration ID and the organization and repository where your post-pr-comment.yml workflow resides. You can find the integration ID on the Data sources page of your portal.

  5. Click Save to save the workflow.

Create the GitHub workflow​

Now let's create the workflow file that contains the logic to post the comment.

Dedicated Workflows Repository

We recommend creating a dedicated repository for the workflows that are used by Port actions.

Choose the authentication method that best fits your setup:

This approach uses your personal GitHub account to comment on pull requests. It's simpler to set up but comments will appear from your personal account.

In your dedicated workflow repository, ensure you have a .github/workflows directory.

  1. Create a new file named post-pr-comment.yml.

  2. Copy and paste the following snippet:

    Post PR comment workflow using PAT (Click to expand)
    name: Comment on PR

    on:
      workflow_dispatch:
        inputs:
          comment:
            description: 'The comment text to post'
            required: true
            type: string
          repo:
            description: 'Repository in "owner/repo" format (e.g., port-labs/puddle-integrations)'
            required: true
            type: string
          pr:
            description: 'Pull request number'
            required: true
            type: string

    jobs:
      comment:
        runs-on: ubuntu-latest
        steps:
          - name: Comment on PR using PAT
            env:
              GITHUB_TOKEN: ${{ secrets.COMMENT_ON_PR_TOKEN }}
              COMMENT: ${{ github.event.inputs.comment }}
              REPO: ${{ github.event.inputs.repo }}
              PR_NUMBER: ${{ github.event.inputs.pr }}
            run: |
              # Split repo into owner and name
              OWNER=$(echo "$REPO" | cut -d'/' -f1)
              REPO_NAME=$(echo "$REPO" | cut -d'/' -f2)
              
              # Post comment to PR using GitHub CLI
              echo "$COMMENT" | gh pr comment "$PR_NUMBER" --repo "$OWNER/$REPO_NAME" --body-file -
    Required GitHub Secrets

    For this workflow to function properly, you need to add the following secret to your GitHub repository:

    • COMMENT_ON_PR_TOKEN: Your Personal Access Token with Read and Write permissions for Pull requests. Follow GitHub's guide to create one.

  3. Commit and push the changes to your repository.

Choosing the right approach
  • Personal Access Token: Simpler setup, comments appear from your personal account, suitable for smaller teams or personal projects.
  • GitHub App: More professional, comments appear from a dedicated app account, better for larger organizations and provides more granular permissions.

Example response​

The PR Enricher workflow will automatically comment on new pull requests with information like:

PR Enricher AI Agent GitHub comment example

Best practices​

To get the most out of your PR Enricher workflow:

  1. Ensure PR-Jira Linking: Verify that your Port setup correctly establishes relations between githubPullRequest entities and jiraIssue entities.

  2. Start with basic assessment: Begin with simple risk assessment and gradually add more context as you see how the agent performs.

  3. Monitor responses: Regularly review the agent's comments to ensure they're helpful and accurate. You may refine the agent's prompt based on the quality of responses and specific needs of your team.

  4. Troubleshoot: If you're not getting the expected results, check our troubleshooting guide for common issues and solutions.