OpenCost

Loading version...

Port's Opencost integration allows you to model Opencost resources in your software catalog and ingest data into them.

Supported resources External link icon

Setup

Choose one of the following installation methods: Not sure which method is right for your use case? Check the available installation methods.

Choose method

Install

Go to the Opencost Connectors page in your portal.
Select the Self-hosted method.
A helm command will be displayed, with default values already filled out (e.g. your Port client ID, client secret, etc). Copy the command, replace the placeholders with your values, then run it in your terminal to install the integration.

Live event support

Currently, live events are not supported for this integration.
Resyncs will be performed periodically (with a configurable interval), or manually triggered by you via Port's UI.

Therefore, real-time events (including GitOps) will not be ingested into Port immediately.
Live events support for this integration is WIP and will be supported in the near future.

The resync interval can be configured using the scheduledResyncInterval parameter. See the optional parameters section below for more details.

Required parameters:

port.clientId - Your Port client ID (How to get the credentials).
port.clientSecret - Your Port client secret (How to get the credentials).
port.baseUrl - Your Port API URL - https://api.port.io for EU, https://api.us.port.io for US.
integration.identifier - Change the identifier to describe your integration.
integration.type - The integration type.
integration.eventListener.type - The event listener type.
integration.config.opencostHost - The Opencost server URL.

Optional parameters (click to expand)

scheduledResyncInterval - The number of minutes between each resync.
initializePortResources - Default true, When set to true the integration will create default blueprints and the port App config Mapping.
sendRawDataExamples - Enable sending raw data examples from the third party API to port for testing and managing the integration mapping. Default is true.

GitHub
Jenkins
Azure DevOps
GitLab

Go to the Opencost Connectors page in your portal.
Under Select your installation method, choose your desired CI/CD platform.
Copy the workflow and replace the placeholders with your values.

Below are the available parameters and their descriptions.

Parameter	Description	Required?	Secret?
`port_client_id`	Your Port client ID (How to get the credentials).	✓	✓
`port_client_secret`	Your Port client secret (How to get the credentials).	✓	✓
`port_base_url`	Your Port API URL (`https://api.port.io` for EU, `https://api.us.port.io` for US).	✓	✗
`opencost_host`	The Opencost server URL.	✓	✗
`initialize_port_resources`	Default `true`. When set to `true`, the integration will create default blueprints and the Port app config mapping.	✗	✗
`integration_identifier`	The identifier of the integration that will be installed.	✗	✗
`send_raw_data_examples`	Enable sending raw data examples from the third-party API to Port for testing and managing the integration mapping. Default is `true`.	✗	✗

Your Jenkins agent must be able to run Docker commands.

Below are the available parameters and their descriptions.

Parameter	Description	Required?	Secret?
`OCEAN__PORT__CLIENT_ID`	Your Port client ID (How to get the credentials).	✓	✓
`OCEAN__PORT__CLIENT_SECRET`	Your Port client secret (How to get the credentials).	✓	✓
`OCEAN__PORT__BASE_URL`	Your Port API URL (`https://api.port.io` for EU, `https://api.us.port.io` for US).	✓	✗
`OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST`	The Opencost server URL.	✓	✗
`OCEAN__INITIALIZE_PORT_RESOURCES`	Default `true`. When set to `true`, the integration will create default blueprints and the Port app config mapping.	✗	✗
`OCEAN__INTEGRATION__IDENTIFIER`	The identifier of the integration that will be installed.	✗	✗
`OCEAN__SEND_RAW_DATA_EXAMPLES`	Enable sending raw data examples from the third-party API to Port for testing and managing the integration mapping. Default is `true`.	✗	✗

Here is an example for a Jenkinsfile pipeline file:

Jenkins pipeline (click to expand)

pipeline {
  agent any

  stages {
    stage('Run Opencost Integration') {
      steps {
        script {
          withCredentials([
            string(credentialsId: 'OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST', variable: 'OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST'),
            string(credentialsId: 'OCEAN__PORT__CLIENT_ID', variable: 'OCEAN__PORT__CLIENT_ID'),
            string(credentialsId: 'OCEAN__PORT__CLIENT_SECRET', variable: 'OCEAN__PORT__CLIENT_SECRET'),
          ]) {
            sh('''
              #Set Docker image and run the container
              integration_type="opencost"
              version="latest"
              image_name="ghcr.io/port-labs/port-ocean-${integration_type}:${version}"
              docker run -i --rm --platform=linux/amd64 \
                -e OCEAN__EVENT_LISTENER='{"type":"ONCE"}' \
                -e OCEAN__INITIALIZE_PORT_RESOURCES=true \
                -e OCEAN__SEND_RAW_DATA_EXAMPLES=true \
                -e OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST=$OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST \
                -e OCEAN__PORT__CLIENT_ID=$OCEAN__PORT__CLIENT_ID \
                -e OCEAN__PORT__CLIENT_SECRET=$OCEAN__PORT__CLIENT_SECRET \
                -e OCEAN__PORT__BASE_URL='https://api.port.io' \
                $image_name

              exit $?
              ''')
            }
          }
        }
      }
    }
  }
}

Prerequisites

Your Azure Devops agent should be able to run docker commands. Learn more about agents here.
You must have permissions to create and manage Azure DevOps variable groups for storing credentials.

Set up your credentials

Create a Variable Group: Name it port-ocean-credentials.
Store the required variables (see the table below).
Authorize Your Pipeline:
- Go to "Library" -> "Variable groups."
- Find port-ocean-credentials and click on it.
- Select "Pipeline Permissions" and add your pipeline to the authorized list.

Below are the available parameters and their descriptions.

Parameter	Description	Required?	Secret?
`OCEAN__PORT__CLIENT_ID`	Your Port client ID (How to get the credentials).	✓	✓
`OCEAN__PORT__CLIENT_SECRET`	Your Port client secret (How to get the credentials).	✓	✓
`OCEAN__PORT__BASE_URL`	Your Port API URL (`https://api.port.io` for EU, `https://api.us.port.io` for US).	✓	✗
`OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST`	The Opencost server URL.	✓	✗
`OCEAN__INITIALIZE_PORT_RESOURCES`	Default `true`. When set to `true`, the integration will create default blueprints and the Port app config mapping.	✗	✗
`OCEAN__INTEGRATION__IDENTIFIER`	The identifier of the integration that will be installed.	✗	✗
`OCEAN__SEND_RAW_DATA_EXAMPLES`	Enable sending raw data examples from the third-party API to Port for testing and managing the integration mapping. Default is `true`.	✗	✗

Here is an example for a azure-pipeline.yml pipeline file:

Azure pipeline (click to expand)

trigger:
- main

pool:
  vmImage: "ubuntu-latest"

variables:
  - group: port-ocean-credentials


steps:
- script: |
    # Set Docker image and run the container
    integration_type="opencost"
    version="latest"

    image_name="ghcr.io/port-labs/port-ocean-$integration_type:$version"

    docker run -i --rm \
        -e OCEAN__EVENT_LISTENER='{"type":"ONCE"}' \
        -e OCEAN__INITIALIZE_PORT_RESOURCES=true \
        -e OCEAN__SEND_RAW_DATA_EXAMPLES=true \
        -e OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST=$(OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST) \
        -e OCEAN__PORT__CLIENT_ID=$(OCEAN__PORT__CLIENT_ID) \
        -e OCEAN__PORT__CLIENT_SECRET=$(OCEAN__PORT__CLIENT_SECRET) \
        -e OCEAN__PORT__BASE_URL='https://api.port.io' \
        $image_name

    exit $?
  displayName: 'Ingest Data into Port'

Go to the Opencost Connectors page in your portal.
Under Select your installation method, choose your desired CI/CD platform.
Copy the workflow and replace the placeholders with your values.

Below are the available parameters and their descriptions.

Parameter	Description	Required?	Secret?
`OCEAN__PORT__CLIENT_ID`	Your Port client ID (How to get the credentials).	✓	✓
`OCEAN__PORT__CLIENT_SECRET`	Your Port client secret (How to get the credentials).	✓	✓
`OCEAN__PORT__BASE_URL`	Your Port API URL (`https://api.port.io` for EU, `https://api.us.port.io` for US).	✓	✗
`OCEAN__INTEGRATION__CONFIG__OPENCOST_HOST`	The Opencost server URL.	✓	✗
`OCEAN__INITIALIZE_PORT_RESOURCES`	Default `true`. When set to `true`, the integration will create default blueprints and the Port app config mapping.	✗	✗
`OCEAN__INTEGRATION__IDENTIFIER`	The identifier of the integration that will be installed.	✗	✗
`OCEAN__SEND_RAW_DATA_EXAMPLES`	Enable sending raw data examples from the third-party API to Port for testing and managing the integration mapping. Default is `true`.	✗	✗

Advanced integration configuration

For advanced configuration such as proxies or self-signed certificates, see Ocean advanced configuration.

Configuration

Port integrations use a YAML mapping block to ingest data from the third-party api into Port.

The mapping makes use of the JQ JSON processor to select, modify, concatenate, transform and perform other operations on existing fields and values from the integration API.

Default mapping configuration

This is the default mapping configuration for this integration:

Default mapping configuration (click to expand)

createMissingRelatedEntities: true
deleteDependentEntities: true
resources:
- kind: cost
  selector:
    query: 'true'
  port:
    entity:
      mappings:
        blueprint: '"openCostResourceAllocation"'
        identifier: .name
        title: .name
        properties:
          cluster: .properties.cluster
          namespace: .properties.namespace
          startDate: .start
          endDate: .end
          cpuCoreHours: .cpuCoreHours
          cpuCost: .cpuCost
          cpuEfficiency: .cpuEfficiency
          gpuHours: .gpuHours
          gpuCost: .gpuCost
          networkCost: .networkCost
          loadBalancerCost: .loadBalancerCost
          pvCost: .pvCost
          ramBytes: .ramBytes
          ramCost: .ramCost
          ramEfficiency: .ramEfficiency
          sharedCost: .sharedCost
          externalCost: .externalCost
          totalCost: .totalCost
          totalEfficiency: .totalEfficiency
- kind: cloudcost
  selector:
    query: 'true'
    cloudcostAggregate: provider
  port:
    entity:
      mappings:
        blueprint: '"openCostCloudcost"'
        identifier: .properties.provider + "-" + .window.start + "-" + .window.end
        title: .properties.provider + "-" + .window.start + "-" + .window.end
        properties:
          startDate: .window.start
          endDate: .window.end
          listCost: .listCost.cost
          netCost: .netCost.cost
          amortizedNetCost: .amortizedNetCost.cost
          invoicedCost: .invoicedCost.cost
          amortizedCost: .amortizedCost.cost

Mapping & selectors per resource

Examples of blueprints and the relevant integration configurations:

- kind: cost
  selector:
    query: 'true'
  port:
    entity:
      mappings:
        blueprint: '"openCostResourceAllocation"'
        identifier: .name
        title: .name
        properties:
          cluster: .properties.cluster
          namespace: .properties.namespace
          startDate: .start
          endDate: .end
          cpuCoreHours: .cpuCoreHours
          cpuCost: .cpuCost
          cpuEfficiency: .cpuEfficiency
          gpuHours: .gpuHours
          gpuCost: .gpuCost
          networkCost: .networkCost
          loadBalancerCost: .loadBalancerCost
          pvCost: .pvCost
          ramBytes: .ramBytes
          ramCost: .ramCost
          ramEfficiency: .ramEfficiency
          sharedCost: .sharedCost
          externalCost: .externalCost
          totalCost: .totalCost
          totalEfficiency: .totalEfficiency

{
  "name": "ingress-nginx",
  "properties": {
    "cluster": "cluster-one",
    "node": "minikube",
    "container": "controller",
    "controller": "ingress-nginx-controller",
    "controllerKind": "deployment",
    "namespace": "ingress-nginx",
    "pod": "ingress-nginx-controller-7799c6795f-29n7j",
    "services": [
      "ingress-nginx-controller-admission",
      "ingress-nginx-controller"
    ],
    "labels": {
      "app_kubernetes_io_component": "controller",
      "app_kubernetes_io_instance": "ingress-nginx",
      "app_kubernetes_io_name": "ingress-nginx",
      "gcp_auth_skip_secret": "true",
      "kubernetes_io_metadata_name": "ingress-nginx",
      "pod_template_hash": "7799c6795f"
    },
    "namespaceLabels": {
      "app_kubernetes_io_instance": "ingress-nginx",
      "app_kubernetes_io_name": "ingress-nginx",
      "kubernetes_io_metadata_name": "ingress-nginx"
    }
  },
  "window": {
    "start": "2023-10-30T00:00:00Z",
    "end": "2023-10-31T00:00:00Z"
  },
  "start": "2023-10-30T09:05:00Z",
  "end": "2023-10-30T11:50:00Z",
  "minutes": 165,
  "cpuCores": 0.1,
  "cpuCoreRequestAverage": 0.1,
  "cpuCoreUsageAverage": 0,
  "cpuCoreHours": 0.275,
  "cpuCost": 0.00869,
  "cpuCostAdjustment": 0,
  "cpuEfficiency": 0,
  "gpuCount": 0,
  "gpuHours": 0,
  "gpuCost": 0,
  "gpuCostAdjustment": 0,
  "networkTransferBytes": 0,
  "networkReceiveBytes": 0,
  "networkCost": 0,
  "networkCrossZoneCost": 0,
  "networkCrossRegionCost": 0,
  "networkInternetCost": 0,
  "networkCostAdjustment": 0,
  "loadBalancerCost": 0,
  "loadBalancerCostAdjustment": 0,
  "pvBytes": 0,
  "pvByteHours": 0,
  "pvCost": 0,
  "pvs": "None",
  "pvCostAdjustment": 0,
  "ramBytes": 94371840,
  "ramByteRequestAverage": 94371840,
  "ramByteUsageAverage": 0,
  "ramByteHours": 259522560,
  "ramCost": 0.00102,
  "ramCostAdjustment": 0,
  "ramEfficiency": 0,
  "externalCost": 0,
  "sharedCost": 0,
  "totalCost": 0.00972,
  "totalEfficiency": 0,
  "lbAllocations": "None"
}

{
  "identifier": "ingress-nginx",
  "title": "ingress-nginx",
  "blueprint": "openCostResourceAllocation",
  "team": [],
  "properties": {
    "cluster": "cluster-one",
    "namespace": "ingress-nginx",
    "startDate": "2023-10-30T09:05:00.000Z",
    "endDate": "2023-10-30T11:50:00.000Z",
    "cpuCoreHours": 0.275,
    "cpuCost": 0.00869,
    "cpuEfficiency": 0,
    "gpuHours": 0,
    "gpuCost": 0,
    "networkCost": 0,
    "loadBalancerCost": 0,
    "pvCost": 0,
    "ramBytes": 94371840,
    "ramCost": 0.00102,
    "ramEfficiency": 0,
    "sharedCost": 0,
    "externalCost": 0,
    "totalCost": 0.00972,
    "totalEfficiency": 0
  },
  "relations": {},
  "createdAt": "2023-10-15T09:30:57.924Z",
  "createdBy": "hBx3VFZjqgLPEoQLp7POx5XaoB0cgsxW",
  "updatedAt": "2023-10-30T11:49:20.881Z",
  "updatedBy": "hBx3VFZjqgLPEoQLp7POx5XaoB0cgsxW"
}

Below are the supported selectors for this resource. Use them under the selector key in your integration mapping configuration to control which resources are returned from the API.

Selector	Type	Description
`exportEnvVariables`	`array`	Environment variable names whose values should be included in lakehouse ingest payloads for DSP processing.
`window`		Time window for the OpenCost allocation query. Default: `"today"`.
`aggregate`	`string`	Field by which to aggregate the results.
`step`	`string`	Duration of a single allocation set (e.g., '30m', '2h', '1d'). Default is window.
`resolution`	`string`	Duration to use as resolution in Prometheus queries

- kind: cloudcost
  selector:
    query: 'true'
    cloudcostAggregate: provider
  port:
    entity:
      mappings:
        blueprint: '"openCostCloudcost"'
        identifier: .properties.provider + "-" + .window.start + "-" + .window.end
        title: .properties.provider + "-" + .window.start + "-" + .window.end
        properties:
          startDate: .window.start
          endDate: .window.end
          listCost: .listCost.cost
          netCost: .netCost.cost
          amortizedNetCost: .amortizedNetCost.cost
          invoicedCost: .invoicedCost.cost
          amortizedCost: .amortizedCost.cost

{
  "properties": {
    "provider": "AWS",
    "accountID": "123456789012",
    "invoiceEntityID": "AWS-123456789012",
    "service": "Amazon Elastic Compute Cloud",
    "category": "Compute",
    "region": "us-east-1",
    "labels": {
      "environment": "production",
      "team": "platform"
    }
  },
  "window": {
    "start": "2024-03-01T00:00:00Z",
    "end": "2024-03-31T23:59:59Z"
  },
  "listCost": {
    "cost": 1250.75,
    "kubernetesPercent": 0.85
  },
  "netCost": {
    "cost": 1100.50,
    "kubernetesPercent": 0.85
  },
  "amortizedNetCost": {
    "cost": 1050.25,
    "kubernetesPercent": 0.85
  },
  "invoicedCost": {
    "cost": 1250.75,
    "kubernetesPercent": 0.85
  },
  "amortizedCost": {
    "cost": 1200.30,
    "kubernetesPercent": 0.85
  },
  "items": [
    {
      "resourceId": "i-0abc123def456789",
      "name": "eks-node-1",
      "properties": {
        "instanceType": "m5.xlarge",
        "operatingSystem": "Linux"
      },
      "tags": {
        "Name": "eks-node-1",
        "kubernetes.io/cluster/my-cluster": "owned"
      },
      "cost": 625.37,
      "start": "2024-03-01T00:00:00Z",
      "end": "2024-03-31T23:59:59Z"
    },
    {
      "resourceId": "i-0def456789abc1234",
      "name": "eks-node-2",
      "properties": {
        "instanceType": "m5.xlarge",
        "operatingSystem": "Linux"
      },
      "tags": {
        "Name": "eks-node-2",
        "kubernetes.io/cluster/my-cluster": "owned"
      },
      "cost": 625.38,
      "start": "2024-03-01T00:00:00Z",
      "end": "2024-03-31T23:59:59Z"
    }
  ]
}

{
  "identifier": "aws_2024_03_01_t_00_00_00_z_2024_03_31_t_23_59_59_z",
  "title": "AWS-2024-03-01T00:00:00Z-2024-03-31T23:59:59Z",
  "properties": {
    "startDate": "2024-03-01T00:00:00Z",
    "endDate": "2024-03-31T23:59:59Z",
    "listCost": 1250.75,
    "netCost": 1100.5,
    "amortizedNetCost": 1050.25,
    "invoicedCost": 1250.75,
    "amortizedCost": 1200.3
  },
  "relations": {},
  "icon": "Opencost"
}

Below are the supported selectors for this resource. Use them under the selector key in your integration mapping configuration to control which resources are returned from the API.

Selector	Type	Description
`exportEnvVariables`	`array`	Environment variable names whose values should be included in lakehouse ingest payloads for DSP processing.
`window`		Time window for the OpenCost cloud cost query. Default: `"today"`.
`aggregate`	`string`	Field by which to aggregate the results of cloudcost
`accumulate`	`string`	Step size of the accumulation.
`filter`	`string`	Filter results by any category which that can be aggregated by, can support multiple filterable items in the same category in a comma-separated list.

Monitoring and sync status

To learn more about how to monitor and check the sync status of your integration, see the relevant documentation.

Setup​

Live event support

Configuration​

Default mapping configuration​

Mapping & selectors per resource​

Monitoring and sync status​

Setup

Configuration

Default mapping configuration

Mapping & selectors per resource

Monitoring and sync status