GCP
Our GCP integration allows you to import your Google Cloud resources into Port, according to your configuration. After the initial import of data, the integration will also listen to live events from Google Cloud to update data in Port in real time.
Core support
Port's Google Cloud integration is open source, view the source code here.
Common use cases
Easily fill your software catalog with data directly from your GCP Organization, for example:
- Map all the resources in your Google Cloud Projects, including Container Clusters, Cloud Run Services, BigQuery Tables, Compute engine Autoscaler and other GCP objects.
- Watch for Google Cloud object changes (create/update/delete) in real-time, and automatically apply the changes to your entities in Port.
- Use relations to create complete, easily digestible views of your Google Cloud infrastructure inside Port.
How it works
Port's Google Cloud integration can retrieve all the resources supported by the Cloud Assets API, and export them to Port as entities of existing blueprints.
The GCP integration allows you to perform extract, transform, load (ETL) on data from the Cloud Assets API into the desired software catalog data model.
Setup
Choose one of the following installation methods: Not sure which method is right for your use case? Check the available installation methods.
For your first deployment of the GCP exporter, we recommend starting with the Helm/scheduled installation method to perform the initial data sync. Once the initial data sync is complete, you can switch to the Terraform deployment method for real-time data sync.
The prerequisites required for installation are listed below.
Prerequisites
Before you begin, ensure that your Google Cloud project has the following APIs enabled. These APIs are crucial for the GCP integration:Required APIs
- Cloud Asset API: Required for accessing your cloud asset data. Enable here
- Cloud Resource Manager API: Required for accessing projects, folders, and organizational units. Enable here
Optional APIs
- Pub/Sub API: Enable this API only if you plan to query Pub/Sub topics or subscriptions. Enable here
- Cloud Quota API: Enabling this API allows the integration to fetch quota data from your enabled services, enabling dynamic rate limiting. If the API is not enabled, the integration will default to a fixed quota value of 400, and dynamic adjustments based on current quota settings will not be applied. Enable here
Creating a service account
To ingest resources from GCP, you need a service account with the following roles:
- Browser
- Cloud Asset Viewer
- Pub/Sub Viewer
Step-by-step setup guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click Create Service Account.
- Enter a Service account name (e.g.
port-ocean-integration) and an optional description. - Click Create and continue.
- Enter a Service account name (e.g.
- In the Grant this service account access to project step, add the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Continue, then Done.
Prerequisites
Before you begin, ensure that your Google Cloud project has the following APIs enabled. These APIs are crucial for the GCP integration:Required APIs
- Cloud Asset API: Required for accessing your cloud asset data. Enable here
- Cloud Resource Manager API: Required for accessing projects, folders, and organizational units. Enable here
Optional APIs
- Pub/Sub API: Enable this API only if you plan to query Pub/Sub topics or subscriptions. Enable here
- Cloud Quota API: Enabling this API allows the integration to fetch quota data from your enabled services, enabling dynamic rate limiting. If the API is not enabled, the integration will default to a fixed quota value of 400, and dynamic adjustments based on current quota settings will not be applied. Enable here
Creating a service account
To ingest resources from GCP, you need a service account with the following roles:
- Browser
- Cloud Asset Viewer
- Pub/Sub Viewer
Step-by-step setup guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click Create Service Account.
- Enter a Service account name (e.g.
port-ocean-integration) and an optional description. - Click Create and continue.
- Enter a Service account name (e.g.
- In the Grant this service account access to project step, add the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Continue, then Done.
Fetching key file
To use the Service Account Key authentication method, you need to fetch a JSON key file from your service account.
According to Google Cloud, this isn't the preferred way for production purposes. The Terraform installation uses Google's native method to authenticate, and is the one we propose for a production setup.
Step-by-step key file creation guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click on the service account you created for the integration.
- Go to the Keys tab → click Add Key → Create new key.
- Select JSON as the key type and click Create. This will download your new Service Account Key configuration file.
Prerequisites
Before you begin, ensure that your Google Cloud project has the following APIs enabled. These APIs are crucial for the GCP integration:Required APIs
- Cloud Asset API: Required for accessing your cloud asset data. Enable here
- Cloud Resource Manager API: Required for accessing projects, folders, and organizational units. Enable here
Optional APIs
- Pub/Sub API: Enable this API only if you plan to query Pub/Sub topics or subscriptions. Enable here
- Cloud Quota API: Enabling this API allows the integration to fetch quota data from your enabled services, enabling dynamic rate limiting. If the API is not enabled, the integration will default to a fixed quota value of 400, and dynamic adjustments based on current quota settings will not be applied. Enable here
Creating a service account
To ingest resources from GCP, you need a service account with the following roles:
- Browser
- Cloud Asset Viewer
- Pub/Sub Viewer
Step-by-step setup guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click Create Service Account.
- Enter a Service account name (e.g.
port-ocean-integration) and an optional description. - Click Create and continue.
- Enter a Service account name (e.g.
- In the Grant this service account access to project step, add the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Continue, then Done.
Fetching key file
To use the Service Account Key authentication method, you need to fetch a JSON key file from your service account.
According to Google Cloud, this isn't the preferred way for production purposes. The Terraform installation uses Google's native method to authenticate, and is the one we propose for a production setup.
Step-by-step key file creation guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click on the service account you created for the integration.
- Go to the Keys tab → click Add Key → Create new key.
- Select JSON as the key type and click Create. This will download your new Service Account Key configuration file.
The GCP integration is deployed using Terraform on Google Cloud Run. It uses our Terraform Ocean Integration Factory module to deploy the integration.
Prerequisites
- Terraform >= 1.9.1
- hashicorp/google Terraform Provider >= 5.25
- hashicorp/google-beta Terraform Provider >= 5.25
- A logged in gCloud CLI with enough Permissions
- Artifact Registry Image
- Enabled APIs:
Artifact Registry image
In order to run the Cloud Run Service, it's mandatory to have a working image. Currently our GHCR based images aren't supported by Google Cloud's Cloud Run platform, so a manual installation to Dockerhub/Artifact Registry is required. In the guide we specify an Artifact Registry approach, but a similar DockerHub approach should yield the same results:
-
Create an Artifact Registry in GCP.
-
Pull our AMD based image from our GHCR registry
docker pull ghcr.io/port-labs/port-ocean-gcp --platform amd64 -
Tag this image
docker tag ghcr.io/port-labs/port-ocean-gcp:latest <your_artifact_registry_/_dockerhub>/port-ocean-gcp:<your_version> -
Push the image to your artifact registry:
docker push <your_artifact_registry_/_dockerhub>/port-ocean-gcp:<your_version>
Required permissions to run terraform apply
In order to successfully deploy the Google Cloud integration, it's crucial to ensure that the user who deploys the integration in the GCP Organization has the appropriate access permissions.
The Google Cloud integration uses the following GCP infrastructure:
- GCP Cloud Run.
- GCP PubSub Topic & Subscription.
- GCP Roles & Service accounts.
- GCP Cloud Assets
- GCP Cloud Asset Inventory
- GCP Cloud Asset Feed (Used for real-time data sync to Port)
Our Terraform module creates a dedicated service account with its own permissions, and is not related to these permissions.
You can either deploy using your own permissions, or impersonating a service account. Either way, the following permissions should be granted:
Required permissions (click to expand)
cloudasset.assets.exportResource
cloudasset.feeds.create
cloudasset.feeds.delete
cloudasset.feeds.get
cloudasset.feeds.list
cloudasset.feeds.update
iam.roles.create
iam.roles.delete
iam.roles.get
iam.roles.undelete
iam.roles.update
iam.serviceAccountKeys.get
iam.serviceAccounts.actAs
iam.serviceAccounts.create
iam.serviceAccounts.delete
iam.serviceAccounts.get
pubsub.subscriptions.consume
pubsub.subscriptions.create
pubsub.subscriptions.delete
pubsub.subscriptions.get
pubsub.subscriptions.list
pubsub.subscriptions.update
pubsub.topics.attachSubscription
pubsub.topics.create
pubsub.topics.delete
pubsub.topics.get
pubsub.topics.list
pubsub.topics.update
resourcemanager.organizations.getIamPolicy
resourcemanager.organizations.setIamPolicy
run.operations.get
run.services.create
run.services.delete
run.services.get
serviceusage.services.use
Setting up Kubernetes
-
Set up a Kubernetes cluster in the Kubernetes Engine in GCP.
-
Enable Workload Identity in the cluster:
gcloud container clusters update CLUSTER_NAME --workload-pool=PROJECT_ID.svc.id.goog --region REGIONReplace the following placeholders:- `CLUSTER_NAME` with the cluster name.- `PROJECT_ID` with the Kubernetes cluster's project id.- `REGION`: the region of the cluster. -
Make sure you are connected to the cluster through the CLI:
gcloud container clusters get-credentials CLUSTER_NAME --region REGION --project PROJECT_IDReplace the following placeholders:
CLUSTER_NAMEwith the cluster name.PROJECT_IDwith the Kubernetes cluster's project id.REGION: the region of the cluster.
-
Bind the Kubernetes service account to the GCP service account.
The Kubernetes service account does not need to exist yet it will be created when we deploy the Helm chart.
gcloud iam service-accounts add-iam-policy-binding GCP_SERVICE_ACCOUNT_EMAIL \--role roles/iam.workloadIdentityUser \--member "serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KUBERNETES_SA_NAME]"Replace the following placeholders:
GCP_SERVICE_ACCOUNT_EMAILwith your GCP service account email.PROJECT_IDwith your GCP project ID.NAMESPACEwith your Kubernetes namespace.KUBERNETES_SA_NAMEwith the Kubernetes service account name that will be used in the integration's values.yaml. (Note: the service account doesn't need to exist)
Configuring and running the Helm command
-
Add the Port Labs Helm repository:
helm repo add --force-update port-labs https://port-labs.github.io/helm-charts -
Create a
values.yamlfile with the following configuration:port:clientId: "<port-client-id>"clientSecret: "<port-client-secret-id>"baseUrl: "https://api.port.io"initializePortResources: truesendRawDataExamples: truescheduledResyncInterval: 1440integration:identifier: "ocean-gcp-integration"type: "gcp"eventListener:type: "POLLING"extraConfig:GCP_PROJECT: "<base-gcp-project>"podServiceAccount:create: truename: "<service-account-from-step-3>"annotations:iam.gke.io/gcp-service-account: "<gcp-service-account-email>"Replace the following placeholders:
<port-client-id>with your Port client ID.<port-client-secret-id>with your Port client secret.<base-gcp-project>with your base GCP project ID.<service-account-from-step-3>with the Kubernetes service account name from step 3.<gcp-service-account-email>with the GCP service account email from step 7 in the Creating a service account guide above.
-
Install the integration using Helm:
helm upgrade --install gcp port-labs/port-ocean -f values.yaml
For more information about GCP workload identity, including how to set it up with other Kubernetes providers, see the official GCP documentation.
Optional - Scale permissions for a service account
The above gives a service account permissions in its project's scope. You can add permissions at other projects, folders, or organization level.
Step-by-step guide to scale permissions (click to expand)
- In the Google Cloud Console, navigate to IAM & Admin → Service Accounts and click on the service account you created for the integration.
- Copy the service account's email address.
- Switch the project dropdown at the top to your organization.
- Navigate to IAM & Admin → Manage Resources (or search for
Manage resourcesin the top search bar). - In the Resources table, select the desired scope (organization, folders, or projects) using the checkboxes on the left.
- A permissions panel will open on the right side. Click Add Principal.
- Paste the service account email and assign the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Save.
Prerequisites
First, let's ensure your cluster meets the following criteria:
- You're running Kubernetes 1.20 or later.
- Previous versions of Kubernetes used a different ServiceAccount token format that is not compatible with the instructions in this document.
- You configured
kube-apiserverso that it supports ServiceAccount token volume projections. - Ensure billing is enabled in your Google Cloud project.
The cluster doesn't need to be accessible over the internet.
Configure workload identity federation
-
Enable the IAM, Resource Manager, Service Account Credentials, and Security Token Service APIs.
-
Now, let's create a workload identity pool and provider:
- Get your cluster's issuer URL, this will be used in later steps:
kubectl get --raw /.well-known/openid-configuration | jq -r .issuer
- Download the cluster's JSON Web Key Set (JWKS):
In one of the following steps, we will upload the JWKS so that Workload Identity Federation can verify the authenticity of the Kubernetes ServiceAccount tokens issued by your cluster.kubectl get --raw /openid/v1/jwks > cluster-jwks.json
- Get your cluster's issuer URL, this will be used in later steps:
-
Create a new workload identity pool:
gcloud iam workload-identity-pools create POOL_ID \--location="global" \--description="DESCRIPTION" \--display-name="DISPLAY_NAME"Replace the following placeholders:
POOL_ID: a unique identifier for the new pool.DESCRIPTION: a description of the pool.DISPLAY_NAME: the name of the pool.
-
Add the Kubernetes cluster as a workload identity pool provider and upload the cluster's JWKS:
gcloud iam workload-identity-pools providers create-oidc WORKLOAD_PROVIDER_ID \--location="global" \--workload-identity-pool="POOL_ID" \--issuer-uri="ISSUER" \--attribute-mapping="google.subject=assertion.sub" \--jwk-json-path="cluster-jwks.json"Replace the following placeholders:
WORKLOAD_PROVIDER_ID: a unique identifier for the new provider.POOL_ID: the ID of the pool created in step 3.ISSUER: the cluster URL we determined earlier.
Grant access to Kubernetes workload
-
Grant the Kubernetes ServiceAccount access to impersonate the IAM service account.
The Kubernetes ServiceAccount does not need to exist yet it will be created in a later step when you deploy the workload. Just take note of the name you want to use.
gcloud iam service-accounts add-iam-policy-binding \SERVICE_ACCOUNT_EMAIL \--member="principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/subject/MAPPED_SUBJECT" \--role=roles/iam.workloadIdentityUserReplace the following placeholders:
SERVICE_ACCOUNT_EMAIL: the email of your service account.PROJECT_NUMBER: the Google Cloud project number (not project ID).POOL_ID: the pool ID we created in step 3.MAPPED_SUBJECT: the mapped subject for the Kubernetes service account. It should have the formatsystem:serviceaccount:NAMESPACE:KUBERNETES_SA_NAME. For example:system:serviceaccount:default:my-kube-workload-id-serviceaccount.
Deploy the Kubernetes workload
-
Create a credentials file:
gcloud iam workload-identity-pools create-cred-config \projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/WORKLOAD_PROVIDER_ID \--service-account=SERVICE_ACCOUNT_EMAIL \--credential-source-file=/var/run/service-account/token \--credential-source-type=text \--sts-location=global \--output-file=credential-configuration.jsonReplace the following placeholders:
SERVICE_ACCOUNT_EMAIL: the email of your service account.PROJECT_NUMBER: the Google Cloud project number (not project ID).POOL_ID: the pool ID we created in step 3.WORKLOAD_PROVIDER_ID: the ID of the cluster workload provider we created in step 4 ofconfigure workload identity federation.
Credential fileUnlike a service account key, a credential configuration file doesn't contain a private key and doesn't need to be kept confidential. Details about the credential configuration file are available at https://google.aip.dev/auth/4117.
-
Create a base64 representation of the credentials file and copy it to your clipboard:
cat credential-configuration.json | base64 | pbcopy -
Create a
values.yamlfile with the following configuration:Yaml configuration (click to expand)
port:clientId: "<port-client-id>"clientSecret: "<port-client-secret>"baseUrl: "https://api.port.io"initializePortResources: truesendRawDataExamples: truescheduledResyncInterval: 1440integration:identifier: "ocean-gcp-integration"type: "gcp"eventListener:type: "POLLING"config:encodedADCConfiguration: "<base64-encoded-credentials-file>"extraConfig:GCP_PROJECT: "<gcp-project-id>"extraVolumeMounts:- name: tokenmountPath: "/var/run/service-account"readOnly: trueextraVolumes:- name: tokenprojected:sources:- serviceAccountToken:audience: "https://iam.googleapis.com/projects/<gcp-project-number>/locations/global/workloadIdentityPools/<workload-identity-pool-id>/providers/<workload-identity-provider-id>"expirationSeconds: 3600path: tokenpodServiceAccount:create: truename: "<k8s-service-account-name>"annotations:iam.gke.io/gcp-service-account: "<gcp-service-account-email>"Replace the following placeholder values:
<port-client-id>: your Port client ID.<port-client-secret>: your Port client secret.<base64-encoded-credentials-file>: the base64 encoded credentials file from step 2.<gcp-project-id>: your GCP project's ID.<gcp-project-number>: your GCP project's number.<workload-identity-pool-id>: the pool ID we created earlier.<workload-identity-provider-id>: the workload provider ID we created earlier.<gcp-service-account-email>: your service account email.<k8s-service-account-name>: the name of the Kubernetes service account to create.
-
Install the integration using Helm:
helm upgrade --install gcp port-labs/port-ocean -f values.yaml
For more information about GCP workload identity, including how to set it up with other Kubernetes providers, see the official GCP documentation.
Optional - Scale permissions for a service account
The above gives a service account permissions in its project's scope. You can add permissions at other projects, folders, or organization level.
Step-by-step guide to scale permissions (click to expand)
- In the Google Cloud Console, navigate to IAM & Admin → Service Accounts and click on the service account you created for the integration.
- Copy the service account's email address.
- Switch the project dropdown at the top to your organization.
- Navigate to IAM & Admin → Manage Resources (or search for
Manage resourcesin the top search bar). - In the Resources table, select the desired scope (organization, folders, or projects) using the checkboxes on the left.
- A permissions panel will open on the right side. Click Add Principal.
- Paste the service account email and assign the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Save.
Fetching key file
To use the Service Account Key authentication method, you need to fetch a JSON key file from your service account.
According to Google Cloud, this isn't the preferred way for production purposes. The Terraform installation uses Google's native method to authenticate, and is the one we propose for a production setup.
Step-by-step key file creation guide (click to expand)
- In the Google Cloud Console, make sure you have selected the correct project from the project dropdown at the top.
- Navigate to IAM & Admin → Service Accounts (or search for
Service accountsin the top search bar). - Click on the service account you created for the integration.
- Go to the Keys tab → click Add Key → Create new key.
- Select JSON as the key type and click Create. This will download your new Service Account Key configuration file.
The Ocean Google Cloud integration uses Google's ADC (Application Default Credentials) with a service account key file.
Running the Helm command
The Ocean integration does not store the encoded file anywhere. It is saved locally, and is NOT sent to Port at any time.
-
Take the service account key file you create, and run this command:
Linux/Mac (Bash/Zsh):
cat <new-configuration-file> | base64 | pbcopyPowerShell:
[Convert]::ToBase64String([System.IO.File]::ReadAllBytes("<new-configuration-file>")) | Set-ClipboardWindows Command Prompt (creates a file: new-configuration-file.b64):
certutil -encode <new-configuration-file> new-configuration-file.b64 && type new-configuration-file.b64 | clip -
Run the following command:
helm repo add --force-update port-labs https://port-labs.github.io/helm-chartshelm upgrade --install gcp port-labs/port-ocean \--set port.clientId="$PORT_CLIENT_ID" \--set port.clientSecret="$PORT_CLIENT_SECRET_ID" \--set port.baseUrl="https://api.port.io" \--set initializePortResources=true \--set sendRawDataExamples=true \--set scheduledResyncInterval=1440 \--set integration.identifier="ocean-gcp-integration" \--set integration.type="gcp" \--set integration.eventListener.type="POLLING" \--set integration.config.encodedADCConfiguration="<paste_the_encoded_file_content_here>"
Optional - Scale permissions for a service account
The above gives a service account permissions in its project's scope. You can add permissions at other projects, folders, or organization level.
Step-by-step guide to scale permissions (click to expand)
- In the Google Cloud Console, navigate to IAM & Admin → Service Accounts and click on the service account you created for the integration.
- Copy the service account's email address.
- Switch the project dropdown at the top to your organization.
- Navigate to IAM & Admin → Manage Resources (or search for
Manage resourcesin the top search bar). - In the Resources table, select the desired scope (organization, folders, or projects) using the checkboxes on the left.
- A permissions panel will open on the right side. Click Add Principal.
- Paste the service account email and assign the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Save.
The Ocean integration does not store the encoded file anywhere. It is saved locally, and is NOT sent to Port at any time.
-
Take the service account key file you create, and run this command:
Linux/Mac (Bash/Zsh):
cat <new-configuration-file> | base64 | pbcopyPowerShell:
[Convert]::ToBase64String([System.IO.File]::ReadAllBytes("<new-configuration-file>")) | Set-ClipboardWindows Command Prompt (creates a file: new-configuration-file.b64):
certutil -encode <new-configuration-file> new-configuration-file.b64 && type new-configuration-file.b64 | clip
- GitHub Actions
Make sure to configure the following Github Secrets:
-
Port API credentials, you can check out the Port API documentation.
OCEAN__PORT_CLIENT_IDOCEAN__PORT_CLIENT_SECRET
-
Google Cloud Service Account Key
OCEAN__SECRET__ENCODED_ADC_CONFIGURATION- Paste the encoded file content here.
Here is an example for gcp-integration.yml workflow file:GitHub Actions workflow (click to expand)
Running the Docker command
The Ocean integration doesn't store the encoded file anywhere but locally. It's NOT being sent to Port.
-
Take the service account key file you create, and run this command:
Linux/Mac (Bash/Zsh):
cat <new-configuration-file> | base64 | pbcopyPowerShell:
[Convert]::ToBase64String([System.IO.File]::ReadAllBytes("<new-configuration-file>")) | Set-ClipboardWindows Command Prompt (creates a file: new-configuration-file.b64):
certutil -encode <new-configuration-file> new-configuration-file.b64 && type new-configuration-file.b64 | clip -
Run the following command:
docker run -i --rm --platform=linux/amd64 \-e OCEAN__PORT__CLIENT_ID="$PORT_CLIENT_ID" \-e OCEAN__PORT__CLIENT_SECRET="$PORT_CLIENT_SECRET" \-e OCEAN__PORT__BASE_URL="https://api.port.io" \-e OCEAN__INITIALIZE_PORT_RESOURCES=true \-e OCEAN__SEND_RAW_DATA_EXAMPLES=true \-e OCEAN__EVENT_LISTENER='{"type": "ONCE"}' \-e OCEAN__INTEGRATION__CONFIG__ENCODED_ADC_CONFIGURATION="<paste_the_encoded_file_content_here>" \ghcr.io/port-labs/port-ocean-gcp:latest
Optional - Scale permissions for a service account
The above gives a service account permissions in its project's scope. You can add permissions at other projects, folders, or organization level.
Step-by-step guide to scale permissions (click to expand)
- In the Google Cloud Console, navigate to IAM & Admin → Service Accounts and click on the service account you created for the integration.
- Copy the service account's email address.
- Switch the project dropdown at the top to your organization.
- Navigate to IAM & Admin → Manage Resources (or search for
Manage resourcesin the top search bar). - In the Resources table, select the desired scope (organization, folders, or projects) using the checkboxes on the left.
- A permissions panel will open on the right side. Click Add Principal.
- Paste the service account email and assign the following roles:
BrowserCloud Asset ViewerPub/Sub Viewer
- Click Save.
Installation walkthrough
Run the following commands. Make sure to replace <placeholders> with actual values.
If you want the integration to collect resources from more projects/folders/organization, make sure that you set up permissions properly.
Terraform installation command (click to expand)
echo 'provider "google-beta" { user_project_override = true }
provider "google" { user_project_override = true }
module "gcp" {
source = "port-labs/integration-factory/ocean//examples/gcp_cloud_run"
version = ">=0.0.31"
port_client_id = "<your_port_client_id>"
port_client_secret = "<your_port_client_secret>"
gcp_ocean_integration_image = "<your_artifact_registry_/_dockerhub>/port-ocean-gcp:<your_version>"
gcp_organization = "<your_gcp_organization>"
gcp_ocean_setup_project = "<your_gcp_project>"
gcp_included_projects = ["<your_gcp_project>"] # The Project list that the integration digests resources from.
gcp_project_filter = "<filter>" # The filter string used to retrieve GCP projects, allowing complex filtering by combining multiple conditions with logical operators (AND | OR).
integration_identifier = "gcp"
scheduled_resync_interval = 1440
event_listener = {
type = "POLLING"
}
}' > main.tf
# Initializing Terraform and Providers
terraform init
# Creating the resources (Cloud Run, PubSub Topic + Subscription, Cloud Assets Feed, Service account + Role)
terraform apply
Configuration options
The Port GCP integration's Terraform module offers a set of configurations:
Terraform configuration options (click to expand)
| Configuration | Default value | Required | Description |
|---|---|---|---|
port_client_id | True | The Port client id. | |
port_client_secret | True | The Port client secret. | |
gcp_organization | True | Your Google Cloud Organization Id. | |
gcp_ocean_setup_project | True | The project to create all the integration's infrastructure (Topic, Subscription, Service account etc.) on. | |
gcp_ocean_integration_image | True | The Artifact Registry / Dockerhub image to deploy. | |
integration_identifier | True | The integration's identifier in Port. | |
port_base_url | https://api.port.io | False | The Port base URL. |
gcp_included_projects | [] | False | The projects list you want the integration to collect from. If left empty, it will collect all projects in the organization. This option will be deprecated soon. |
gcp_cloud_run_cpu | 2 | False | The CPU limit for the Cloud Run service. |
gcp_cloud_run_memory | 1024Mi | False | The memory limit for the Cloud Run service. |
gcp_project_filter | False | The filter string used to retrieve GCP projects. Follows GCP's filter expressions syntax. | |
gcp_excluded_projects | [] | False | The projects list you want the integration NOT to collect from. Overridden by gcp_included_projects. This option will be deprecated soon. |
assets_types_for_monitoring | See default | False | The list of asset types the integration will digest real-time events for. |
ocean_integration_service_account_permissions | See default | False | The permissions granted to the integration's service account. We recommend not changing it. |
assets_feed_topic_id | ocean-integration-topic | False | The name of the topic created to receive real-time events. |
assets_feed_id | ocean-gcp-integration-assets-feed | False | The ID for the Ocean GCP integration feed. |
service_account_name | ocean-service-account | False | The name of the service account used by the Ocean integration. |
role_name | OceanIntegrationRole | False | The name of the role created for the integration's service account. |
gcp_ocean_integration_cloud_run_location | europe-west1 | False | Location in which the Cloud Run will run. |
environment_variables | [] | False | List of environment variables set to the Cloud Run job. We recommend not changing this variable. |
initialize_port_resources | True | False | Boolean to initialize Port resources. |
event_listener | Polling | False | Port's event listener configurations. |
integration_version | latest | False | The version of the integration to deploy. |
integration_type | gcp | False | The type of the integration. |
scheduled_resync_interval | 1440 | False | The interval to resync the integration (in minutes). |
ocean_service_account_custom_roles | [] | False | A list of custom roles to grant the integration's service account across available projects and the setup project. |
Optional - Project filtering
You have the option to specify which projects are included or excluded for real-time events. This can be particularly useful when you have a large number of projects and want to target specific ones based on certain criteria.
The variables gcp_included_projects and gcp_excluded_projects are deprecated and will be removed in future releases. We recommend using the gcp_project_filter variable for project filtering moving forward.
You can use the following three filtering strategies together:
gcp_excluded_projectsgcp_included_projectsgcp_project_filter
However, please note the priority conditions when using them simultaneously.
Priority conditions
You can use all three filtering strategies together, but it's important to understand how they interact. The following priority conditions apply:
-
gcp_included_projects (Highest Priority):
- When specified, only the projects listed in
gcp_included_projectsare included. - All other filters (
gcp_excluded_projectsandgcp_project_filter) are ignored. - Use this when you have a specific list of projects to include, regardless of other criteria.
- When specified, only the projects listed in
-
gcp_excluded_projects:
- If
gcp_included_projectsis not specified butgcp_excluded_projectsis provided, all projects are included except those listed. - The
gcp_project_filteris still applied, further refining the included projects.
- If
-
gcp_project_filter:
- If neither
gcp_included_projectsnorgcp_excluded_projectsare specified, andgcp_project_filteris provided, only projects matching the filter criteria are included. - This allows for flexible and complex filtering using GCP's native filtering syntax.
- If neither
-
Default Behavior (Lowest Priority):
- If none of the above variables are specified, all projects in your GCP organization are included by default.
Optional - Scaling the permissions
If you want the integration to collect resources from multiple projects/folders or to have it collect from the entire organization, you need to have permissions to create/view additional resources.
Step-by-step guide to scale Terraform permissions (click to expand)
- In the Google Cloud Console, switch the project dropdown to your organization.
- Navigate to IAM & Admin → Roles and click Create Role.
- Click Add Permissions and select all the permissions from the above section.
- Navigate to IAM & Admin → Manage Resources (or search for
Manage resourcesin the top search bar). - In the Resources table, select the desired scope (organization, folders, or projects) using the checkboxes on the left.
- A permissions panel will open on the right side. Click Add Principal.
- Enter your or your service account's email, and assign the newly created role.
Configuration
Port integrations use a YAML mapping block to ingest data from the third-party api into Port.
The mapping makes use of the JQ JSON processor to select, modify, concatenate, transform and perform other operations on existing fields and values from the integration API.
Default mapping configuration
This is the default mapping configuration you get after installing the GCP integration.
Default mapping configuration (click to expand)
resources:
- kind: cloudresourcemanager.googleapis.com/Project
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
blueprint: '"gcpProject"'
title: .display_name
properties:
labels: .labels
- kind: container.googleapis.com/Cluster
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
- kind: cloudfunctions.googleapis.com/CloudFunction
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
- kind: pubsub.googleapis.com/Topic
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
- kind: compute.googleapis.com/Disk
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
- kind: pubsub.googleapis.com/Subscription
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
- kind: iam.googleapis.com/ServiceAccount
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name | split("/") | last
blueprint: '"gcpCloudResource"'
properties:
type: .name | split("/") | .[-2]
location: .location
labels: .labels
relations:
project: .__project.name
Monitoring and sync status
To learn more about how to monitor and check the sync status of your integration, see the relevant documentation.
Mapping & selectors per resource
Use the explorer below to view the default mapping and blueprint definitions for each resource type.
{
"identifier": "gcpProject",
"title": "GCP Project",
"icon": "GoogleCloud",
"schema": {
"properties": {}
},
"mirrorProperties": {},
"calculationProperties": {},
"aggregationProperties": {}
}
- kind: cloudresourcemanager.googleapis.com/Project
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
blueprint: '"gcpProject"'
title: .display_name
properties:
labels: .labels
Below are the supported selectors for this resource. Use them under the selector key in your integration mapping configuration to control which resources are returned from the API.
| Selector | Type | Description |
|---|---|---|
exportEnvVariables | array | Environment variable names whose values should be included in lakehouse ingest payloads for DSP processing. |
preserveApiResponseCaseStyle | boolean | Controls whether to preserve the Google Cloud API's original field format instead of using protobuf's default snake case. When False (default): Uses protobuf's default snake_case format (existing behavior). When True: Preserves the specific API's original format (e.g., camelCase for PubSub). If not set, defaults to False to maintain existing behavior (snake_case for all APIs).Note that this setting does not affect resources fetched from the cloud asset API |
Further examples
Refer to the examples page for practical configurations and their corresponding blueprint definitions.