> For the complete documentation index, see llms.txt.
Skip to main content

Check out Port for yourself ➜ 

Setup & configuration

This guide covers all technical details for setting up and configuring LLM providers, including permissions, changing defaults, validation flow, and troubleshooting common issues.

Permissions & access control​

Admin access required

Managing LLM provider settings requires organization administrator permissions. Only admins can modify default providers or add new provider configurations.

Administrators can perform all LLM provider management operations:

Configuration Operations

Management Capabilities

  • Set organization-wide default providers and models
  • Configure provider-specific settings and credentials
  • Manage provider access and permissions
  • Test provider connections with validation

Prerequisites​

Before configuring LLM providers, ensure you have:

  1. Access to Port AI: Your organization has access to the Port AI features.
  2. Provider Accounts: Active accounts with the LLM providers you want to use
  3. Admin Permissions: Organization administrator role in Port

Some providers require additional setup before you can configure them in Port. See Step 1: Configure provider policies and settings for provider-specific configuration instructions.

Step 1: Configure provider policies and settings (optional)​

Some providers require additional setup before you can register them in Port.

Step 2: Store API keys in secrets​

Before configuring providers, store your API keys in Port's secrets system. The secret names you choose are flexible - you'll reference them in your provider configuration.

One-time view

After creating a secret, you will be able to view its value only once. Afterwards, you will be able to delete the secret or edit its value, but not to view it.

  1. Open the Credentials modal.
  2. Click on the Secrets tab.
  3. Click on + Secret and add the required secrets for your chosen provider(s):

Required Secret:

  • API Key secret (e.g., openai-api-key) - Your OpenAI API key
Secret naming flexibility

You can choose any names for your secrets. The examples above are suggestions - use names that make sense for your organization. You will reference these exact names in your provider configuration.

For more details on managing secrets, see the Port Secrets documentation.

Step 3: Configure LLM providers​

Use the Create or connect an LLM provider API to configure your providers. The interactive API reference provides detailed examples and allows you to test the configuration for each provider type (OpenAI, Anthropic, Azure OpenAI, Azure Anthropic, AWS Bedrock, OpenAI compatible).

After configuration

Once providers are configured, you can view and select default providers and models through the UI (Builder → Organization Settings → AI tab) or continue using the API for all operations.

Model overrides​

By default, Port tests all supported models when you register a provider. If your account only has access to a subset of models, the validation will fail for the ones you don't have, and the entire registration will be rejected.

To register a provider with only specific models, use the overrides field to explicitly enable the models you want and disable the rest:

Example: Bedrock with selective model enablement (click to expand)
curl -s -X POST 'https://api.port.io/v1/llm-providers?validate_connection=true' \
  -H "Authorization: Bearer $PORT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "bedrock",
    "enabled": true,
    "config": {
      "roleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/<YOUR_ROLE_NAME>",
      "region": "eu-central-1",
      "externalIdSecretName": "BEDROCK_ROLE_EXTERNAL_ID"
    },
    "overrides": {
      "models": {
        "claude-haiku-4-5-20251001": {
          "enabled": true
        },
        "claude-sonnet-4-5-20250929": {
          "enabled": true
        },
        "claude-sonnet-4-20250514": {
          "enabled": false
        },
        "claude-opus-4-5-20251101": {
          "enabled": false
        },
        "claude-opus-4-6": {
          "enabled": false
        }
      }
    }
  }'
All unsupported models must be explicitly disabled

Setting only the models you want to enabled: true is not enough. Models you don't have access to must be explicitly set to enabled: false, otherwise Port will still attempt to validate them and the registration will fail.

Step 4: Validate configuration​

Test your provider configuration with connection validation using the Create or connect an LLM provider API with the validate_connection=true parameter. The interactive API reference shows how to test your configuration before saving it.

Getting your current configuration​

You can view your organization's current LLM provider defaults through the UI or API:

Using the UI:

  1. Go to Builder → Organization Settings → AI tab.
  2. View all configured providers and models.
  3. See which provider and model are currently set as defaults.

Using the API: Retrieve your organization's current LLM provider defaults using the Get default LLM provider and model API. The interactive API reference shows the response format and allows you to test the endpoint.

System defaults​

When no organization-specific defaults are configured, Port uses these system defaults:

  • Default Provider: port
  • Default Model: claude-sonnet-4-5-20250929

Changing default providers​

You can change your organization's default LLM provider and model through the UI or API:

Using the UI:

  1. Go to Builder → Organization Settings → AI tab.
  2. Select your preferred Default LLM provider from the dropdown.
  3. Select your preferred Default model from the dropdown.
  4. Click Save to apply your changes.
Adding new providers

To add a new custom LLM provider, you still need to use the Create or connect an LLM provider API. Once a provider is configured, it will appear in the UI dropdown for selection.

Using the API: Update your organization's default LLM provider and model using the Change default LLM provider and model API. The interactive API reference provides the request format and response examples.

Validation flow​

The system validates provider configurations to ensure they work correctly before saving. This includes checking credentials, testing connections, and verifying model availability.

For detailed information about how validation works during API requests, see Selecting LLM Provider.

Configuration hierarchy​

LLM provider settings follow a hierarchy from organization defaults to system defaults.

For detailed information about how defaults are selected during API requests, see Selecting LLM Provider.

FAQ​