LLM provider management
Manage and configure the Large Language Model (LLM) providers that power all AI interactions in Port. This feature gives you control over which AI models are used across Port AI Assistant, AI Agents, and other AI-powered features.
LLM approach overview
Port offers two approaches for AI model usage:
Port's managed AI infrastructure (default)
Port provides state-of-the-art LLMs through our secured cloud infrastructure by default:
- No data training: We never train models on your data
- Enterprise security: Your data is processed securely and not stored
- Automatic updates: Always access to the latest model versions
- Zero setup: Works immediately with no configuration required
Learn more about our data controls and security measures.
Bring your own LLM
For organizations requiring additional control, Port also supports configuring your own LLM providers:
- Data privacy: Keep AI processing within your own infrastructure
- Compliance: Meet specific regulatory or organizational requirements
Supported LLMs and providers
Port supports the following LLM providers and models:
| Provider | Supported models |
|---|---|
| OpenAI | gpt-5, gpt-5.6-luna, gpt-5.6-terra |
| Anthropic | claude-haiku-4-5-20251001, claude-sonnet-4-20250514, claude-sonnet-4-5-20250929, claude-sonnet-4-6, claude-sonnet-5, claude-sonnet-5-5, claude-opus-4-5-20251101, claude-opus-4-6 |
| Azure OpenAI | gpt-5, gpt-5.6-luna, gpt-5.6-terra |
| Azure Anthropic | claude-haiku-4-5-20251001, claude-sonnet-4-20250514, claude-sonnet-4-5-20250929, claude-sonnet-4-6, claude-sonnet-5, claude-sonnet-5-5, claude-opus-4-5-20251101, claude-opus-4-6 |
| AWS Bedrock | claude-haiku-4-5-20251001, claude-sonnet-4-20250514, claude-sonnet-4-5-20250929, claude-sonnet-4-6, claude-sonnet-5, claude-sonnet-5-5, claude-opus-4-5-20251101, claude-opus-4-6 |
| Vertex Gemini ( vertex-gemini) | You choose the model names. See the Vertex Gemini setup guide. |
Vertex Anthropic (vertex-anthropic) | You choose the model names. See the Vertex Anthropic setup guide. |
OpenAI compatible (openai-compatible) | You choose the model names. See the OpenAI compatible setup guide for a full walkthrough using LiteLLM as the example gateway, or use the Create or connect an LLM provider API for the exact JSON schema. |
Anthropic compatible (anthropic-compatible) | You choose the model names. See the Anthropic compatible setup guide for gateway configuration, or use the Create or connect an LLM provider API for the exact JSON schema. |
Port AI leverages claude-sonnet-5-5 by default when no custom provider is configured.
How this affects Port AI features
All Port AI features rely on the configured default providers. When working directly with the API, you can even select which provider and model to use for specific requests.
FAQ
- Zero setup: Works immediately with no configuration required.
- Automatic updates: Always access to the latest model versions.
- Enterprise security: Your data is processed securely and not stored.
- No data training: We never train models on your data.
- Reliable performance: Optimized for Port's AI features.
This is the recommended approach for most organizations as it provides the best balance of security, performance, and ease of use.
Consider bringing your own LLM provider when you need:
- Enhanced data privacy: Keep AI processing within your own infrastructure.
- Compliance requirements: Meet specific regulatory or organizational requirements.
- Custom models: Define custom configuration on models not available through Port's managed infrastructure.
- Integration requirements: Connect with existing AI infrastructure.
Port's Managed Infrastructure:
- Data is processed within Port's secure cloud infrastructure.
- Your data is not used for model training.
- Complete logical separation between different customers' data.
- Data processing occurs within Port's controlled environment.
Bring Your Own LLM:
- Data is processed by your chosen LLM provider.
- You control where and how your data is processed.
- Must comply with your provider's data handling policies.
- Port may still retain AI inference data for up to 90 days with LLM providers and sub-processors (separate from AI invocation audit records in your organization, which cannot be deleted).
Detailed Information:
- How is my data processed by LLM providers? - How your data is processed by AI models.
- Does Port still retain AI interaction data when using bring-your-own LLM? - Data handling with custom LLM providers.
- Data privacy and retention - What data we store and why.
When using your own LLM provider, you maintain the same security controls as Port's managed infrastructure:
- RBAC compliance: Port AI still respects your organization's access controls.
- Data governance: All interactions respect your configured data access policies.
- Audit trail: AI interactions are still logged and trackable.
- Permission inheritance: Port AI cannot access data you don't have permission to view.
Additional considerations:
- You're responsible for your LLM provider's security measures.
- Ensure your provider meets your compliance requirements.
- Monitor your provider's data handling practices.
Detailed Information:
- Data access and permissions - How AI respects your organization's access controls.
- Compliance and security standards - Integration with compliance requirements.
Yes, you can opt out of data storage even when using your own LLM provider. However, there are important considerations:
What you can opt out of:
- Retaining AI inference data with Port's LLM providers and sub-processors for up to 90 days.
Impact of opting out:
- May impact Port's ability to provide support and troubleshoot issues.
- Could affect AI feature performance and reliability.
Detailed Information:
- Can I opt out of data storage? - How to opt out of data storage.
- Does Port still retain AI interaction data when using bring-your-own LLM? - Data handling with custom LLM providers.
To configure your own LLM providers:
- Configure your providers - Set up your preferred LLM providers using the Create or connect an LLM provider API endpoint.
- Select defaults - Once providers are configured, you can view and select default providers and models through the UI (click your profile picture, select Settings, then open the AI tab) or via the Change default LLM provider and model API.
- Viewing and selecting defaults: Available in both UI and API.
- Adding new custom providers: Requires the API.
This feature is designed for organizations with specific compliance, privacy, or integration requirements that cannot be met by Port's managed infrastructure.
When you switch LLM providers:
- Existing AI invocations: All previous interactions remain accessible and unchanged.
- New interactions: Will use your newly configured provider.
- Data continuity: No data is lost during the transition.
- Performance: Response times and capabilities may change based on your new provider.
Important considerations:
- Test your new provider configuration thoroughly.
- Monitor performance and adjust settings as needed.
- Ensure your new provider meets your security and compliance requirements.