AI Model Integrations

AI model integrations connect your workspace to large language model providers. These integrations power LLM Response nodes, RAG Query nodes, and other AI-driven features in your bot flows.

Supported Providers

Provider	Models	Authentication
Anthropic	Claude Opus, Claude Sonnet, Claude Haiku	API key
OpenAI	GPT-4o, GPT-4, GPT-3.5 Turbo	API key
Google Vertex AI	Gemini Pro, Gemini Flash	GCP service account

Configuring Anthropic (Claude)

1. Go to Settings > Integrations > Add Integration.
2. Select AI Model as the type, then choose Anthropic.
3. Enter a label (e.g., "Anthropic Production").
4. Paste your API key from the Anthropic Console.
5. Select a default model (e.g., Claude Sonnet).
6. Click Test Connection, then Save.

Configuring OpenAI (GPT)

1. Go to Settings > Integrations > Add Integration.
2. Select AI Model as the type, then choose OpenAI.
3. Enter a label (e.g., "OpenAI Development").
4. Paste your API key from the OpenAI dashboard.
5. Optionally enter an Organization ID if your account belongs to an organization.
6. Select a default model (e.g., GPT-4o).
7. Click Test Connection, then Save.

Configuring Google Vertex AI (Gemini)

1. Go to Settings > Integrations > Add Integration.
2. Select AI Model as the type, then choose Vertex AI.
3. Enter a label (e.g., "Vertex AI Production").
4. Enter your GCP Project ID.
5. Enter the region (e.g., us-central1).
6. Upload or paste your service account JSON key.
7. Select a default model (e.g., Gemini Pro).
8. Click Test Connection, then Save.

imageAI provider configuration form showing fields for label, API key, default model selector, and test connection button, with Anthropic Claude selected as the provider

AI model integration configuration

TIP

For Vertex AI, the service account needs the Vertex AI User role in your GCP project. You can create a dedicated service account with minimal permissions for better security.

Setting the Default Model

Each workspace has a default AI model that is used when a flow node does not specify a particular integration.

1. Go to Settings > Integrations.
2. Find the AI model integration you want as the default.
3. Click the three-dot menu and select Set as Default.

The default integration is indicated with a badge in the integrations list.

Per-Node Override

Individual flow nodes can override the workspace default by selecting a specific integration. In the flow builder:

1. Select an LLM Response or RAG Query node.
2. In the configuration panel, find the AI Integration dropdown.
3. Choose the desired integration from the list.

This allows you to use different models for different tasks within the same flow. For example, use a faster, cheaper model for simple classification and a more capable model for complex responses.

Rate Limits

You can configure rate limits on each AI model integration to control usage and costs.

Setting	Description
Requests per minute	Maximum API calls per minute across all bots using this integration
Tokens per minute	Maximum total tokens (input + output) per minute
Monthly budget	Optional spending cap that disables the integration when reached

WARNING

When a rate limit is reached, bot nodes using that integration return a fallback response or an error, depending on your flow design. Set rate limits conservatively at first and adjust based on actual usage patterns.

Cost Tracking

OmniBots tracks token usage and estimated costs for each AI model integration. To view usage:

1. Go to Settings > Integrations.
2. Click on an AI model integration.
3. Select the Usage tab.

The usage view shows daily token consumption, request counts, and estimated costs broken down by model. Use this data to optimize model selection and manage spending.

imageToken usage and cost tracking chart showing daily token consumption bars with a cost trend line overlay, broken down by AI model

AI model usage and cost tracking

Next Steps

• Connect storage providers for Knowledge Base document sources
• Set up CCaaS platforms for live agent handoff
• Learn how to use AI integrations in flow builder nodes