LLM Response Node

The LLM Response node sends the user's message (along with conversation history and a system prompt) to a large language model and stores the generated response in a variable. This is the primary node for AI-driven conversations.

When to Use

• You want the bot to generate a natural-language answer to the user's question.
• You need the LLM to reason about context, summarize information, or follow complex instructions.
• You are building a hybrid flow where some paths use scripted messages and others use AI responses.

Configuration

Property	Description	Default
Integration	The AI model integration to use (e.g., Gemini 2.0 Flash, Claude Sonnet, GPT-4o). Selected via the integration selector.	Tenant default AI model
System Prompt	Instructions that tell the LLM how to behave. Supports {{variable}} interpolation.	Empty
Temperature	Controls randomness. Lower values (0.0 - 0.3) produce deterministic answers; higher values (0.7 - 1.0) produce more creative responses.	0.7
Max Tokens	Maximum number of tokens the LLM can generate in its response.	1024
Output Variable	The variable name where the generated text is stored.	llm_response

AI Prompt Management

Instead of writing the system prompt inline, you can select a Managed Prompt from the AI Prompts library. Managed prompts are version-controlled and can be shared across multiple nodes and bots.

Per-Node Integration Selection

Each LLM Response node can use a different AI integration. This allows you to use a fast, low-cost model for simple responses and a more capable model for complex reasoning -- all within the same flow.

If no integration is selected, the node falls back to the tenant's default AI model configured in Settings > AI Models.

imageLLM Response node config panel showing integration selector dropdown with AI model options, system prompt text area, temperature slider, max tokens field, and output variable name

LLM Response configuration with integration selector

Agentic Mode

When Agentic Mode is enabled, the LLM can call tools during its response generation. The node will loop -- sending tool results back to the LLM -- until the LLM produces a final text response or reaches the iteration limit.

Property	Description	Default
Agentic Mode	Enable tool calling within this node.	true
Enabled Tools	Select which tools the LLM can call. Tools are defined in the flow's tool library or in the global Tools section.	None
Max Iterations	Maximum number of tool-call loops before the node stops.	5
Max Iterations Message	Message to send if the iteration limit is reached without a final response.	Empty

imageLLM Response node agentic mode section showing the tool calling toggle enabled, a multi-select list of available tools with checkboxes, and max iterations field

Agentic mode with tool selection

Knowledge Base Integration

Toggle Use RAG to attach one or more Knowledge Bases to this node. When enabled, the node retrieves relevant document chunks before generating a response, grounding the LLM's answer in your documents.

TIP

For dedicated RAG workflows with more control over retrieval settings, use the KB Search node instead of the built-in RAG toggle.

System Prompt Tips

Use {{variable}} syntax to inject session context into the system prompt:

You are a support agent for {{company_name}}.
The customer's name is {{user_name}} and their account tier is {{user_tier}}.
Answer questions about their account. Be concise and professional.

WARNING

Avoid placing sensitive data (API keys, passwords) in the system prompt. Use tools or API Call nodes for operations that require credentials.

Output

The node writes the LLM's text response to the configured output variable. Use this variable in a subsequent Message node to send the response to the user, or in a Condition node to branch based on the content.

Handles

Handle	Direction	Description
Input	In	Receives execution from the previous node
Output	Out	Continues to the next node after the response is generated