Skip to content

Architecture Overview ​

This page describes the complete technical architecture of OmniBots, covering every cloud service, microservice, data store, and external integration that makes up the platform.

High-Level Architecture Diagram ​

imageHigh-level architecture diagram showing client portals, GCP edge layer, Cloud Run microservices, data layer, operations layer, and AI/ML layer
OmniBots platform architecture on Google Cloud
┌─────────────────────────────────────────────────────────────────────────────────┐
│                              INTERNET / CLIENTS                                 │
│                                                                                 │
│   ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐       │
│   │  Operations   │  │ Tenant Admin │  │ Super Admin  │  │  Chat Widget │       │
│   │   Portal      │  │   Portal     │  │   Portal     │  │  (Embedded)  │       │
│   │  (Vue 3/Vite) │  │  (Vue 3/Vite)│  │ (Vue 3/Vite) │  │ (Vue 3/Vite) │       │
│   └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘       │
│          │                  │                  │                  │               │
│   ┌──────┴──────────────────┴──────────────────┴──────────────────┘               │
│   │                    HTTPS / WSS                                                │
└───┼──────────────────────────────────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│                        GOOGLE CLOUD PLATFORM (GCP)                              │
│                                                                                 │
│  ┌──────────────────────────────────────────────────────────────────────────┐   │
│  │  EDGE LAYER                                                              │   │
│  │  ┌─────────────────┐  ┌──────────────────┐  ┌────────────────────┐      │   │
│  │  │ Global HTTP(S)   │  │  Cloud Armor     │  │   Cloud CDN        │      │   │
│  │  │ Load Balancer    │──│  (WAF)           │  │   (Widget Assets)  │      │   │
│  │  │ (SSL termination │  │  - OWASP Top 10  │  │   - CACHE_ALL_     │      │   │
│  │  │  multi-region)   │  │  - Rate limiting │  │     STATIC         │      │   │
│  │  └────────┬─────────┘  │  - Bot mgmt      │  │   - Edge caching   │      │   │
│  │           │            │  - Geo blocking   │  │   - Compression    │      │   │
│  │           │            │  - IP allowlist   │  └────────────────────┘      │   │
│  │           │            └──────────────────┘                               │   │
│  └───────────┼──────────────────────────────────────────────────────────────┘   │
│              │                                                                   │
│              ▼                                                                   │
│  ┌──────────────────────────────────────────────────────────────────────────┐   │
│  │  VPC NETWORK  (omnibots-vpc)                                             │   │
│  │  ┌────────────────────────────┐    ┌──────────────────────────────────┐  │   │
│  │  │ Serverless VPC Connector   │    │  Private Service Access          │  │   │
│  │  │ (Cloud Run ↔ VPC)          │    │  (VPC Peering for Cloud SQL)     │  │   │
│  │  └────────────┬───────────────┘    └──────────────────────────────────┘  │   │
│  │               │                                                          │   │
│  │  ┌────────────┴──────────────────────────────────────────────────────┐   │   │
│  │  │                     CLOUD RUN SERVICES                            │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────┐   PUBLIC ENTRY POINTS                        │   │   │
│  │  │  │  API Gateway     │   Port 8000 │ 512Mi │ 1 CPU                  │   │   │
│  │  │  │  (FastAPI)       │   min: 1, max: 10 │ concurrency: 100         │   │   │
│  │  │  │  Routes all      │   Session affinity (WebSocket)               │   │   │
│  │  │  │  HTTP + WS       │   512Mi max request body (file uploads)      │   │   │
│  │  │  └────────┬─────────┘                                              │   │   │
│  │  │           │                                                         │   │   │
│  │  │  ┌────────┴───────────────────────────────────────────────────┐    │   │   │
│  │  │  │                 INTERNAL SERVICES (private)                 │    │   │   │
│  │  │  │                                                             │    │   │   │
│  │  │  │  CORE                          INTELLIGENCE                 │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ Auth Service    │           │ Orchestrator    │         │    │   │   │
│  │  │  │  │ :8001 │ 256Mi   │           │ :8003 │ 1Gi     │         │    │   │   │
│  │  │  │  │ JWT, MFA, SSO   │           │ 2 CPU │ LLM     │         │    │   │   │
│  │  │  │  │ RBAC, Anomaly   │           │ Flow Executor   │         │    │   │   │
│  │  │  │  │ IP reputation   │           │ WebSocket handler│         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ Bot Service     │           │ RAG Service     │         │    │   │   │
│  │  │  │  │ :8002 │ 256Mi   │           │ :8012 │ 512Mi   │         │    │   │   │
│  │  │  │  │ Bot CRUD, Flows │           │ Vector search   │         │    │   │   │
│  │  │  │  │ Tools, Templates│           │ pgvector queries│         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ Tenant Service  │           │ KB Service      │         │    │   │   │
│  │  │  │  │ :8008 │ 256Mi   │           │ :8004 │ 256Mi   │         │    │   │   │
│  │  │  │  │ Tenant/Partner  │           │ Knowledge Base  │         │    │   │   │
│  │  │  │  │ Settings, i18n  │           │ CRUD, documents │         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ User Service    │           │ Indexing Service │         │    │   │   │
│  │  │  │  │ :8011 │ 256Mi   │           │ :8013 │ 1Gi     │         │    │   │   │
│  │  │  │  │ User management │           │ 2 CPU │ Embedding│         │    │   │   │
│  │  │  │  │ Roles, Profiles │           │ Doc processing  │         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │                                                             │    │   │   │
│  │  │  │  CHANNELS                      BACKGROUND                   │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ CCaaS Service   │           │ Indexing Worker  │         │    │   │   │
│  │  │  │  │ :8005 │ 256Mi   │           │ 2Gi │ 2 CPU     │         │    │   │   │
│  │  │  │  │ Genesys, 8x8   │           │ Celery worker   │         │    │   │   │
│  │  │  │  │ Connect, CCAI   │           │ Doc chunking    │         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │  ┌─────────────────┐           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ Voice Service   │ (public)  │ Indexing Beat   │         │    │   │   │
│  │  │  │  │ :8006 │ 512Mi   │           │ 256Mi │ 500m CPU│         │    │   │   │
│  │  │  │  │ STT/TTS, Telnyx │           │ Celery Beat     │         │    │   │   │
│  │  │  │  │ Voice auth      │           │ (exactly 1)     │         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  │  ┌─────────────────┐                                        │    │   │   │
│  │  │  │  │ Conversation    │           AUXILIARY                     │    │   │   │
│  │  │  │  │ Service         │           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ :8011 │ 512Mi   │           │ Reporting Svc   │         │    │   │   │
│  │  │  │  │ Session mgmt    │           │ :8007 │ 512Mi   │         │    │   │   │
│  │  │  │  └─────────────────┘           │ Analytics, Usage│         │    │   │   │
│  │  │  │  ┌─────────────────┐           └─────────────────┘         │    │   │   │
│  │  │  │  │ Notification    │           ┌─────────────────┐         │    │   │   │
│  │  │  │  │ Service         │           │ Billing Service │         │    │   │   │
│  │  │  │  │ :8010 │ 256Mi   │           │ :8009 │ 256Mi   │         │    │   │   │
│  │  │  │  │ Push, Triggers  │           │ Usage metering  │         │    │   │   │
│  │  │  │  └─────────────────┘           └─────────────────┘         │    │   │   │
│  │  │  └─────────────────────────────────────────────────────────────┘    │   │   │
│  │  └────────────────────────────────────────────────────────────────────┘   │   │
│  │                                                                           │   │
│  │  ┌────────────────────────────────────────────────────────────────────┐   │   │
│  │  │  DATA LAYER                                                        │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────────┐   ┌──────────────────────┐              │   │   │
│  │  │  │ Cloud SQL            │   │ Memorystore          │              │   │   │
│  │  │  │ (PostgreSQL 15)      │   │ (Redis 7.0)          │              │   │   │
│  │  │  │                      │   │                      │              │   │   │
│  │  │  │ • pgvector extension │   │ Prod: STANDARD_HA    │              │   │   │
│  │  │  │ • Private IP only    │   │ (5 GB, HA failover)  │              │   │   │
│  │  │  │ • SSD storage        │   │                      │              │   │   │
│  │  │  │ • Auto-resize disk   │   │ Dev: BASIC (1 GB)    │              │   │   │
│  │  │  │ • Query Insights     │   │                      │              │   │   │
│  │  │  │ • PITR (prod)        │   │ Uses:                │              │   │   │
│  │  │  │ • Daily backups      │   │ • Session cache      │              │   │   │
│  │  │  │ • 500 connections    │   │ • Task queue broker  │              │   │   │
│  │  │  │   (prod)             │   │ • Rate limiting      │              │   │   │
│  │  │  │                      │   │ • Translation cache  │              │   │   │
│  │  │  │ Prod: REGIONAL HA    │   │ • Pub/Sub channels   │              │   │   │
│  │  │  │ Dev:  ZONAL          │   │                      │              │   │   │
│  │  │  └──────────────────────┘   └──────────────────────┘              │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────────────────────────────────────────────────┐  │   │   │
│  │  │  │ Cloud Storage Buckets                                        │  │   │   │
│  │  │  │                                                              │  │   │   │
│  │  │  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │  │   │   │
│  │  │  │  │ Documents    │  │ Knowledge    │  │ Recordings   │      │  │   │   │
│  │  │  │  │ Bot files,   │  │ RAG docs,    │  │ Voice calls  │      │  │   │   │
│  │  │  │  │ uploads      │  │ embeddings   │  │ Compliance   │      │  │   │   │
│  │  │  │  │ Versioned    │  │ Versioned    │  │ retention    │      │  │   │   │
│  │  │  │  │ Nearline→    │  │ Nearline→    │  │ locked (prod)│      │  │   │   │
│  │  │  │  │ Coldline     │  │ lifecycle    │  │              │      │  │   │   │
│  │  │  │  └──────────────┘  └──────────────┘  └──────────────┘      │  │   │   │
│  │  │  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │  │   │   │
│  │  │  │  │ Reports      │  │ Branding     │  │ Widget       │      │  │   │   │
│  │  │  │  │ Exported CSV │  │ Assets       │  │ Assets       │      │  │   │   │
│  │  │  │  │ 30-day TTL   │  │ Logos, icons │  │ JS/CSS bundle│      │  │   │   │
│  │  │  │  │              │  │ Public read  │  │ Public read  │      │  │   │   │
│  │  │  │  └──────────────┘  └──────────────┘  └──────────────┘      │  │   │   │
│  │  │  └──────────────────────────────────────────────────────────────┘  │   │   │
│  │  └────────────────────────────────────────────────────────────────────┘   │   │
│  │                                                                           │   │
│  │  ┌────────────────────────────────────────────────────────────────────┐   │   │
│  │  │  OPERATIONS LAYER                                                  │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────┐  ┌──────────────────┐  ┌────────────────┐  │   │   │
│  │  │  │ Secret Manager   │  │ Cloud Scheduler   │  │ Artifact       │  │   │   │
│  │  │  │                  │  │                    │  │ Registry       │  │   │   │
│  │  │  │ • JWT secret     │  │ • Daily metrics   │  │                │  │   │   │
│  │  │  │ • Encryption key │  │   aggregation     │  │ Docker images  │  │   │   │
│  │  │  │ • OAuth key      │  │   (2 AM daily)    │  │ for all Cloud  │  │   │   │
│  │  │  │ • DB password    │  │ • Conversation    │  │ Run services   │  │   │   │
│  │  │  │ • Anthropic key  │  │   cleanup (3 AM)  │  │                │  │   │   │
│  │  │  │ • OpenAI key     │  │ • Usage metering  │  │ us-docker.     │  │   │   │
│  │  │  │ • Telnyx key     │  │   (hourly)        │  │ pkg.dev        │  │   │   │
│  │  │  │ • Deepgram key   │  │                    │  │                │  │   │   │
│  │  │  │ • SendGrid key   │  │ OIDC auth to      │  │                │  │   │   │
│  │  │  │ • Genesys creds  │  │ Cloud Run         │  │                │  │   │   │
│  │  │  └──────────────────┘  └──────────────────┘  └────────────────┘  │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────┐  ┌──────────────────┐  ┌────────────────┐  │   │   │
│  │  │  │ Cloud Build      │  │ Cloud Logging     │  │ Cloud          │  │   │   │
│  │  │  │                  │  │                    │  │ Monitoring     │  │   │   │
│  │  │  │ CI/CD pipeline   │  │ Log sinks:        │  │                │  │   │   │
│  │  │  │ Build → Push →   │  │ • App → BigQuery  │  │ • Dashboards   │  │   │   │
│  │  │  │ Deploy to        │  │ • Errors → BQ     │  │ • Uptime checks│  │   │   │
│  │  │  │ Cloud Run        │  │ • All → GCS       │  │ • SLOs (prod)  │  │   │   │
│  │  │  │                  │  │   (archive)       │  │ • Alert policies│  │   │   │
│  │  │  │                  │  │ • Audit → BQ      │  │                │  │   │   │
│  │  │  │                  │  │ • Security → GCS  │  │ Channels:      │  │   │   │
│  │  │  │                  │  │                    │  │ Email, Slack,  │  │   │   │
│  │  │  │                  │  │ Log-based metrics: │  │ PagerDuty      │  │   │   │
│  │  │  │                  │  │ error_count,       │  │                │  │   │   │
│  │  │  │                  │  │ request_latency,   │  │                │  │   │   │
│  │  │  │                  │  │ auth_failures,     │  │                │  │   │   │
│  │  │  │                  │  │ bot_conversations  │  │                │  │   │   │
│  │  │  └──────────────────┘  └──────────────────┘  └────────────────┘  │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────────────────────────────────────────────────┐  │   │   │
│  │  │  │ BigQuery                                                     │  │   │   │
│  │  │  │ Log analytics dataset — partitioned tables for app logs,     │  │   │   │
│  │  │  │ error logs, and audit logs. 90-day retention default.        │  │   │   │
│  │  │  └──────────────────────────────────────────────────────────────┘  │   │   │
│  │  └────────────────────────────────────────────────────────────────────┘   │   │
│  │                                                                           │   │
│  │  ┌────────────────────────────────────────────────────────────────────┐   │   │
│  │  │  AI / ML LAYER                                                     │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────┐  ┌──────────────────┐                       │   │   │
│  │  │  │ Vertex AI        │  │ Dialogflow CX    │                       │   │   │
│  │  │  │                  │  │                    │                       │   │   │
│  │  │  │ • Gemini models  │  │ • NLU engine      │                       │   │   │
│  │  │  │ • Text embeddings│  │ • Intent matching │                       │   │   │
│  │  │  │   (RAG)          │  │ • Entity          │                       │   │   │
│  │  │  │ • Prediction API │  │   extraction      │                       │   │   │
│  │  │  └──────────────────┘  └──────────────────┘                       │   │   │
│  │  │                                                                    │   │   │
│  │  │  ┌──────────────────┐                                              │   │   │
│  │  │  │ Pub/Sub          │                                              │   │   │
│  │  │  │                  │                                              │   │   │
│  │  │  │ • CCAI webhook   │                                              │   │   │
│  │  │  │   events         │                                              │   │   │
│  │  │  │ • Async event    │                                              │   │   │
│  │  │  │   bus            │                                              │   │   │
│  │  │  └──────────────────┘                                              │   │   │
│  │  └────────────────────────────────────────────────────────────────────┘   │   │
│  │                                                                           │   │
│  │  IAM / Service Accounts:                                                  │   │
│  │  • omnibots-cloudrun-{env}  — Cloud Run services                          │   │
│  │    Roles: cloudsql.client, storage.objectAdmin, secretmanager.accessor,   │   │
│  │           aiplatform.user, dialogflow.client, pubsub.subscriber           │   │
│  │  • omnibots-scheduler-{env} — Cloud Scheduler                             │   │
│  │    Roles: run.invoker                                                     │   │
│  │  • omnibots-log-writer-{env} — Log writer to BigQuery                     │   │
│  │    Roles: bigquery.dataEditor                                             │   │
│  │                                                                           │   │
└──┴───────────────────────────────────────────────────────────────────────────┘

GCP Services Inventory ​

Compute & Networking ​

ServiceResourcePurpose
Cloud Run16 servicesServerless containers for all backend microservices
VPC Networkomnibots-vpc-{env}Private network isolating all services
Serverless VPC Connectoromnibots-connector-{env}Bridges Cloud Run to VPC (2-10 instances)
Global HTTP(S) Load BalancerExternal LBSSL termination, multi-region routing, health checks
Cloud Armoromnibots-waf-{env}WAF with OWASP CRS rules, rate limiting, geo-blocking
Cloud CDNWidget + Frontend backendsEdge caching for static widget and portal assets
Compute NetworkFirewall rules, Private Service AccessInternal traffic rules, VPC peering for Cloud SQL

Data & Storage ​

ServiceResourcePurpose
Cloud SQLPostgreSQL 15 (omnibots-postgres-{env})Primary relational database with pgvector for embeddings
MemorystoreRedis 7.0 (omnibots-redis-{env})Session cache, Celery broker, rate limiting, translation cache
Cloud Storage6 bucketsDocuments, knowledge base files, voice recordings, reports, branding assets, widget assets
BigQueryomnibots_logs_{env} datasetLog analytics — partitioned tables for app, error, and audit logs

Security & Secrets ​

ServiceResourcePurpose
Secret Manager10+ secretsJWT key, encryption keys, DB password, API keys (Anthropic, OpenAI, Telnyx, Deepgram, SendGrid, Genesys)
IAM3 service accountsLeast-privilege access for Cloud Run, Scheduler, and Log Writer
Cloud Armor WAF8+ security rulesSQLi, XSS, LFI, RFI, RCE protection; rate limiting (1000 req/min API, 20 req/min auth)

AI & ML ​

ServiceResourcePurpose
Vertex AIAI Platform APIGemini models for LLM orchestration, text embeddings for RAG
Dialogflow CXAgent + IntentsNLU engine for intent matching and entity extraction
Pub/SubTopics + SubscriptionsAsync event bus for CCAI webhook events

Operations ​

ServiceResourcePurpose
Cloud Scheduler3 cron jobsDaily metrics aggregation (2 AM), conversation cleanup (3 AM), hourly usage metering
Cloud BuildCI/CD pipelineBuild Docker images, push to Artifact Registry, deploy to Cloud Run
Artifact Registryus-docker.pkg.devDocker container image registry for all services
Cloud MonitoringDashboards, uptime checks, SLOsService health monitoring, availability SLO (99.5%), latency SLO (95%)
Cloud Logging5 log sinks, 4 log-based metricsApp/error logs to BigQuery, all logs to GCS archive, audit + security logs
AlertingAlert policiesError rate, latency, CPU, memory, DB connections, Redis memory, disk, budget alerts

Cloud Run Services Detail ​

OmniBots runs 16 Cloud Run services (including worker and beat variants).

Service Map ​

ServicePortMemoryCPUMin/MaxConcurrencyVisibilityRole
api-gateway8000512Mi11/10100PublicReverse proxy, routes all HTTP + WebSocket traffic
auth-service8001256Mi11/580PrivateJWT auth, MFA, SSO (Azure AD, Google), RBAC, anomaly detection
bot-service8002256Mi11/580PrivateBot CRUD, flow management, tools, templates
orchestrator-service80031Gi21/1050PrivateLLM orchestration, flow execution, WebSocket handler, intent classification
kb-service8004256Mi11/580PrivateKnowledge base CRUD, document management
ccaas-service8005256Mi11/550PrivateCCaaS escalation (Genesys, 8x8, Amazon Connect, Google CCAI)
voice-service8006512Mi11/1050PublicTelnyx webhooks, STT/TTS, voice authentication, transcriptions
reporting-service8007512Mi10/320PrivateAnalytics, custom reports, usage metering, scheduler task targets
tenant-service8008256Mi11/580PrivateTenant/partner management, settings, language config
billing-service8009256Mi10/350PrivateUsage metering, billing calculations
notification-service8010256Mi10/350PrivatePush notifications (FCM, APNS, Web Push), triggers, event-based alerts
user-service8011256Mi11/580PrivateUser management, roles, profiles
conversation-service8011512Mi11/550PrivateConversation session management, history
rag-service8012512Mi11/550PrivateVector similarity search via pgvector
indexing-service80131Gi20/520PrivateDocument processing HTTP API (chunking, embedding triggers)
indexing-worker—2Gi21/51PrivateCelery worker — document chunking, embedding generation
indexing-beat—256Mi500m1/11PrivateCelery Beat scheduler — exactly 1 instance, drives periodic tasks
imageMicroservices topology diagram showing all 16 Cloud Run services with their connections, ports, and visibility labels
Cloud Run microservices topology

Service Communication ​

  • Public ingress: Only api-gateway and voice-service accept external traffic (allUsers invoker)
  • Internal auth: All private services use Cloud Run IAM (roles/run.invoker bound to Cloud Run service account)
  • API Gateway routing: Reverse proxy fans out to internal services based on URL prefix
  • WebSocket: Session affinity enabled on API Gateway; orchestrator handles real-time bot conversations

Database Schema Highlights ​

Cloud SQL (PostgreSQL 15 + pgvector)

CategoryKey Tables
Multi-tenanttenants, partners, users, roles, permissions
Botsbots, bot_flows, bot_flow_nodes, bot_variables
Toolstenant_tools, tool_templates, tool_execution_analytics
Knowledgeknowledge_bases, kb_documents, kb_chunks, kb_document_sources
Conversationsconversations, messages, conversation_risk_scores
Integrationsplatform_integrations, tenant_integrations, tenant_integration_assignments
Voicecall_transcriptions, voice_call_analytics
Notificationspush_subscriptions, notification_triggers, notification_events, notification_preferences
Securityanomaly_rules, anomaly_events, ip_reputation
i18ntenant_language_configs, translation_cache
Intentstenant_intents, intent_predictions

Embeddings stored as vector(1536) columns via pgvector extension, used for RAG similarity search.


Cloud Storage Buckets ​

BucketLifecycleAccessContent
omnibots-documents-{env}Versioned → Nearline (1yr) → Coldline (2yr)PrivateBot documents, file uploads
omnibots-knowledge-{env}Versioned → Nearline (1yr)PrivateRAG source documents
omnibots-recordings-{env}Retention-locked (prod: 1yr) → Nearline (90d)PrivateVoice recordings for compliance
omnibots-reports-{env}Auto-delete after 30 daysPrivateExported report CSVs
omnibots-assets-{env}Versioned (3 versions kept)PublicTenant logos, branding images
omnibots-widget-assets-{env}No versioningPublicWidget JS/CSS bundles

External Integrations ​

LLM Providers ​

ProviderProtocolUsed By
Anthropic (Claude)REST APIOrchestrator — conversation, tool calling
OpenAI (GPT)REST APIOrchestrator — conversation; Indexing — embeddings
Google Vertex AI (Gemini)gRPC / RESTOrchestrator — conversation; Indexing — embeddings

CCaaS Platforms ​

ProviderProtocolUsed By
Genesys CloudOAuth2 Client Credentials → REST APICCaaS Service — agent escalation, queue routing, screen pop
8x8API Key → REST APICCaaS Service — agent escalation
Amazon ConnectAWS IAM → REST APICCaaS Service — agent escalation
Google CCAIService Account → gRPCCCaaS Service — Dialogflow CX, Agent Assist, Insights

Voice & Telephony ​

ProviderProtocolUsed By
TelnyxREST API + WebhooksVoice Service — SIP trunking, call control, phone numbers
DeepgramWebSocket / RESTVoice Service — real-time speech-to-text

Authentication (SSO) ​

ProviderProtocolUsed By
Azure ADOAuth2 / OIDCAuth Service — enterprise SSO
Google OAuthOAuth2 / OIDCAuth Service — Google SSO

Notifications ​

ProviderProtocolUsed By
Firebase Cloud Messaging (FCM)REST APINotification Service — Android/Web push
Apple Push Notification Service (APNS)HTTP/2Notification Service — iOS push
Web PushVAPIDNotification Service — browser push
SendGridREST APIAuth/Notification Service — email delivery

External Document Sources ​

ProviderProtocolUsed By
SharePointMicrosoft Graph APIIndexing Service — KB document sync
Google DriveDrive API v3Indexing Service — KB document sync
OneDriveMicrosoft Graph APIIndexing Service — KB document sync
DropboxDropbox SDKIndexing Service — KB document sync
BoxBox SDKIndexing Service — KB document sync
AWS S3boto3 SDKIndexing Service — KB document sync
Azure Blob StorageAzure SDKIndexing Service — KB document sync

Security (IP Reputation) ​

ProviderProtocolUsed By
MaxMindREST APIAuth Service — IP geolocation, impossible travel detection
IPinfoREST APIAuth Service — IP metadata
IPQualityScoreREST APIAuth Service — fraud scoring

Frontend Applications ​

All frontends are built with Vue 3 + Composition API + TypeScript + Vite + Tailwind CSS + PrimeVue.

ApplicationPurposeHosting
Operations PortalBot builder, flow editor, analytics, tools, monitoring, settingsFirebase Hosting / Cloud CDN
Tenant Admin PortalTenant-level admin (integrations, users, branding)Firebase Hosting / Cloud CDN
Super Admin PortalPlatform-level admin (partners, tenants, global settings)Firebase Hosting / Cloud CDN
Chat WidgetEmbeddable customer-facing chat widget (script tag, React, Vue, iframe)Cloud Storage + Cloud CDN
Shared LibraryReusable services, composables, types, styles across all portalsnpm package (local)

Key Frontend Libraries ​

LibraryVersionPurpose
Vue 33.xUI framework (Composition API + <script setup>)
Pinia2.xState management
Vue Router4.xClient-side routing
Vue Flow—Visual flow builder canvas (nodes, edges, drag-and-drop)
PrimeVue—UI component library
Tailwind CSS3.xUtility-first CSS
Axios—HTTP client for API calls
Vite5.xBuild tool and dev server

Infrastructure as Code ​

All infrastructure is managed with Terraform >= 1.5.0 using the hashicorp/google and hashicorp/google-beta providers (v5.x).

Terraform Modules ​

ModulePathResources Created
networkingmodules/networkingVPC, subnet, VPC connector, Private Service Access, Redis, firewall rules
cloud-sqlmodules/cloud-sqlPostgreSQL 15 instance, database, user, password in Secret Manager
storagemodules/storage6 Cloud Storage buckets with lifecycle policies
secret-managermodules/secret-managerJWT secret, encryption keys, DB password, API key placeholders
cloud-runmodules/cloud-run16 Cloud Run services, service account, IAM bindings, domain mapping
schedulermodules/scheduler3 Cloud Scheduler jobs, service account
cloud-armormodules/cloud-armorWAF security policy with OWASP CRS, rate limiting, bot management
load-balancermodules/load-balancerGlobal HTTP(S) LB, NEGs, backend services, SSL, health checks
cdnmodules/cdnBackend buckets for widget and frontend CDN
monitoringmodules/monitoringDashboards, uptime checks, SLOs, notification channels (Email, Slack, PagerDuty)
alertingmodules/alertingAlert policies (error rate, latency, CPU, memory, DB, Redis, disk, budget, SLO burn)
loggingmodules/loggingBigQuery dataset, 5 log sinks, 4 log-based metrics, log exclusions, GCS archive bucket

Environments ​

EnvironmentRegionCloud SQL HARedis HAMin InstancesCloud ArmorSLOs
devus-east1ZonalBasic (1 GB)0 (cold start OK)OptionalNo
stagingus-east1ZonalBasic (1 GB)0YesNo
produs-east1 (+ us-west1 secondary)Regional HAStandard HA (5 GB)1 (always warm)YesYes (99.5% avail, 95% latency)

GCP APIs Enabled ​

The following Google Cloud APIs are enabled for the project:

APIPurpose
run.googleapis.comCloud Run serverless containers
sqladmin.googleapis.comCloud SQL administration
secretmanager.googleapis.comSecret Manager
vpcaccess.googleapis.comServerless VPC Access
servicenetworking.googleapis.comPrivate Service Access (VPC peering)
cloudbuild.googleapis.comCloud Build CI/CD
artifactregistry.googleapis.comDocker image registry
cloudscheduler.googleapis.comCloud Scheduler cron jobs
pubsub.googleapis.comPub/Sub messaging
storage.googleapis.comCloud Storage
redis.googleapis.comMemorystore for Redis
aiplatform.googleapis.comVertex AI (LLM, embeddings)
compute.googleapis.comLoad Balancer, Cloud Armor, networking
monitoring.googleapis.comCloud Monitoring, dashboards, alerting
logging.googleapis.comCloud Logging
bigquery.googleapis.comBigQuery log analytics

Security Architecture ​

Network Security ​

  • All services run inside a private VPC
  • Cloud SQL accessible only via private IP (no public IP)
  • Redis accessible only within VPC
  • Only API Gateway and Voice Service are publicly accessible
  • Cloud Armor WAF protects all public endpoints

Application Security ​

  • JWT authentication with secrets from Secret Manager
  • MFA support (TOTP, recovery codes, trusted devices)
  • SSO via Azure AD and Google OAuth (OIDC)
  • RBAC with granular permissions
  • Tenant isolation on every database query (tenant_id filter)
  • PII encryption at rest
  • Anomaly detection (brute force, impossible travel, session anomaly)
  • Conversation fraud detection (prompt injection, PII leakage, social engineering)
  • IP reputation checking (MaxMind, IPinfo, IPQualityScore)

Compliance ​

  • Voice recording retention locks (1 year in production)
  • Audit logging to BigQuery
  • Security log archival to Cloud Storage
  • Log retention policies (90-day default, 730-day archive)
  • HIPAA/FedRAMP/SOC 2 readiness

Data Flow ​

Conversation Flow ​

imageData flow diagram showing a user message flowing through API Gateway, Orchestrator, LLM, RAG, and CCaaS services
Conversation data flow from user to backend services
User (Widget/SMS/Voice/WhatsApp)
  │
  ▼
API Gateway (Cloud Run)
  │
  ├── WebSocket upgrade ──► Orchestrator Service
  │                              │
  │                              ├── LLM call (Vertex AI / Anthropic / OpenAI)
  │                              ├── Tool execution (HTTP API calls)
  │                              ├── KB Search ──► RAG Service ──► pgvector
  │                              ├── Fraud check ──► Auth Service
  │                              └── Handoff ──► CCaaS Service ──► Genesys/8x8/Connect/CCAI
  │
  └── REST API ──► Auth / Bot / Tenant / KB / Reporting services

Document Indexing Flow ​

Upload (Portal or External Source)
  │
  ▼
KB Service (CRUD) ──► Cloud Storage (knowledge bucket)
  │
  ▼
Indexing Service (HTTP) ──► Celery Task Queue (Redis)
  │
  ▼
Indexing Worker ──► Chunk document ──► Generate embeddings (Vertex AI / OpenAI)
  │
  ▼
PostgreSQL (pgvector) ──► kb_chunks table with vector(1536) column

Scheduled Tasks Flow ​

Cloud Scheduler
  │
  ├── 2 AM daily ──► Reporting Service /aggregate-daily
  ├── 3 AM daily ──► Reporting Service /cleanup
  └── Hourly     ──► Reporting Service /usage-metering

Celery Beat (Indexing Beat)
  │
  └── Every 60s  ──► check_source_sync_schedules (external document source sync)

Last updated:

OmniBots AI Bot Platform