Architecture Overview This page describes the complete technical architecture of OmniBots, covering every cloud service, microservice, data store, and external integration that makes up the platform.
High-Level Architecture Diagram image High-level architecture diagram showing client portals, GCP edge layer, Cloud Run microservices, data layer, operations layer, and AI/ML layer
OmniBots platform architecture on Google Cloud ┌─────────────────────────────────────────────────────────────────────────────────┐ │ INTERNET / CLIENTS │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Operations │ │ Tenant Admin │ │ Super Admin │ │ Chat Widget │ │ │ │ Portal │ │ Portal │ │ Portal │ │ (Embedded) │ │ │ │ (Vue 3/Vite) │ │ (Vue 3/Vite)│ │ (Vue 3/Vite) │ │ (Vue 3/Vite) │ │ │ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ │ │ │ │ │ │ │ ┌──────┴──────────────────┴──────────────────┴──────────────────┘ │ │ │ HTTPS / WSS │ └───┼──────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────────┐ │ GOOGLE CLOUD PLATFORM (GCP) │ │ │ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │ EDGE LAYER │ │ │ │ ┌─────────────────┐ ┌──────────────────┐ ┌────────────────────┐ │ │ │ │ │ Global HTTP(S) │ │ Cloud Armor │ │ Cloud CDN │ │ │ │ │ │ Load Balancer │──│ (WAF) │ │ (Widget Assets) │ │ │ │ │ │ (SSL termination │ │ - OWASP Top 10 │ │ - CACHE_ALL_ │ │ │ │ │ │ multi-region) │ │ - Rate limiting │ │ STATIC │ │ │ │ │ └────────┬─────────┘ │ - Bot mgmt │ │ - Edge caching │ │ │ │ │ │ │ - Geo blocking │ │ - Compression │ │ │ │ │ │ │ - IP allowlist │ └────────────────────┘ │ │ │ │ │ └──────────────────┘ │ │ │ └───────────┼──────────────────────────────────────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │ VPC NETWORK (omnibots-vpc) │ │ │ │ ┌────────────────────────────┐ ┌──────────────────────────────────┐ │ │ │ │ │ Serverless VPC Connector │ │ Private Service Access │ │ │ │ │ │ (Cloud Run ↔ VPC) │ │ (VPC Peering for Cloud SQL) │ │ │ │ │ └────────────┬───────────────┘ └──────────────────────────────────┘ │ │ │ │ │ │ │ │ │ ┌────────────┴──────────────────────────────────────────────────────┐ │ │ │ │ │ CLOUD RUN SERVICES │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────┐ PUBLIC ENTRY POINTS │ │ │ │ │ │ │ API Gateway │ Port 8000 │ 512Mi │ 1 CPU │ │ │ │ │ │ │ (FastAPI) │ min: 1, max: 10 │ concurrency: 100 │ │ │ │ │ │ │ Routes all │ Session affinity (WebSocket) │ │ │ │ │ │ │ HTTP + WS │ 512Mi max request body (file uploads) │ │ │ │ │ │ └────────┬─────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌────────┴───────────────────────────────────────────────────┐ │ │ │ │ │ │ │ INTERNAL SERVICES (private) │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ CORE INTELLIGENCE │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Auth Service │ │ Orchestrator │ │ │ │ │ │ │ │ │ │ :8001 │ 256Mi │ │ :8003 │ 1Gi │ │ │ │ │ │ │ │ │ │ JWT, MFA, SSO │ │ 2 CPU │ LLM │ │ │ │ │ │ │ │ │ │ RBAC, Anomaly │ │ Flow Executor │ │ │ │ │ │ │ │ │ │ IP reputation │ │ WebSocket handler│ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Bot Service │ │ RAG Service │ │ │ │ │ │ │ │ │ │ :8002 │ 256Mi │ │ :8012 │ 512Mi │ │ │ │ │ │ │ │ │ │ Bot CRUD, Flows │ │ Vector search │ │ │ │ │ │ │ │ │ │ Tools, Templates│ │ pgvector queries│ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Tenant Service │ │ KB Service │ │ │ │ │ │ │ │ │ │ :8008 │ 256Mi │ │ :8004 │ 256Mi │ │ │ │ │ │ │ │ │ │ Tenant/Partner │ │ Knowledge Base │ │ │ │ │ │ │ │ │ │ Settings, i18n │ │ CRUD, documents │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ User Service │ │ Indexing Service │ │ │ │ │ │ │ │ │ │ :8011 │ 256Mi │ │ :8013 │ 1Gi │ │ │ │ │ │ │ │ │ │ User management │ │ 2 CPU │ Embedding│ │ │ │ │ │ │ │ │ │ Roles, Profiles │ │ Doc processing │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ CHANNELS BACKGROUND │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ CCaaS Service │ │ Indexing Worker │ │ │ │ │ │ │ │ │ │ :8005 │ 256Mi │ │ 2Gi │ 2 CPU │ │ │ │ │ │ │ │ │ │ Genesys, 8x8 │ │ Celery worker │ │ │ │ │ │ │ │ │ │ Connect, CCAI │ │ Doc chunking │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Voice Service │ (public) │ Indexing Beat │ │ │ │ │ │ │ │ │ │ :8006 │ 512Mi │ │ 256Mi │ 500m CPU│ │ │ │ │ │ │ │ │ │ STT/TTS, Telnyx │ │ Celery Beat │ │ │ │ │ │ │ │ │ │ Voice auth │ │ (exactly 1) │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Conversation │ AUXILIARY │ │ │ │ │ │ │ │ │ Service │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ :8011 │ 512Mi │ │ Reporting Svc │ │ │ │ │ │ │ │ │ │ Session mgmt │ │ :8007 │ 512Mi │ │ │ │ │ │ │ │ │ └─────────────────┘ │ Analytics, Usage│ │ │ │ │ │ │ │ │ ┌─────────────────┐ └─────────────────┘ │ │ │ │ │ │ │ │ │ Notification │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ Service │ │ Billing Service │ │ │ │ │ │ │ │ │ │ :8010 │ 256Mi │ │ :8009 │ 256Mi │ │ │ │ │ │ │ │ │ │ Push, Triggers │ │ Usage metering │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ │ │ DATA LAYER │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────────┐ ┌──────────────────────┐ │ │ │ │ │ │ │ Cloud SQL │ │ Memorystore │ │ │ │ │ │ │ │ (PostgreSQL 15) │ │ (Redis 7.0) │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • pgvector extension │ │ Prod: STANDARD_HA │ │ │ │ │ │ │ │ • Private IP only │ │ (5 GB, HA failover) │ │ │ │ │ │ │ │ • SSD storage │ │ │ │ │ │ │ │ │ │ • Auto-resize disk │ │ Dev: BASIC (1 GB) │ │ │ │ │ │ │ │ • Query Insights │ │ │ │ │ │ │ │ │ │ • PITR (prod) │ │ Uses: │ │ │ │ │ │ │ │ • Daily backups │ │ • Session cache │ │ │ │ │ │ │ │ • 500 connections │ │ • Task queue broker │ │ │ │ │ │ │ │ (prod) │ │ • Rate limiting │ │ │ │ │ │ │ │ │ │ • Translation cache │ │ │ │ │ │ │ │ Prod: REGIONAL HA │ │ • Pub/Sub channels │ │ │ │ │ │ │ │ Dev: ZONAL │ │ │ │ │ │ │ │ │ └──────────────────────┘ └──────────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ │ │ │ │ Cloud Storage Buckets │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ │ │ │ │ Documents │ │ Knowledge │ │ Recordings │ │ │ │ │ │ │ │ │ │ Bot files, │ │ RAG docs, │ │ Voice calls │ │ │ │ │ │ │ │ │ │ uploads │ │ embeddings │ │ Compliance │ │ │ │ │ │ │ │ │ │ Versioned │ │ Versioned │ │ retention │ │ │ │ │ │ │ │ │ │ Nearline→ │ │ Nearline→ │ │ locked (prod)│ │ │ │ │ │ │ │ │ │ Coldline │ │ lifecycle │ │ │ │ │ │ │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ │ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ │ │ │ │ Reports │ │ Branding │ │ Widget │ │ │ │ │ │ │ │ │ │ Exported CSV │ │ Assets │ │ Assets │ │ │ │ │ │ │ │ │ │ 30-day TTL │ │ Logos, icons │ │ JS/CSS bundle│ │ │ │ │ │ │ │ │ │ │ │ Public read │ │ Public read │ │ │ │ │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ │ │ │ │ └──────────────────────────────────────────────────────────────┘ │ │ │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ │ │ OPERATIONS LAYER │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────┐ ┌──────────────────┐ ┌────────────────┐ │ │ │ │ │ │ │ Secret Manager │ │ Cloud Scheduler │ │ Artifact │ │ │ │ │ │ │ │ │ │ │ │ Registry │ │ │ │ │ │ │ │ • JWT secret │ │ • Daily metrics │ │ │ │ │ │ │ │ │ │ • Encryption key │ │ aggregation │ │ Docker images │ │ │ │ │ │ │ │ • OAuth key │ │ (2 AM daily) │ │ for all Cloud │ │ │ │ │ │ │ │ • DB password │ │ • Conversation │ │ Run services │ │ │ │ │ │ │ │ • Anthropic key │ │ cleanup (3 AM) │ │ │ │ │ │ │ │ │ │ • OpenAI key │ │ • Usage metering │ │ us-docker. │ │ │ │ │ │ │ │ • Telnyx key │ │ (hourly) │ │ pkg.dev │ │ │ │ │ │ │ │ • Deepgram key │ │ │ │ │ │ │ │ │ │ │ │ • SendGrid key │ │ OIDC auth to │ │ │ │ │ │ │ │ │ │ • Genesys creds │ │ Cloud Run │ │ │ │ │ │ │ │ │ └──────────────────┘ └──────────────────┘ └────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────┐ ┌──────────────────┐ ┌────────────────┐ │ │ │ │ │ │ │ Cloud Build │ │ Cloud Logging │ │ Cloud │ │ │ │ │ │ │ │ │ │ │ │ Monitoring │ │ │ │ │ │ │ │ CI/CD pipeline │ │ Log sinks: │ │ │ │ │ │ │ │ │ │ Build → Push → │ │ • App → BigQuery │ │ • Dashboards │ │ │ │ │ │ │ │ Deploy to │ │ • Errors → BQ │ │ • Uptime checks│ │ │ │ │ │ │ │ Cloud Run │ │ • All → GCS │ │ • SLOs (prod) │ │ │ │ │ │ │ │ │ │ (archive) │ │ • Alert policies│ │ │ │ │ │ │ │ │ │ • Audit → BQ │ │ │ │ │ │ │ │ │ │ │ │ • Security → GCS │ │ Channels: │ │ │ │ │ │ │ │ │ │ │ │ Email, Slack, │ │ │ │ │ │ │ │ │ │ Log-based metrics: │ │ PagerDuty │ │ │ │ │ │ │ │ │ │ error_count, │ │ │ │ │ │ │ │ │ │ │ │ request_latency, │ │ │ │ │ │ │ │ │ │ │ │ auth_failures, │ │ │ │ │ │ │ │ │ │ │ │ bot_conversations │ │ │ │ │ │ │ │ │ └──────────────────┘ └──────────────────┘ └────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ │ │ │ │ BigQuery │ │ │ │ │ │ │ │ Log analytics dataset — partitioned tables for app logs, │ │ │ │ │ │ │ │ error logs, and audit logs. 90-day retention default. │ │ │ │ │ │ │ └──────────────────────────────────────────────────────────────┘ │ │ │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ │ │ AI / ML LAYER │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────┐ ┌──────────────────┐ │ │ │ │ │ │ │ Vertex AI │ │ Dialogflow CX │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • Gemini models │ │ • NLU engine │ │ │ │ │ │ │ │ • Text embeddings│ │ • Intent matching │ │ │ │ │ │ │ │ (RAG) │ │ • Entity │ │ │ │ │ │ │ │ • Prediction API │ │ extraction │ │ │ │ │ │ │ └──────────────────┘ └──────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────────────┐ │ │ │ │ │ │ │ Pub/Sub │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ • CCAI webhook │ │ │ │ │ │ │ │ events │ │ │ │ │ │ │ │ • Async event │ │ │ │ │ │ │ │ bus │ │ │ │ │ │ │ └──────────────────┘ │ │ │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ │ IAM / Service Accounts: │ │ │ │ • omnibots-cloudrun-{env} — Cloud Run services │ │ │ │ Roles: cloudsql.client, storage.objectAdmin, secretmanager.accessor, │ │ │ │ aiplatform.user, dialogflow.client, pubsub.subscriber │ │ │ │ • omnibots-scheduler-{env} — Cloud Scheduler │ │ │ │ Roles: run.invoker │ │ │ │ • omnibots-log-writer-{env} — Log writer to BigQuery │ │ │ │ Roles: bigquery.dataEditor │ │ │ │ │ │ └──┴───────────────────────────────────────────────────────────────────────────┘ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222
GCP Services Inventory Compute & Networking Service Resource Purpose Cloud Run 16 services Serverless containers for all backend microservices VPC Network omnibots-vpc-{env}Private network isolating all services Serverless VPC Connector omnibots-connector-{env}Bridges Cloud Run to VPC (2-10 instances) Global HTTP(S) Load Balancer External LB SSL termination, multi-region routing, health checks Cloud Armor omnibots-waf-{env}WAF with OWASP CRS rules, rate limiting, geo-blocking Cloud CDN Widget + Frontend backends Edge caching for static widget and portal assets Compute Network Firewall rules, Private Service Access Internal traffic rules, VPC peering for Cloud SQL
Data & Storage Service Resource Purpose Cloud SQL PostgreSQL 15 (omnibots-postgres-{env}) Primary relational database with pgvector for embeddings Memorystore Redis 7.0 (omnibots-redis-{env}) Session cache, Celery broker, rate limiting, translation cache Cloud Storage 6 buckets Documents, knowledge base files, voice recordings, reports, branding assets, widget assets BigQuery omnibots_logs_{env} datasetLog analytics — partitioned tables for app, error, and audit logs
Security & Secrets Service Resource Purpose Secret Manager 10+ secrets JWT key, encryption keys, DB password, API keys (Anthropic, OpenAI, Telnyx, Deepgram, SendGrid, Genesys) IAM 3 service accounts Least-privilege access for Cloud Run, Scheduler, and Log Writer Cloud Armor WAF 8+ security rules SQLi, XSS, LFI, RFI, RCE protection; rate limiting (1000 req/min API, 20 req/min auth)
AI & ML Service Resource Purpose Vertex AI AI Platform API Gemini models for LLM orchestration, text embeddings for RAG Dialogflow CX Agent + Intents NLU engine for intent matching and entity extraction Pub/Sub Topics + Subscriptions Async event bus for CCAI webhook events
Operations Service Resource Purpose Cloud Scheduler 3 cron jobs Daily metrics aggregation (2 AM), conversation cleanup (3 AM), hourly usage metering Cloud Build CI/CD pipeline Build Docker images, push to Artifact Registry, deploy to Cloud Run Artifact Registry us-docker.pkg.devDocker container image registry for all services Cloud Monitoring Dashboards, uptime checks, SLOs Service health monitoring, availability SLO (99.5%), latency SLO (95%) Cloud Logging 5 log sinks, 4 log-based metrics App/error logs to BigQuery, all logs to GCS archive, audit + security logs Alerting Alert policies Error rate, latency, CPU, memory, DB connections, Redis memory, disk, budget alerts
Cloud Run Services Detail OmniBots runs 16 Cloud Run services (including worker and beat variants).
Service Map Service Port Memory CPU Min/Max Concurrency Visibility Role api-gateway 8000 512Mi 1 1/10 100 Public Reverse proxy, routes all HTTP + WebSocket traffic auth-service 8001 256Mi 1 1/5 80 Private JWT auth, MFA, SSO (Azure AD, Google), RBAC, anomaly detection bot-service 8002 256Mi 1 1/5 80 Private Bot CRUD, flow management, tools, templates orchestrator-service 8003 1Gi 2 1/10 50 Private LLM orchestration, flow execution, WebSocket handler, intent classification kb-service 8004 256Mi 1 1/5 80 Private Knowledge base CRUD, document management ccaas-service 8005 256Mi 1 1/5 50 Private CCaaS escalation (Genesys, 8x8, Amazon Connect, Google CCAI) voice-service 8006 512Mi 1 1/10 50 Public Telnyx webhooks, STT/TTS, voice authentication, transcriptions reporting-service 8007 512Mi 1 0/3 20 Private Analytics, custom reports, usage metering, scheduler task targets tenant-service 8008 256Mi 1 1/5 80 Private Tenant/partner management, settings, language config billing-service 8009 256Mi 1 0/3 50 Private Usage metering, billing calculations notification-service 8010 256Mi 1 0/3 50 Private Push notifications (FCM, APNS, Web Push), triggers, event-based alerts user-service 8011 256Mi 1 1/5 80 Private User management, roles, profiles conversation-service 8011 512Mi 1 1/5 50 Private Conversation session management, history rag-service 8012 512Mi 1 1/5 50 Private Vector similarity search via pgvector indexing-service 8013 1Gi 2 0/5 20 Private Document processing HTTP API (chunking, embedding triggers) indexing-worker — 2Gi 2 1/5 1 Private Celery worker — document chunking, embedding generation indexing-beat — 256Mi 500m 1/1 1 Private Celery Beat scheduler — exactly 1 instance, drives periodic tasks
image Microservices topology diagram showing all 16 Cloud Run services with their connections, ports, and visibility labels
Cloud Run microservices topology Service Communication Public ingress : Only api-gateway and voice-service accept external traffic (allUsers invoker)Internal auth : All private services use Cloud Run IAM (roles/run.invoker bound to Cloud Run service account)API Gateway routing : Reverse proxy fans out to internal services based on URL prefixWebSocket : Session affinity enabled on API Gateway; orchestrator handles real-time bot conversationsDatabase Schema Highlights Cloud SQL (PostgreSQL 15 + pgvector)
Category Key Tables Multi-tenant tenants, partners, users, roles, permissions Bots bots, bot_flows, bot_flow_nodes, bot_variables Tools tenant_tools, tool_templates, tool_execution_analytics Knowledge knowledge_bases, kb_documents, kb_chunks, kb_document_sources Conversations conversations, messages, conversation_risk_scores Integrations platform_integrations, tenant_integrations, tenant_integration_assignments Voice call_transcriptions, voice_call_analytics Notifications push_subscriptions, notification_triggers, notification_events, notification_preferences Security anomaly_rules, anomaly_events, ip_reputation i18n tenant_language_configs, translation_cache Intents tenant_intents, intent_predictions
Embeddings stored as vector(1536) columns via pgvector extension, used for RAG similarity search.
Cloud Storage Buckets Bucket Lifecycle Access Content omnibots-documents-{env}Versioned → Nearline (1yr) → Coldline (2yr) Private Bot documents, file uploads omnibots-knowledge-{env}Versioned → Nearline (1yr) Private RAG source documents omnibots-recordings-{env}Retention-locked (prod: 1yr) → Nearline (90d) Private Voice recordings for compliance omnibots-reports-{env}Auto-delete after 30 days Private Exported report CSVs omnibots-assets-{env}Versioned (3 versions kept) Public Tenant logos, branding images omnibots-widget-assets-{env}No versioning Public Widget JS/CSS bundles
External Integrations LLM Providers Provider Protocol Used By Anthropic (Claude) REST API Orchestrator — conversation, tool calling OpenAI (GPT) REST API Orchestrator — conversation; Indexing — embeddings Google Vertex AI (Gemini) gRPC / REST Orchestrator — conversation; Indexing — embeddings
Provider Protocol Used By Genesys Cloud OAuth2 Client Credentials → REST API CCaaS Service — agent escalation, queue routing, screen pop 8x8 API Key → REST API CCaaS Service — agent escalation Amazon Connect AWS IAM → REST API CCaaS Service — agent escalation Google CCAI Service Account → gRPC CCaaS Service — Dialogflow CX, Agent Assist, Insights
Voice & Telephony Provider Protocol Used By Telnyx REST API + Webhooks Voice Service — SIP trunking, call control, phone numbers Deepgram WebSocket / REST Voice Service — real-time speech-to-text
Authentication (SSO) Provider Protocol Used By Azure AD OAuth2 / OIDC Auth Service — enterprise SSO Google OAuth OAuth2 / OIDC Auth Service — Google SSO
Notifications Provider Protocol Used By Firebase Cloud Messaging (FCM) REST API Notification Service — Android/Web push Apple Push Notification Service (APNS) HTTP/2 Notification Service — iOS push Web Push VAPID Notification Service — browser push SendGrid REST API Auth/Notification Service — email delivery
External Document Sources Provider Protocol Used By SharePoint Microsoft Graph API Indexing Service — KB document sync Google Drive Drive API v3 Indexing Service — KB document sync OneDrive Microsoft Graph API Indexing Service — KB document sync Dropbox Dropbox SDK Indexing Service — KB document sync Box Box SDK Indexing Service — KB document sync AWS S3 boto3 SDK Indexing Service — KB document sync Azure Blob Storage Azure SDK Indexing Service — KB document sync
Security (IP Reputation) Provider Protocol Used By MaxMind REST API Auth Service — IP geolocation, impossible travel detection IPinfo REST API Auth Service — IP metadata IPQualityScore REST API Auth Service — fraud scoring
Frontend Applications All frontends are built with Vue 3 + Composition API + TypeScript + Vite + Tailwind CSS + PrimeVue .
Application Purpose Hosting Operations Portal Bot builder, flow editor, analytics, tools, monitoring, settings Firebase Hosting / Cloud CDN Tenant Admin Portal Tenant-level admin (integrations, users, branding) Firebase Hosting / Cloud CDN Super Admin Portal Platform-level admin (partners, tenants, global settings) Firebase Hosting / Cloud CDN Chat Widget Embeddable customer-facing chat widget (script tag, React, Vue, iframe) Cloud Storage + Cloud CDN Shared Library Reusable services, composables, types, styles across all portals npm package (local)
Key Frontend Libraries Library Version Purpose Vue 3 3.x UI framework (Composition API + <script setup>) Pinia 2.x State management Vue Router 4.x Client-side routing Vue Flow — Visual flow builder canvas (nodes, edges, drag-and-drop) PrimeVue — UI component library Tailwind CSS 3.x Utility-first CSS Axios — HTTP client for API calls Vite 5.x Build tool and dev server
Infrastructure as Code All infrastructure is managed with Terraform >= 1.5.0 using the hashicorp/google and hashicorp/google-beta providers (v5.x).
Module Path Resources Created networking modules/networkingVPC, subnet, VPC connector, Private Service Access, Redis, firewall rules cloud-sql modules/cloud-sqlPostgreSQL 15 instance, database, user, password in Secret Manager storage modules/storage6 Cloud Storage buckets with lifecycle policies secret-manager modules/secret-managerJWT secret, encryption keys, DB password, API key placeholders cloud-run modules/cloud-run16 Cloud Run services, service account, IAM bindings, domain mapping scheduler modules/scheduler3 Cloud Scheduler jobs, service account cloud-armor modules/cloud-armorWAF security policy with OWASP CRS, rate limiting, bot management load-balancer modules/load-balancerGlobal HTTP(S) LB, NEGs, backend services, SSL, health checks cdn modules/cdnBackend buckets for widget and frontend CDN monitoring modules/monitoringDashboards, uptime checks, SLOs, notification channels (Email, Slack, PagerDuty) alerting modules/alertingAlert policies (error rate, latency, CPU, memory, DB, Redis, disk, budget, SLO burn) logging modules/loggingBigQuery dataset, 5 log sinks, 4 log-based metrics, log exclusions, GCS archive bucket
Environments Environment Region Cloud SQL HA Redis HA Min Instances Cloud Armor SLOs dev us-east1 Zonal Basic (1 GB) 0 (cold start OK) Optional No staging us-east1 Zonal Basic (1 GB) 0 Yes No prod us-east1 (+ us-west1 secondary) Regional HA Standard HA (5 GB) 1 (always warm) Yes Yes (99.5% avail, 95% latency)
GCP APIs Enabled The following Google Cloud APIs are enabled for the project:
API Purpose run.googleapis.comCloud Run serverless containers sqladmin.googleapis.comCloud SQL administration secretmanager.googleapis.comSecret Manager vpcaccess.googleapis.comServerless VPC Access servicenetworking.googleapis.comPrivate Service Access (VPC peering) cloudbuild.googleapis.comCloud Build CI/CD artifactregistry.googleapis.comDocker image registry cloudscheduler.googleapis.comCloud Scheduler cron jobs pubsub.googleapis.comPub/Sub messaging storage.googleapis.comCloud Storage redis.googleapis.comMemorystore for Redis aiplatform.googleapis.comVertex AI (LLM, embeddings) compute.googleapis.comLoad Balancer, Cloud Armor, networking monitoring.googleapis.comCloud Monitoring, dashboards, alerting logging.googleapis.comCloud Logging bigquery.googleapis.comBigQuery log analytics
Security Architecture Network Security All services run inside a private VPC Cloud SQL accessible only via private IP (no public IP) Redis accessible only within VPC Only API Gateway and Voice Service are publicly accessible Cloud Armor WAF protects all public endpoints Application Security JWT authentication with secrets from Secret Manager MFA support (TOTP, recovery codes, trusted devices) SSO via Azure AD and Google OAuth (OIDC) RBAC with granular permissions Tenant isolation on every database query (tenant_id filter) PII encryption at rest Anomaly detection (brute force, impossible travel, session anomaly) Conversation fraud detection (prompt injection, PII leakage, social engineering) IP reputation checking (MaxMind, IPinfo, IPQualityScore) Compliance Voice recording retention locks (1 year in production) Audit logging to BigQuery Security log archival to Cloud Storage Log retention policies (90-day default, 730-day archive) HIPAA/FedRAMP/SOC 2 readiness Data Flow Conversation Flow image Data flow diagram showing a user message flowing through API Gateway, Orchestrator, LLM, RAG, and CCaaS services
Conversation data flow from user to backend services User (Widget/SMS/Voice/WhatsApp) │ ▼ API Gateway (Cloud Run) │ ├── WebSocket upgrade ──► Orchestrator Service │ │ │ ├── LLM call (Vertex AI / Anthropic / OpenAI) │ ├── Tool execution (HTTP API calls) │ ├── KB Search ──► RAG Service ──► pgvector │ ├── Fraud check ──► Auth Service │ └── Handoff ──► CCaaS Service ──► Genesys/8x8/Connect/CCAI │ └── REST API ──► Auth / Bot / Tenant / KB / Reporting services 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Document Indexing Flow Upload (Portal or External Source) │ ▼ KB Service (CRUD) ──► Cloud Storage (knowledge bucket) │ ▼ Indexing Service (HTTP) ──► Celery Task Queue (Redis) │ ▼ Indexing Worker ──► Chunk document ──► Generate embeddings (Vertex AI / OpenAI) │ ▼ PostgreSQL (pgvector) ──► kb_chunks table with vector(1536) column 1 2 3 4 5 6 7 8 9 10 11 12 13
Scheduled Tasks Flow Cloud Scheduler │ ├── 2 AM daily ──► Reporting Service /aggregate-daily ├── 3 AM daily ──► Reporting Service /cleanup └── Hourly ──► Reporting Service /usage-metering Celery Beat (Indexing Beat) │ └── Every 60s ──► check_source_sync_schedules (external document source sync) 1 2 3 4 5 6 7 8 9
