Video Calls

The Agent Desktop supports video calling through a floating, draggable video panel powered by the Telnyx Video SDK. Video calls allow agents to engage face-to-face with customers during chat conversations, adding a visual dimension to the interaction.

Starting a Video Call

To initiate a video call:

1. Open a chat conversation (web chat channel)
2. Click the Start Video button (videocam icon) in the ChatHeader
3. A video session is created and the FloatingVideoPanel appears in the center of the screen
4. The panel shows a "Waiting for user to accept..." message while connecting
5. Once the remote user accepts, both video feeds become active

INFO

The Start Video button is only available for chat channel conversations and is hidden when a video session is already active.

FloatingVideoPanel

The video panel is a floating overlay that can be dragged anywhere on the screen. It operates in three display modes.

Default Mode

The default mode is the standard video call view with the following layout:

• Title bar -- A draggable dark header showing:
  • A drag indicator icon (grab cursor)
  • "Video Call" label
  • Duration timer (MM:SS format, starts counting when the call becomes active)
  • Minimize button (reduces to PiP)
  • Maximize button (expands to full screen)
  • Close button (ends the call, shown in red)
• Split view -- The body is divided into two equal halves:
  • Left half -- Your local camera feed, labeled "You"
  • Right half -- The remote participant's video feed, labeled "User"
• Control bar -- A centered row of circular control buttons at the bottom

If your camera is turned off, the local video area shows a gray overlay with a camera-off icon. If the remote participant has not connected yet, the remote area shows a person icon with "Waiting..." text.

imageVideo call floating panel showing draggable title bar with duration timer, split view with local camera feed labeled You on the left and remote participant feed labeled User on the right, and control bar with mute, camera, blur, screen share, and end call buttons

Video call overlay with remote and local feeds

Panel dimensions: 560px wide by 400px tall.

Minimized Mode (PiP)

The minimized mode condenses the panel into a small picture-in-picture (PiP) view:

• Remote video only -- Shows only the remote participant's video feed
• Duration badge -- A small overlay badge in the bottom-left showing the call duration
• Expand button -- Appears on hover in the top-right corner to restore to default mode
• Draggable -- The entire PiP can be dragged to any position on the screen
• Click to restore -- Click anywhere on the PiP or double-click to return to default mode

Panel dimensions: 200px wide by 150px tall.

TIP

Use the minimized mode when you need to see the customer's video while focusing on the chat conversation or referring to other panels. The PiP stays on top so you maintain visual contact.

Maximized Mode

The maximized mode fills the entire viewport for a full-screen video experience:

• Remote video -- Fills the entire screen as the primary view
• Local video (PiP) -- Your camera feed appears as a small draggable overlay (160x120px) in the bottom-left corner. You can drag this PiP to any position within the maximized view.
• Top bar -- A floating bar with:
  • Duration badge (pill-shaped with MM:SS timer)
  • Exit fullscreen button (close_fullscreen icon)
• Control bar -- A floating, rounded, semi-transparent control bar centered at the bottom

Double-click the title bar in default mode to switch to maximized mode, or click the maximize button. Click the exit fullscreen button (or close_fullscreen) to return to default mode.

Video Controls

The control bar appears in both default and maximized modes with the same set of buttons:

Control	Icon (Active)	Icon (Inactive)	Description
Mute	mic_off (red)	mic (gray)	Toggle microphone on/off. Red highlight when muted.
Camera	videocam_off (red)	videocam (gray)	Toggle camera on/off. Red highlight when camera is off. When disabled, the local video area shows a camera-off overlay.
Background Blur	blur_on (blue)	blur_on (gray)	Toggle background blur. Blue highlight when active. Shows a loading spinner (hourglass) while the blur model is being loaded.
Screen Share	stop_screen_share (blue)	screen_share (gray)	Toggle screen sharing. Blue highlight when sharing.
End Call	call_end (red always)	--	End the video call. Always displayed with a red background.

Mute / Unmute

Toggles the audio track of your local stream. When muted, the button turns red and the mic_off icon is shown. Audio is stopped at the track level so the remote participant hears silence.

Camera Toggle

Toggles the video track of your local stream. When the camera is off:

• The button turns red with the videocam_off icon
• Your local video area shows a gray overlay with a camera-off icon
• The camera hardware light turns off (the raw camera track is disabled)
• The remote participant sees a frozen/black frame

Background Blur

Background blur uses MediaPipe's selfie segmentation model to separate you from your background in real time. The implementation:

1. On first activation, the blur model is loaded from a CDN (this takes a few seconds)
2. While loading, the button shows an hourglass icon and is disabled
3. Once loaded, the model runs each video frame through segmentation
4. Your figure is composited over a blurred version of the background
5. The model is cached after the first load -- subsequent activations are instant

WARNING

Background blur requires GPU acceleration and may increase CPU/GPU usage. On lower-powered devices, you may notice a slight reduction in frame rate when blur is enabled.

Screen Share

Screen sharing routes your screen content through the existing video stream:

1. Click the screen share button
2. The browser's native screen picker dialog appears (choose a screen, window, or tab)
3. Your screen content replaces the camera feed sent to the remote participant
4. Your local video preview switches to show your raw camera feed (so you can still see yourself)
5. Click the button again or use the browser's "Stop sharing" control to end screen sharing

When screen sharing is active, background blur is automatically bypassed (blur only applies to the camera feed).

End Call

Clicking the End Call button immediately terminates the video session:

• All media tracks (camera, microphone, screen share) are stopped
• The WebRTC connection is disconnected
• The floating panel closes
• The video store is reset to idle

Connecting State

While the video call is being established, a connecting overlay appears over the split view:

• Displays a videocam icon with the text "Waiting for user to accept..."
• If a connection error occurs, the icon changes to error_outline (red) and the error message is displayed
• Connection errors auto-dismiss after 3 seconds and the panel closes

Dragging Behavior

The FloatingVideoPanel supports drag-to-reposition in both default and minimized modes:

• Default mode -- Drag the title bar to reposition the panel
• Minimized mode -- Drag anywhere on the PiP to reposition it
• Maximized mode -- The main panel is fixed (full viewport), but the local PiP overlay can be dragged independently within the viewport

Panel position is persisted in the video store, so the panel returns to its last position when switching between modes.

INFO

Elements within the panel that are marked as interactive (buttons, controls) do not initiate a drag when clicked. Only the title bar (default mode) or the video area (minimized mode) respond to drag gestures.

Telnyx Video SDK Integration

Video calls are powered by the Telnyx Video SDK. The integration handles:

• Room creation -- A video session is created via the backend API, which returns a room ID and client token
• Room connection -- The Agent Desktop connects to the Telnyx video room using WebRTC
• Stream publishing -- The agent's camera and microphone are published to the room via a canvas proxy pipeline (this enables background blur and screen share switching)
• Stream subscription -- Remote participant streams are automatically subscribed to when they join or are already present
• Reconnection -- If no remote stream is found immediately, the system retries participant enumeration at 1.5s, 3s, and 5s intervals
• Cleanup -- On disconnect, all media tracks are stopped, canvas proxy loops are cancelled, and the room connection is closed