Skip to content

Live and Voice Agents

Supported in ADKPython v0.5.0Java v0.2.0Experimental

Live agents hold an open, two-way connection with the user. Instead of sending a message and waiting for a reply, the user and the agent can both speak, listen, and respond at the same time — and the user can interrupt the agent mid-sentence, the way people interrupt each other in real conversation. Live agents accept text, audio, and video input, and reply with text or speech.

This capability is powered by the Gemini Live API. ADK wraps it in the same agent, tool, and session abstractions you use everywhere else, so a live agent is an ADK agent — not a separate framework.

Start here

What is bidirectional streaming?

Bidi-streaming (Bidirectional streaming) represents a fundamental shift from traditional AI interactions. Instead of the rigid "ask-and-wait" pattern, it enables real-time, two-way communication where both human and AI can speak, listen, and respond simultaneously. This creates natural, human-like conversations with immediate responses and the revolutionary ability to interrupt ongoing interactions.

Think of the difference between sending emails and having a phone conversation. Traditional AI interactions are like emails—you send a complete message, wait for a complete response, then send another complete message. Bidi-streaming is like a phone conversation—fluid, natural, with the ability to interrupt, clarify, and respond in real-time.

Key characteristics

These characteristics distinguish Bidi-streaming from traditional AI interactions and make it uniquely powerful for creating engaging user experiences:

  • Two-way Communication: Continuous data exchange without waiting for complete responses. Users can interrupt the AI mid-response with new input, creating a natural conversational flow. The AI responds after detecting the user has finished speaking (via automatic voice activity detection or explicit activity signals).

  • Responsive Interruption: Perhaps the most important feature for the natural user experience—users can interrupt the agent mid-response with new input, just like in human conversation. If an AI is explaining quantum physics and you suddenly ask "wait, what's an electron?", the AI stops immediately and addresses your question.

  • Best for Multimodal: Bidi-streaming excels at multimodal interactions because it can process different input types simultaneously through a single connection. Users can speak while showing documents, type follow-up questions during voice calls, or seamlessly switch between communication modes without losing context. This unified approach eliminates the complexity of managing separate channels for each modality.

sequenceDiagram
    participant Client as User
    participant Agent

    Client->>Agent: "Hi!"
    Client->>Agent: "Explain the history of Japan"
    Agent->>Client: "Hello!"
    Agent->>Client: "Sure! Japan's history is a..." (partial content)
    Client->>Agent: "Ah, wait."

    Agent->>Client: "OK, how can I help?" [interrupted: true]

Which kind of streaming do you need?

"Streaming" covers three different things in ADK, and picking the wrong one is a common source of confusion. Use this table to find the one you want:

What it does Can the user interrupt? Use it when Where it lives
Server-side streaming One-way flow from server to client. Like watching a live video feed — continuous data, no interaction. No You are pushing dashboard or feed updates, not holding a conversation. Outside ADK
Token-level streaming The model's text response arrives word by word, but you must wait for it to finish before sending more input. Like watching someone type. No You want a responsive-feeling text chat. StreamingMode.SSE — see Configuration
Bidirectional streaming Both sides can speak, listen, and respond at once, over one open connection. Yes You are building voice or video conversation. runner.run_live() — these pages

The rest of this section is about the third row.

Real-world applications

Bidi-streaming revolutionizes agentic AI applications by enabling agents to operate with human-like responsiveness and intelligence. These applications showcase how streaming transforms static AI interactions into dynamic, agent-driven experiences that feel genuinely intelligent and proactive.

In a video of the Shopper's Concierge demo, the multimodal Bidi-streaming feature significantly improve the user experience of e-commerce by enabling a faster and more intuitive shopping experience. The combination of conversational understanding and rapid, parallelized searching culminates in advanced capabilities like virtual try-on, boosting buyer confidence and reducing the friction of online shopping.

Also, there are many possible real-world applications for Bidi-streaming:

Customer service and contact centers

This is the most direct application. The technology can create sophisticated virtual agents that go far beyond traditional chatbots.

  • Use case: A customer calls a retail company's support line about a defective product.
  • Multimodality (video): The customer can say, "My coffee machine is leaking from the bottom, let me show you." They can then use their phone's camera to stream live video of the issue. The AI agent can use its vision capabilities to identify the model and the specific point of failure.
  • Live Interaction & Interruption: If the agent says, "Okay, I'm processing a return for your Model X coffee maker," the customer can interrupt with, "No, wait, it's the Model Y Pro," and the agent can immediately correct its course without restarting the conversation.

E-commerce and personalized shopping

The agent can act as a live, interactive personal shopper, enhancing the online retail experience.

  • Use Case: A user is browsing a fashion website and wants styling advice.
  • Multimodality (Voice & Image): The user can hold up a piece of clothing to their webcam and ask, "Can you find me a pair of shoes that would go well with these pants?" The agent analyzes the color and style of the pants.
  • Live Interaction: The conversation can be a fluid back-and-forth: "Show me something more casual." ... "Okay, how about these sneakers?" ... "Perfect, add the blue ones in size 10 to my cart."

Field service and technical assistance

Technicians working on-site can use a hands-free, voice-activated assistant to get real-time help.

  • Use Case: An HVAC technician is on-site trying to diagnose a complex commercial air conditioning unit.
  • Multimodality (Video & Voice): The technician, wearing smart glasses or using a phone, can stream their point-of-view to the AI agent. They can ask, "I'm hearing a strange noise from this compressor. Can you identify it and pull up the diagnostic flowchart for this model?"
  • Live Interaction: The agent can guide the technician step-by-step, and the technician can ask clarifying questions or interrupt at any point without taking their hands off their tools.

Healthcare and telemedicine

The agent can serve as a first point of contact for patient intake, triage, and basic consultations.

  • Use Case: A patient uses a provider's app for a preliminary consultation about a skin condition.
  • Multimodality (Video/Image): The patient can securely share a live video or high-resolution image of a rash. The AI can perform a preliminary analysis and ask clarifying questions.

Financial services and wealth management

An agent can provide clients with a secure, interactive, and data-rich way to manage their finances.

  • Use Case: A client wants to review their investment portfolio and discuss market trends.
  • Multimodality (Screen Sharing): The agent can share its screen to display charts, graphs, and portfolio performance data. The client could also share their screen to point to a specific news article and ask, "What is the potential impact of this event on my tech stocks?"
  • Live Interaction: Analyze the client's current portfolio allocation by accessing their account data.Simulate the impact of a potential trade on the portfolio's risk profile.

Live API platforms

ADK Gemini Live API Toolkit capabilities are powered by Live API technology, available through two platforms: Gemini Live API (via Google AI Studio) and Gemini Live API (Agent Platform) (via Google Cloud). Both provide real-time, low-latency streaming conversations with Gemini models, but serve different development and deployment needs.

Throughout this guide, we use "Live API" to refer to both platforms collectively, specifying "Gemini Live API" or "Gemini Live API (Agent Platform)" only when discussing platform-specific features or differences.

What is the Live API?

Live API is Google's real-time conversational AI technology that enables low-latency Bidi-streaming with Gemini models. Unlike traditional request-response APIs, Live API establishes persistent WebSocket connections that support:

Core Capabilities:

  • Multimodal streaming: Processes continuous streams of audio, video, and text in real-time
  • Voice Activity Detection (VAD): Automatically detects when users finish speaking, enabling natural turn-taking without explicit signals. The AI knows when to start responding and when to wait for more input
  • Immediate responses: Delivers human-like spoken or text responses with minimal latency
  • Intelligent interruption: Enables users to interrupt the AI mid-response, just like human conversations
  • Audio Transcription: Real-time transcription of both user input and model output, enabling accessibility features and conversation logging without separate transcription services
  • Session Management: Long conversations can span multiple connections through session resumption, with the API preserving full conversation history and context across reconnections
  • Tool Integration: Function calling works seamlessly in streaming mode, with tools executing in the background while conversation continues

Native Audio Model Features:

  • Proactive Audio: The model can initiate responses based on context awareness, creating more natural interactions where the AI offers help or clarification proactively (Native Audio models only)
  • Affective Dialog: Advanced models understand tone of voice and emotional context, adapting responses to match the conversational mood and user sentiment (Native Audio models only)

Learn More

For detailed information about Native Audio models and these features, see Proactivity and affective dialog.

Technical Specifications:

  • Audio input: 16-bit PCM at 16kHz (mono)
  • Audio output: 16-bit PCM at 24kHz (native audio models)
  • Video input: 1 frame per second, recommended 768x768 resolution
  • Context windows: Varies by model (typically 32k-128k tokens for Live API models). See Gemini models for specific limits.
  • Languages: 24+ languages supported with automatic detection

Choosing a platform

Both APIs provide the same core Live API technology, but differ in deployment platform, authentication, and enterprise features:

Aspect Gemini Live API Gemini Live API (Agent Platform)
Access Google AI Studio Google Cloud
Authentication API key (GOOGLE_API_KEY) Google Cloud credentials (GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION)
Best for Rapid prototyping, development, experimentation Production deployments, enterprise applications
Session Duration Audio-only: 15 min
Audio+video: 2 min
With context window compression: Unlimited		 Both: 10 min
With context window compression: Unlimited
Concurrent Sessions Tier-based quotas (see API quotas) Up to 1,000 per project (configurable via quota requests)
Enterprise Features Basic Advanced monitoring, logging, SLAs, session resumption (24h)
Setup Complexity Minimal (API key only) Requires Google Cloud project setup
API Version v1beta v1beta1
API Endpoint generativelanguage.googleapis.com {location}-aiplatform.googleapis.com
Billing Usage tracked via API key Google Cloud project billing

Live API Reference Notes

Concurrent session limits: Quota-based and may vary by account tier or configuration. Check your current quotas in Google AI Studio or Google Cloud Console.

Official Documentation: Gemini Live API Guide | Gemini Live API (Agent Platform) Overview

Why build live agents on ADK

Building realtime Agent applications from scratch presents significant engineering challenges. While Live API provides the underlying streaming technology, integrating it into production applications requires solving complex problems: managing WebSocket connections and reconnection logic, orchestrating tool execution and response handling, persisting conversation state across sessions, coordinating concurrent data flows for multimodal inputs, and handling platform differences between development and production environments.

ADK transforms these challenges into simple, declarative APIs. Instead of spending months building infrastructure for session management, tool orchestration, and state persistence, developers can focus on defining agent behavior and creating user experiences. This section explores what ADK handles automatically and why it's the recommended path for building production-ready streaming applications.

Raw Live API v. ADK Gemini Live API Toolkit:

Feature Raw Live API (google-genai SDK) ADK Gemini Live API Toolkit (adk-python and adk-java SDK)
Agent Framework ❌ Not available ✅ Single agent, multi-agent with sub-agents, and sequential workflow agents, Tool ecosystem, Deployment ready, Evaluation, Security and more (see ADK Agent docs)
Tool Execution ❌ Manual tool execution and response handling ✅ Automatic tool execution (see Tool call events)
Connection Management ❌ Manual reconnection and session resumption ✅ Automatic reconnection and session resumption (see Live API session resumption)
Event Model ❌ Custom event structures and serialization ✅ Unified event model with metadata (see Events)
Async Event Processing Framework ❌ Manual async coordination and stream handling ✅ LiveRequestQueue, run_live() async generator, automatic bidirectional flow coordination (see Sessions and Events)
App-level Session Persistence ❌ Manual implementation ✅ SQL databases (PostgreSQL, MySQL, SQLite), Agent Platform, in-memory (see ADK Session docs)

Platform flexibility

One of ADK's most powerful features is its transparent support for both Gemini Live API and Gemini Live API (Agent Platform). This platform flexibility enables a seamless development-to-production workflow: develop locally with Gemini API using free API keys, then deploy to production with Agent Platform using enterprise Google Cloud infrastructure—all without changing application code, only environment configuration.

How platform selection works

ADK uses the GOOGLE_GENAI_USE_ENTERPRISE environment variable to determine which Live API platform to use:

  • GOOGLE_GENAI_USE_ENTERPRISE=FALSE (or not set): Uses Gemini Live API via Google AI Studio
  • GOOGLE_GENAI_USE_ENTERPRISE=TRUE: Uses Gemini Live API (Agent Platform) via Google Cloud

This environment variable is read by the underlying google-genai SDK when ADK creates the LLM connection. No code changes are needed when switching platforms—only environment configuration changes.

Development: Gemini Live API (Google AI Studio)
# .env.development
GOOGLE_GENAI_USE_ENTERPRISE=FALSE
GOOGLE_API_KEY=your_api_key_here

Benefits:

  • Rapid prototyping with free API keys from Google AI Studio
  • No Google Cloud setup required
  • Instant experimentation with streaming features
  • Zero infrastructure costs during development
Production: Gemini Live API (Agent Platform)
# .env.production
GOOGLE_GENAI_USE_ENTERPRISE=TRUE
GOOGLE_CLOUD_PROJECT=your_project_id
GOOGLE_CLOUD_LOCATION=us-central1

Benefits:

  • Enterprise-grade infrastructure via Google Cloud
  • Advanced monitoring, logging, and cost controls
  • Integration with existing Google Cloud services
  • Production SLAs and support
  • No code changes required - just environment configuration

By handling the complexity of session management, tool orchestration, state persistence, and platform differences, ADK lets you focus on building intelligent agent experiences rather than wrestling with streaming infrastructure. The same code works seamlessly across development and production environments, giving you the full power of Bidi-streaming without the implementation burden.

Architecture

Now that you understand Live API technology and why ADK adds value, let's explore how ADK actually works. This section maps the complete data flow from your application through ADK's pipeline to Live API and back, showing which components handle which responsibilities.

You'll see how key components like LiveRequestQueue, Runner, and Agent orchestrate streaming conversations without requiring you to manage WebSocket connections, coordinate async flows, or handle platform-specific API differences.

High-level architecture

graph TB
    subgraph "Application"
        subgraph "Client"
            C1["Web / Mobile"]
        end

        subgraph "Transport Layer"
            T1["WebSocket / SSE (e.g. FastAPI)"]
        end
    end

    subgraph "ADK"
        subgraph "ADK Gemini Live API Toolkit"
            L1[LiveRequestQueue]
            L2[Runner]
            L3[Agent]
            L4[LLM Flow]
        end

        subgraph "LLM Integration"
            G1[GeminiLlmConnection]
            G2[Gemini Live API / Gemini Live API on Agent Platform]
        end
    end

    C1 <--> T1
    T1 -->|"live_request_queue.send()"| L1
    L1 -->|"runner.run_live(queue)"| L2
    L2 -->|"agent.run_live()"| L3
    L3 -->|"_llm_flow.run_live()"| L4
    L4 -->|"llm.connect()"| G1
    G1 <--> G2
    G1 -->|"yield LlmResponse"| L4
    L4 -->|"yield Event"| L3
    L3 -->|"yield Event"| L2
    L2 -->|"yield Event"| T1

    classDef external fill:#e1f5fe,stroke:#01579b,stroke-width:2px
    classDef adk fill:#f3e5f5,stroke:#4a148c,stroke-width:2px

    class C1,T1 external
    class L1,L2,L3,L4,G1,G2 adk
Developer provides: ADK provides: Live API provide:
Web / Mobile: Frontend applications that users interact with, handling UI/UX, user input capture, and response display

WebSocket / SSE Server: Real-time communication server (such as FastAPI) that manages client connections, handles streaming protocols, and routes messages between clients and ADK

Agent: Custom AI agent definition with specific instructions, tools, and behavior tailored to your application's needs		 LiveRequestQueue: Message queue that buffers and sequences incoming user messages (text content, audio blobs, control signals) for orderly processing by the agent

Runner: Execution engine that orchestrates agent sessions, manages conversation state, and provides the run_live() streaming interface

RunConfig: Configuration for streaming behavior, modalities, and advanced features

Internal components (managed automatically, not directly used by developers): LLM Flow for processing pipeline and GeminiLlmConnection for protocol translation		 Gemini Live API (via Google AI Studio) and Gemini Live API (Agent Platform) (via Google Cloud): Google's real-time language model services that process streaming input, generate responses, handle interruptions, support multimodal content (text, audio, video), and provide advanced AI capabilities like function calling and contextual understanding

This architecture demonstrates ADK's clear separation of concerns: your application handles user interaction and transport protocols, ADK manages the streaming orchestration and state, and Live API provide the AI intelligence. By abstracting away the complexity of LLM-side streaming connection management, event loops, and protocol translation, ADK enables you to focus on building agent behavior and user experiences rather than streaming infrastructure.

Live demos

  • LensMosaic: Visual Shopping with Live AI

    LensMosaic screenshot

    A demo app that merges live camera input, voice interaction, and intelligent product discovery. Point your camera at any object to find similar products, combine visual and voice input for personalized recommendations, or chat with a real-time AI shopping assistant. Built with ADK live agents, Gemini Embedding, Vector Search, and FastAPI.

  • Bidi Demo

    Bidi Demo screenshot

    A reference implementation with multimodal support (text, audio, image). This FastAPI-based demo shows real-time WebSocket communication, automatic transcription, tool calling with Google Search, and complete streaming lifecycle management. It is the demo referenced throughout these pages.

More resources

  • Blog post: A Visual Guide to Bidi-streaming

    A visual guide to real-time multimodal AI agent development with ADK, with diagrams and illustrations covering how streaming works and how to build interactive agents.

  • Blog post: Google ADK + Gemini Live API

    How to use live agents in ADK for real-time audio/video streaming, with a Python server example using LiveRequestQueue to build custom, interactive agents.

  • Blog post: Supercharge ADK Development with Claude Code Skills

    Using Claude Code Skills to accelerate ADK development, with an example of building a streaming chat app.