Three generations of conversational architecture
Over the past decade, the stack behind these calls evolved through three distinct eras, moving from bespoke neural pipelines to a unified architecture using Gemini.
1. Pre-LLM closed-domain models
We started work on Google Duplex before modern language models and even before the transformer itself was invented. In these early days, to ensure businesses and users had a great experience, we constrained Duplex to narrow domains and trained custom recurrent neural network models on anonymized, task-specific phone conversations. Without pre-trained world knowledge, we taught our models how to handle every specific type of task efficiently. In practice, this meant relying on extensive training data for each task type — such as interpreting the rigid flow of restaurant booking versus checking holiday hours — to ensure the system could reliably navigate domain-specific logic.
2. Early generative models and hybrid stacks
Google Duplex started employing open domain foundation models in production as they emerged. For example, in 2022 Duplex used LaMDA in live calls. These pre-trained models simplified comprehension and common-sense reasoning, allowing the system to handle unexpected conversational turns and tangents, while remaining grounded in the context like a standard restaurant booking.
However, these early generative models were less predictable and sometimes slower than today’s, so we built hybrid stacks that paired generative models for flexible phrasing with deterministic state tracking and strict policy guardrails. This helped ensure that both users and businesses had a great experience.
3. Unbounded tasks and Gemini integration
As foundation models advanced, we removed the constraint of fixed verticals entirely by shifting to prompting the system with free-text task descriptions. This allowed models to define a task simply via a prompt, enabling the system to generalize to new task types and nuances (like asking for window seating or extra space for a stroller) and new business categories, without building separate dialogue models from scratch.
Our calling stack uses Gemini to unify multimodal understanding, planning, and dialogue generation so the system can navigate multi-constraint requests — like scheduling car maintenance around calendar conflicts — and adapt when a receptionist suggests another slot.
