A voice agent powered by wake word detection, streaming speech recognition, an on-device large language model, and streaming text-to-speech. Runs entirely on the device across mobile, embedded, desktop, and browser with no cloud processing.
On-device AI voice agent listens for wake word, transcribes the user's speech in real time, generates a response using a local large language model, and speaks the answer back, all without sending any data to a cloud service. Picovoice's Porcupine Wake Word, Cheetah Streaming Speech-to-Text, picoLLM on-device LLM, and Orca Streaming Text-to-Speech compose into a single pipeline.
Porcupine Wake Word enables always-on, hands-free activation for the voice agent. It listens continuously with minimal CPU, hence battery, usage. So the device can stay ready without draining resources. When the user says the wake word, Porcupine immediately interrupts any in-progress LLM response and hands control to the speech-to-text engine. Enterprises can train branded wake words in seconds using the Picovoice Console and deploy them across all supported platforms.
Cheetah transcribes the user's question in real time as they speak, streaming words with an average emission latency of 590 ms. Cheetah matches or beats cloud STT API accuracy and supports custom vocabulary for industry-specific terms: product names in retail, drug names in clinical settings, part numbers in manufacturing. Cheetah requires less compute than any other local engine tested, efficiently running alongside picoLLM and Orca in the same session.
picoLLM runs custom-trained LLMs and open-weight language models (Llama, Gemma, Phi, Mistral) locally on CPU or GPU with no cloud dependency. picoCompression quantizes language models to run on phones, browsers, and embedded boards with no cloud dependency while preserving the task accuracy. picoLLM's minimal memory footprint allows it to run alongside Cheetah and Orca in a single session without network latency or privacy exposure.
Orca speaks the LLM's response as tokens stream in, so the user hears the answer before generation is complete. Orca achieves first-token-to-speech latency of 130 ms and a peak memory footprint of 29 MB, allowing it to run alongside Porcupine, Cheetah, and picoLLM with no performance issues. The result is a natural-sounding conversational voice agent with no perceptible pause between the user's question and the spoken answer.
LLM voice AI agents can handle commonly asked questions, such as "How do I connect to the office printer?" instantly without waiting in a ticket queue. The on-device AI voice agent resolves routine queries from an internal knowledge base, keeping employee interactions off third-party servers.
Retailers, banks, and telcos can deploy conversational voice agents on kiosks, POS terminals, and in-store displays that answer customer questions, look up information, and guide users through processes. The on-device stack has no dependency on network availability in stores, branches, or service centers.
Clinicians can query drug interactions, dosing guidelines, or documentation templates by voice during patient encounters. Running the voice agent on-device eliminates the need for a BAA covering voice data with a cloud LLM provider. No audio or text is ever transmitted to any third-party service.
Automakers and fleet operators can embed voice agents that let drivers control navigation, query vehicle diagnostics, or dictate messages without taking their hands off the wheel. The on-device stack eliminates dead zones on rural highways or in parking garages where cellular connectivity drops.
A complete working recipe in Python. Open-source on GitHub. Runs 100% on-device.