Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**qwen-audio-agent** is a realtime voice runtime that sits in front of a coding agent you already use. It handles full-duplex speech — listening, interrupting, responding — while a backend agent such as OpenCode, Claude Code or Codex executes long-running work in the background, and it returns the result into the same conversation when the task finishes. It was open-sourced on **28 July 2026** by the **QwenAudio team** — Alibaba Tongyi Lab's speech group, the org behind CosyVoice, SenseVoice, FunASR and ThinkSound. In roughly ten days it has reached about **1,964 stars** and **142 forks** across **14 contributors**, shipping from v0.9.0 to **v1.6.0** (6 August) at nearly a release a day. It is Apache-2.0 licensed and distributed on npm. ## The Problem It Targets Voice interfaces for agents tend to break at the same point: you ask for something that takes real work, and the conversation stops. The agent goes quiet while it runs tools, and you sit in silence with no idea whether it is thinking, stuck, or done. qwen-audio-agent splits the interaction into two planes. Questions answerable directly get an immediate spoken response from the realtime voice model. Work that needs tools or sustained processing is delegated to a backend agent, and **the conversation keeps going while that runs** — you can ask for progress, start additional tasks, or cancel. When a background task completes, its result flows back into the current conversation for follow-up. From the user's side it stays one assistant throughout. ## Bring Your Own Agent The integration layer is **ACP (Agent Client Protocol)**, which makes the backend swappable rather than bundled: | Backend | Integration | |---|---| | OpenCode, OpenClaw, Qoder, Kimi Code | Native ACP, one-click install | | Hermes, CodeBuddy | Native ACP, user configuration required | | Codex, Claude Code | External ACP adapter | | Any ACP-over-stdio agent | Generic entry point via `ACP_COMMAND` / `ACP_ARGS` | | None | Frontend-only voice chat, no setup | The important detail is what happens with an already-configured agent: qwen-audio-agent reuses its existing model, tools, MCP servers, Skills and authentication rather than asking you to re-declare them. The generic ACP entry point means adding a new backend requires no gateway code changes. The voice frontend is swappable too. The default path uses Alibaba's DashScope realtime models (`qwen-audio-3.0-realtime-flash` or `-plus`), but the runtime also speaks to a self-hosted **Hugging Face speech-to-speech** server exposing an OpenAI Realtime-compatible WebSocket — VAD, STT, LLM and TTS all local, no cloud key required. The gateway does not override the models configured there. ## Surfaces and Recent Additions Three interfaces ship: a terminal TUI, a WebUI, and a desktop app with a persistent voice orb that embeds the gateway. The orb hides when idle — you can dismiss it by saying "you can go now" — and returns via a menu-bar item or `⇧⌘ Space`. The gateway can also be installed as a background user service (`qwenaudio gateway install`). The August release train has been dense: Kimi Code backend (v1.1.0), desktop auto-update (v1.2.0), the local speech-to-speech frontend (v1.3.0), one-click backend install and behavior rules (v1.4.x), scheduled reminders, progress reporting and a "你好千问" wake word (v1.5.0), then Windows desktop support and automatic memory extraction after each session (v1.6.0). Personal data lives in `~/.config/qwaudio/`, including a `USER.md` profile. ## Caveats **The default path is a cloud dependency.** Without the local speech-to-speech route, this needs a DashScope API key from Alibaba Cloud Model Studio. There is a free tier for new accounts, but the out-of-box experience is billable Chinese-cloud infrastructure, which is a procurement question for some teams and a latency question for users far from those regions. **Full duplex is a macOS privilege.** Only macOS defaults to full duplex with echo cancellation. Linux and Windows run half duplex, where interrupting means pressing `x` during playback. You can force full duplex, but the docs tell you to wear headphones or speaker output will be transcribed as your own speech. **Desktop distribution is uneven.** Official installers are macOS-only. Linux users build AppImage or deb themselves; Windows support landed one day before this writing. **Version churn is real.** Eight minor releases in nine days is a healthy signal about momentum and a warning about stability. Configuration keys and adapter behavior are still moving. **One permission flag deserves attention.** `QWEN_AUDIO_AGENT_BACKEND_PERMISSION_MODE=full` removes the backend agent's per-action confirmation. Combined with voice input and background execution, that means spoken commands can run tools and modify files without a prompt. The default (`native`) is the right choice outside trusted projects. ## Verdict The interesting design decision here is that qwen-audio-agent does not try to be an agent. It is a voice layer with an opinion about conversational continuity, and it delegates the actual work to whatever agent you have already configured — including direct competitors to its own vendor. That makes it unusually easy to evaluate: point it at your existing setup and see whether talking to your agent while it works is better than watching a terminal. Teams already on OpenCode or OpenClaw get the smoothest path; Claude Code and Codex users go through an adapter. The local speech-to-speech route is what makes it viable for anyone who cannot send audio to a cloud endpoint, and it is the part most worth testing before committing.