Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
FluidAudio is a Swift SDK from FluidInference for local, low-latency audio AI on Apple devices, offloading inference to the Apple Neural Engine via CoreML for background and always-on workloads on macOS and iOS. It covers speech recognition, text-to-speech, voice activity detection, and speaker diarization, plus a beta speech-enhancement module, using open-source models FluidInference converts to CoreML. The repo describes these broadly as "MIT/Apache 2.0," but licenses vary by model: Kokoro TTS and LocalVQE are Apache-2.0, Silero VAD and the streaming pyannote-3.1 pipeline are MIT, while the default Parakeet TDT ASR models and the default offline pyannote-community-1 diarization model are CC-BY-4.0, and NVIDIA's Sortformer diarizer carries NVIDIA's own Open Model License. Batch transcription defaults to Parakeet TDT v3 (0.6B, 25 European languages), with moondream's Parakeet Ultra and Parakeet Redux as alternatives. On the FLEURS 24-language benchmark (2024 MacBook Pro, M4 Pro), Parakeet TDT v3 averages 14.7% WER at 209.8x realtime; on LibriSpeech test-clean it averages 2.5% WER at 155.6x overall RTFx. Streaming ASR with end-of-utterance detection uses the 120M Parakeet EOU model. The default offline diarizer, re-run on the official AMI SDM 16-meeting split on an Apple M5 Pro, averages 10.6% DER at 323.2x RTFx, near pyannote-community-1's published ~11-12%. Two streaming diarizers are also included: LS-EEND (Westlake University, up to 10 speakers) and Sortformer (up to 4 speakers). TTS is marked beta with two backends, PocketTTS (streaming, voice cloning from a 1-30s sample) and Kokoro (82M, parallel synthesis, SSML/lexicon control) — its own beta note still limits official support to American English. LocalVQE, an echo canceller/noise suppressor for 16kHz audio, is also beta: on an exploratory AEC-Challenge synthetic subset it raises ASR near-end word recall from 44.1% to 77.6% and cuts far-end word leakage from 34.0% to 1.1%, though it hasn't been exercised in production call pipelines yet. It ships via Swift Package Manager and CocoaPods with React Native/Rust wrappers, plus an opt-in offline mode for bundled models; the SDK itself is Apache-2.0.