Voice

Overview FairyField is a desktop AI companion that lives on your desktop as a transparent overlay. Unlike typical AI tools, she’s designed to be a companion — a 3D character with emotions, voice, and long-term memory who remembers your conversations from yesterday. 有灵魂的桌面 AI 伴侣 — 悬浮在桌面上，会笑、会哭、会生气。记得你昨天的烦恼，会用真实的声音跟你聊天。 Philosophy In an era flooded with AI tools, FairyField takes a different path — she’s not a tool, she’s a companion. Not another ChatGPT wrapper Not a cold command-line bot A 3D character with emotions, living on your desktop 100% offline voice processing — your voice data never leaves your machine Long-term memory — she remembers what you said Key Features Module Description 3D Character VRM model with emotion-driven expressions, lip-sync, and eye tracking Offline Voice Paraformer speech recognition + Kokoro speech synthesis + Silero voice detection AI Soul Powered by Claude / GPT with autonomous tool calling and emotional companionship Voice Input Click microphone → speak → auto-transcribe → AI response → voice output Long-Term Memory 4-layer memory stack + temporal knowledge graph + vector semantic search Privacy First Voice 100% offline, memory stored in local SQLite Desktop Overlay Transparent always-on-top window with click-through, non-intrusive Security Prompt injection detection + command whitelist + automatic key sanitization Architecture ┌──────────────────────────────────┐ │ Tauri Transparent Window │ │ ┌──────────┐ ┌───────────────┐ │ │ │ 3D Char │ │ Chat Panel │ │ │ │ Three.js │ │ Vue 3 Bubbles │ │ │ │ + VRM │ │ + Voice Input │ │ │ └──────────┘ └───────────────┘ │ ├──────────────────────────────────┤ │ Tauri IPC Layer │ ├──────────────────────────────────┤ │ Rust Backend │ │ ├── agent/ AI Soul │ │ ├── voice/ ASR/TTS/VAD │ │ ├── memory/ Memory + KG │ │ ├── tools/ Tool System │ │ ├── security/ Safety Guard │ │ ├── gateway/ Discord Notify │ │ └── growth/ Autonomous Growth │ └──────────────────────────────────┘ Tech Stack Layer Technology 3D Rendering Three.js + @pixiv/three-vrm Desktop Framework Tauri 2.0 (Rust) Frontend Vue 3 + TypeScript AI Conversation OpenAI / Anthropic API Speech Recognition sherpa-onnx Paraformer Speech Synthesis sherpa-onnx Kokoro Voice Detection sherpa-onnx Silero VAD Storage SQLite + FTS5 + sqlite-vec Knowledge Graph Temporal Triples Security Prompt Injection Detection + Command Guard + Key Sanitization Project Status Phase Content Status 0 – 4.5 Architecture → Core → Voice → Intelligence → Polish Done 5 MVP Release (real voice engine + voice input UI) Done Future Holographic projection · Data collection · More platforms Planned Testing 341 tests passed (248 Rust + 93 frontend) Source Code Available on GitHub.