Overview
FairyField is a desktop AI companion that lives on your desktop as a transparent overlay. Unlike typical AI tools, she’s designed to be a companion — a 3D character with emotions, voice, and long-term memory who remembers your conversations from yesterday.
有灵魂的桌面 AI 伴侣 — 悬浮在桌面上，会笑、会哭、会生气。记得你昨天的烦恼，会用真实的声音跟你聊天。
Philosophy
In an era flooded with AI tools, FairyField takes a different path — she’s not a tool, she’s a companion.
- Not another ChatGPT wrapper
- Not a cold command-line bot
- A 3D character with emotions, living on your desktop
- 100% offline voice processing — your voice data never leaves your machine
- Long-term memory — she remembers what you said
Key Features
| Module | Description |
|---|---|
| 3D Character | VRM model with emotion-driven expressions, lip-sync, and eye tracking |
| Offline Voice | Paraformer speech recognition + Kokoro speech synthesis + Silero voice detection |
| AI Soul | Powered by Claude / GPT with autonomous tool calling and emotional companionship |
| Voice Input | Click microphone → speak → auto-transcribe → AI response → voice output |
| Long-Term Memory | 4-layer memory stack + temporal knowledge graph + vector semantic search |
| Privacy First | Voice 100% offline, memory stored in local SQLite |
| Desktop Overlay | Transparent always-on-top window with click-through, non-intrusive |
| Security | Prompt injection detection + command whitelist + automatic key sanitization |
Architecture
┌──────────────────────────────────┐
│ Tauri Transparent Window │
│ ┌──────────┐ ┌───────────────┐ │
│ │ 3D Char │ │ Chat Panel │ │
│ │ Three.js │ │ Vue 3 Bubbles │ │
│ │ + VRM │ │ + Voice Input │ │
│ └──────────┘ └───────────────┘ │
├──────────────────────────────────┤
│ Tauri IPC Layer │
├──────────────────────────────────┤
│ Rust Backend │
│ ├── agent/ AI Soul │
│ ├── voice/ ASR/TTS/VAD │
│ ├── memory/ Memory + KG │
│ ├── tools/ Tool System │
│ ├── security/ Safety Guard │
│ ├── gateway/ Discord Notify │
│ └── growth/ Autonomous Growth │
└──────────────────────────────────┘
Tech Stack
| Layer | Technology |
|---|---|
| 3D Rendering | Three.js + @pixiv/three-vrm |
| Desktop Framework | Tauri 2.0 (Rust) |
| Frontend | Vue 3 + TypeScript |
| AI Conversation | OpenAI / Anthropic API |
| Speech Recognition | sherpa-onnx Paraformer |
| Speech Synthesis | sherpa-onnx Kokoro |
| Voice Detection | sherpa-onnx Silero VAD |
| Storage | SQLite + FTS5 + sqlite-vec |
| Knowledge Graph | Temporal Triples |
| Security | Prompt Injection Detection + Command Guard + Key Sanitization |
Project Status
| Phase | Content | Status |
|---|---|---|
| 0 – 4.5 | Architecture → Core → Voice → Intelligence → Polish | Done |
| 5 | MVP Release (real voice engine + voice input UI) | Done |
| Future | Holographic projection · Data collection · More platforms | Planned |
Testing
- 341 tests passed (248 Rust + 93 frontend)
Source Code
Available on GitHub.