Julio AI Assistant
A voice-controlled business assistant with a holographic interface. Say the wake word and it checks your sites, follows up on CRM leads, drafts replies for your approval and keeps compliance reminders in view. Built on the open-source JARVIS framework and customised for running a small group of companies.
- Runs on
- Your own computer and phone, not a public website
- Base
- Open-source JARVIS framework (MIT licence), customised
- Brain
- Claude through the Claude Agent SDK, with an offline mode on local models
- Sites watched
- 20 sites, checked every 30 minutes
What it does
Features, as built.
Voice and interface
- Wake word ("Hey Julio"), then spoken questions and answers
- Three.js holographic face with custom shaders
- Hand-gesture control and a camera mode through the browser
- Speech recognition tuned for Indian English
- Tappable commands menu and an on-screen suggestions strip
Business tools we added
- Health check across all 20 websites, with Vercel and security-checkpoint states told apart from real outages
- Site-down alerts that notify only when a site goes down or comes back
- CRM follow-ups and pipeline counts read from the Odoo database, read-only
- A reminders calendar for compliance dates and bills, with advance notice
- A morning briefing built from calendar, mail, sites and CRM
- Notes memory that survives between sessions
Safety
- The bridge listens on localhost only
- Hooks block reads of secret files and redact keys from tool output
- Web access is locked for the session once mail has been read, to blunt prompt injection
- Email drafts and calendar changes need an on-screen click to approve, and nothing is sent automatically
- Phone mode over Wi-Fi uses HTTPS and a per-device key
Modes
- Fast mode on a Sonnet model for everyday use
- Offline mode on local Ollama models for private or bulk text
- Phone mode with a QR link to open it on a handset
Tech stack
Every technology used, and what it is used for.
Interface
| Technology | Used for |
|---|---|
| React 19 | UI |
| Vite 8 | Dev server and build |
| TypeScript 6 | Typed interface code |
| Three.js, React Three Fiber, drei, postprocessing | 3D holographic scene with custom GLSL shaders |
| Framer Motion | Interface animation |
| Zustand | Application state |
Voice and vision
| Technology | Used for |
|---|---|
| Picovoice Porcupine | Wake word detection in the browser |
| Web Speech API | Default speech recognition and speech |
| Kokoro (kokoro-js) | Local text to speech |
| ElevenLabs (optional) | Higher-quality voice and transcription |
| MediaPipe Tasks Vision | Hand tracking for gestures |
Brain and bridge
| Technology | Used for |
|---|---|
| Node.js 20+ | Local bridge server |
| Claude Agent SDK | Runs the assistant and its tools |
| WebSocket (ws) | Live link between the interface and the bridge |
| Zod | Tool input validation |
| DOMPurify | Sanitises content shown in panels |
| Ollama (optional) | Local models for offline mode |
Business integrations
| Technology | Used for |
|---|---|
| PostgreSQL via Docker | Read-only CRM queries against the local Odoo database |
| Scheduled tasks | Site-down alerts, morning brief and due-date reminders |
| Vite basic SSL plugin, qrcode | HTTPS and QR link for phone mode |
Tooling
| Technology | Used for |
|---|---|
| oxlint, tsx | Linting and running scripts |
Julio is built on JARVIS, an open-source framework released under the MIT licence. The voice interface, 3D scene and core assistant come from that framework. The business tools, safety layer, theme, phone mode and scheduled alerts listed above are our additions. It runs locally, so there is no public link.
Want something built the same way?
Tell us about your business and what you need, and we will come back with a plan, the stack we would use and why.
Get Your Free Growth Audit