An alpha assistant combining Ollama and optional MLX models with rule-based routing, memory, tool experiments, voice and a React/Three.js interface.
Prototype
Python
FastAPI
Ollama
MLX
Redis
ChromaDB
SQLite
React
Three.js
Original code-generated category illustration; not a product screenshot.
Overview
J.A.R.V.I.S brings several model and interface components into one assistant prototype. Its Python package includes routing and orchestration, Redis/ChromaDB/SQLite memory, and tool and voice modules. A React/Three.js visual client accompanies the FastAPI backend. The MLX path requires supported Apple hardware. This repository is separate from jarvis-core and its deterministic policy-centered runtime.
Problem and objectives
An assistant needs to handle short conversations, deeper analysis, tool use and user context within a coherent flow. Coupling every task to one model or module can mix hardware requirements with service failures. Adding memory, voice and powerful tools also makes data boundaries and execution limits more important. A modular prototype provides a place to explore these concerns without treating every integration as production-ready.
My role
Development of assistant orchestration, model adapters, memory and visual interface components.
Solution approach
A rule-based router selects fast/deep flows from message length, keywords and complexity. Ollama and optional MLX adapters abstract model access, while orchestration connects memory and tool modules. Redis, ChromaDB and SQLite serve distinct memory concerns. FastAPI supports a React/Three.js client; voice and separate device clients remain experimental layers. Setup has explicit requirements for model services and platform-specific extras rather than assuming every dependency runs on every device.
Implementation details
01
Keeping routing decisions explainable
Router evaluates Turkish/English deep-task keywords, message length and sentence count to select a fast or deep lane. Its decision includes a human-readable reason, and Orchestrator maps the lane to the selected model client. The code therefore exposes why a message took a heavier path. This avoids running a separate learned classifier for the choice, but does not establish semantic routing accuracy or a measured improvement in latency or cost.
02
Separating memory failure from response generation
For each request, Orchestrator assembles a system message, session history and the new user message. If short-term memory is absent or fails to read, it can continue stateless with empty history; after responding it attempts to store the user/assistant turn. Write failures are logged without canceling an already generated response. This defines behavior when memory is unavailable, while accepting possible conversational context loss. It is not a uniform continuity guarantee across every memory component.
03
The actual scope of profile encryption
The SQLite profile component stores key/value records; values written with encryption enabled use AES-GCM and are decrypted on reads. Schema and key names remain visible, and unencrypted records are also supported. Profile encryption therefore cannot be generalized to the entire database, logs, Chroma indexes or voice files. This distinction makes storage boundaries explicit during development. Key changes introduce availability and migration concerns; existing content is not assumed to be automatically re-encrypted.
Architecture flow
01
React / Three.js client → FastAPI
02
Rule-based router → fast / deep flow
03
Ollama / optional MLX adapters
04
Orchestration → memory / tool / voice modules
05
Redis / ChromaDB / SQLite → context and response
Conceptual flow derived from project descriptions; not an application screenshot.
Core components
Rule-based fast/deep model routing
Ollama and optional Apple MLX adapters
Multi-component memory layer
Web / code / shell tool experiments
Voice and wake-word modules
React / Three.js visual client
Technical challenges
Separating platform-specific MLX and voice dependencies from baseline tests.
Testing health contracts without reaching real model providers.
Keeping profile encryption, authentication and log/cache boundaries explicit.
Engineering decisions
Provider adapters separate model implementations; current routing uses deterministic rules.
The interface and Python backend have separate build paths.
Open PR #14 proposes loopback defaults and production-secret validation.
Testing and validation
Historical validation of open PR #14 passed 158 tests with the MLX extra and external sockets blocked; seven platform/gated cases were skipped.
Ruff/Black and mypy across 45 source files passed; the UI built with a large-chunk warning. Real-model and voice acceptance were not performed.
Outcomes
Model, memory, tool, API and UI components are organized as separate modules.
Fast/deep routing produces an inspectable deterministic decision and reason.
Limitations and current status
Alpha prototype without end-to-end validation for every device and service.
Encrypting selected profile values does not encrypt the whole SQLite file or logs.
Shell/file tools carry powerful access and remote services may transmit data.
Loopback and production-configuration hardening remain in unmerged PR #14, not an applied main-branch release.
The technical account also draws on documentation in the open improvement PR. This does not imply that the PR has been merged or the product released.
A compact language-model study exposing Transformer internals through a custom BPE tokenizer, PyTorch components and local Apple Silicon training experiments.