The Deck
The written record.
A ledger-style console: the conversation, the approvals answered and declined, and the consent stamps, kept as a record you can scroll back through rather than a feed that evaporates.
01The research goal
The aim is an aide, not a chat window: one system you hand real work to, that gets it done on your own machines, and that interrupts you the way a competent assistant would — rarely, and with reason.
It runs at home, on hardware the lab owns. Conversations, files, and the record of what was done stay on the private network. The cloud is an escalation tier, not the default.
The hard problem is trust. An assistant that touches your files has to be honest about what it touched. So the same rule that governs Nocturn's sensors governs this system's hands: never present inference as observation, never claim an action the record cannot show.
02How it works
Work arrives by voice or text on any paired device. A router decides whether the turn needs hands; turns that do go to a broker that holds the only keys to the file system, git, and the network, jailed to the workspace and governed by consent gates. A request the operator does not approve within the window is declined, and a refusal on policy ends the turn rather than being retried elsewhere.
Before a reply is spoken, a checker compares its claims against the per-turn action record: what ran, what was written, what was committed. A reply that claims work the record cannot show is corrected. Longer jobs can be handed off — the agent acknowledges, works in the background, and narrates progress only from real tool events, never from imagination.
The voice is its own: streaming ears that can be interrupted mid-sentence, and a cloned voice on the reply side. Latency budgets are measured, not hoped for.
The surfaces
3 instruments, one discipline
Voice
Hands-free, interruptible.
Streaming speech recognition with barge-in — talk over it and it stops — and spoken replies in a consistent voice. The same turns, the same record, no separate mode.
The Broker
Hands under consent.
The only component with authority to act. Sixteen tools for files, git, search, and the shell, jailed to the workspace, with secrets refused at the read layer and every gated call answered on whatever device is closest.
What the lab built
- A per-turn action record that every reply is verified against before it is delivered
- Consent gates with a decline default: no answer in two minutes means no
- Local-first routing with measured escalation, so the private tier does the work it can prove it can do
- Background dispatch that narrates from real tool events only
- Lines of code
- 59,131Lines of code
- Commits
- 126Commits
- Files
- 199Files
- Building since
- Aug 2026Building since
- Python
- FastAPI
- WebSockets
- NATS
- llama.cpp
- Local models
At a glance
How the system flows
- 01
Ask
Voice or text, from any paired device
- 02
Route
Local first; the cloud only when a turn earns it
- 03
Act
The broker's tools, behind consent gates
- 04
Verify
Claims checked against the action record
- 05
Answer
Spoken or written, with its provenance attached