1. The Latency, Privacy, and Shell-Synthesis Problem
Most terminal assistants suffer from two architectural flaws: they depend on remote cloud APIs with 1,500ms–4,000ms round-trip latency, and they rely on raw shell command synthesis (bash -c "..."). Allowing a small language model to synthesize arbitrary shell strings on a live workstation creates an unacceptable risk of syntax corruption or destructive flag hallucination.
On local workstations, allocating 16GB–48GB of VRAM solely for an always-on terminal assistant is equally impractical. The engineering objective of the NanoHat project was straightforward: build an on-demand CLI tool-calling runtime powered by a sub-1B parameter model (~700 MB GGUF) that runs on local CPU cores in under 700ms and dispatches strictly through typed Python functions with shell=False.
2. Orthogonal 10-Tool Schema Contracts
Sub-1B models degrade rapidly when tool definitions overlap or accept free-form command strings. To maximize argument accuracy and enforce a clear separation between read-only telemetry and state-mutating actions, NanoHat exposes exactly 10 typed tools in runtime/tools.py:
# NanoHat v4.0 / v4.1 Canonical 10-Tool Signatures
system_health(metric: "battery" | "ram" | "cpu" | "all")
power_profile(action: "get" | "set", profile?: "power-saver" | "balanced" | "performance")
toggle_wifi(state: "status" | "on" | "off" | "toggle")
toggle_bluetooth(state: "status" | "on" | "off" | "toggle")
service_status(service_name: str)
restart_service(service_name: "pipewire" | "pipewire-pulse" | "wireplumber" | "xdg-desktop-portal" | "xdg-desktop-portal-gnome" | "ollama")
disk_usage(path: str = "/")
package_manager(action: "search" | "is_installed" | "install_cmd", package_name: str, provider: "dnf" | "flatpak" = "dnf")
audio_control(action: "status" | "mute" | "unmute" | "set_volume", level?: int)
desktop_appearance(setting: "theme" | "night_light", value: "status" | "dark" | "light" | "on" | "off")
Every multi-action tool explicitly separates read-only status inspection (action="get", state="status", value="status") from state mutation. When a user asks an informational question like "Is Wi-Fi turned on?", the model emits toggle_wifi(state="status") rather than accidentally toggling the radio.
3. Four-Layer Runtime Execution Guardrails
Even with a specialized tool-calling model, the host execution harness must never blindly trust raw model output. In NanoHat, every `NanoHat-0.6B passes through four deterministic runtime layers in runtime/engine.py and runtime/functions.py:
Canonical Argument Coercion
Maps minor parameter alias variations to canonical schema keys and strips duplicate tool calls before dispatch—without using keyword or regex heuristics on the user prompt.
Wayland Crash Shield & Daemon Allowlist
Blocks critical display/session services (gnome-shell, gdm, wayland) inside restart_service and restricts restarts strictly to allowlisted user-space daemons.
Zero-Shell Native Dispatch
Invokes system utilities (nmcli, powerprofilesctl, wpctl, gsettings, systemctl) exclusively via subprocess.run([...], shell=False) argument lists.
Dry-Run Verification Mode
Running with --sandbox (NANOHAT_SANDBOX=1) executes read-only telemetry queries against live hardware while safely simulating state-mutating calls.
4. Minimal Python Harness vs. Heavyweight Agent Frameworks
General-purpose agent frameworks (LangChain, LlamaIndex, smolagents) pull in dozens of transitive packages and add noticeable import overhead to CLI startup. NanoHat's runtime harness uses Python's standard library (subprocess, shutil, json, re) alongside two lightweight libraries: psutil for hardware telemetry and ollama for local IPC over 127.0.0.1:11434.
| Runtime Dimension | Heavyweight Agent Frameworks | NanoHat CLI Harness (v4.0) |
|---|---|---|
| Runtime Dependencies | 45+ packages (LangChain, Pydantic, etc.) | Python stdlib + psutil & ollama |
| Command Execution Model | Often wraps bash -c shell strings |
10 typed functions with shell=False |
| Median Inference Latency (P50) | 1,500 ms – 4,000 ms (Cloud API) | 672 ms (v4.0) / 607 ms (v4.1) on Local CPU |
| Model Footprint | Remote API or 8B+ Local LLM | ~700 MB GGUF (Qwen3-0.6B Q8_0) |
5. Evaluation Takeaways: Why Safety Gates Govern Default Releases
Across our 299-query unseen evaluation benchmark (239 system tool queries + 60 arithmetic, conceptual, and destructive security refusal queries), NanoHat v4.0 improved exact tool match on Qwen3-0.6B from 66.95% to 90.38% (97.61% on single-tool queries) and canonical argument accuracy from 51.05% to 86.19%, while achieving a 99.58% sandbox execution pass rate at 672 ms P50 latency.
Equally important is how multi-dimensional benchmarking informs deployment discipline: while our experimental v4.1 checkpoint reduced P50 latency to 607 ms and boosted multi-tool chain exact match from 40.00% to 56.67%, it regressed on destructive security refusal compliance (60.0% vs 90.0%) and non-tool clean rate (83.33% vs 96.67%). Maintaining v4.0 as the default production model (asimansari-o/nanohat:latest) while curating v4.1.1 ensures that raw tool-chaining gains never come at the expense of host system safety.