agentic tui for jetson & edge devices

Run LLMs on-device. Agentic tools. Fully offline.

Name: open-jet
Author: open-jet

A terminal-native agent that runs quantized models on NVIDIA Jetson and Linux devices via llama.cpp — shell execution, file operations, memory-aware context, and structured logging, all without a network connection.

Why not cloud? Keep data local for privacy, cut round-trip latency, and stay reliable even when the network is down.

Get started How it works

  ____                    _      _   
 / __ \ _ __   ___ _ __  (_) ___| |_ 
| |  | | '_ \ / _ \ '_ \ | |/ _ \ __|
| |__| | |_) |  __/ | | || |  __/ |_ 
 \____/| .__/ \___|_| |_|| |\___|\__|
       | |              _/ |         
       |_|             |__/

Loading Qwen2.5-7B.Q4_K_M.gguf...
Ready.

> Check the system load and summarize it.

[agent] Tool request: shell("uptime") → approved
The system has a load average of 1.45. It looks like it has been running normally without excessive CPU pressure.

tokens: 284/6144 | prompt<= 4915 | remaining: 4631

util: cpu 14.2% | mem 61.3% (9.8/16.0 GB) | tps 12.4 | pwr 8.1W [min 4.2W max 15.0W]

process

How it works

Run setup in terminal

Auto-detects your device, configures GPU layers, and recommends a model sized for your RAM.

Load a local model

Point to a local GGUF file or pull from Ollama. Inference stays on-device via llama-server.

Chat with tools

Run shell commands and file operations. Anything that mutates state requires operator approval.

features

Built for constrained hardware

$ open-jet --features

Fully offline inference — Zero network dependency after initial model setup. Runs via llama.cpp.

4 agentic tools — shell, read_file, write_file, and load_file — LLM driven, gated by you.

Approval gates — State changes require explicit y/n approval with a compact execution preview.

Memory-aware context — Auto-condenses conversation when system RAM or token limits approach.

@file mentions & slash commands — @path loads files. Context-aware commands like /status, /condense, /load.

Structured session logging — JSONL event logs (prompts, tool calls) and metrics logs (CPU, memory, load).

use cases

Where it fits

NVIDIA Jetson devices

First-class support for Nano, Xavier NX, Orin Nano/NX, and AGX Orin.

Air-gapped & Edge deployments

Fully local operation. No cloud transport, no API keys, no latency. All inference and logs stay on-device.

Developer workstations

Lightweight terminal agent for local code assistance and shell automation.

compatibility

Supported hardware & stack

RuntimePython 3.10+ · Textual TUI

Inferencellama.cpp (llama-server)

Model formatGGUF (quantized)

Model sourcesLocal GGUF files · Ollama

Jetson targetsNano · Xavier NX · Orin Nano/NX · AGX Orin

Host OSLinux (CUDA or CPU-only)

deployment planner

What can your device run?

Device

16GB unified · 102.4GB/s · 25W TDP

Model

Qwen3.5 27B (preferred) · Q4_K_M · 17GB on disk

Context length

KV cache: 262,144 bytes/tok × 4K= 1.07GB (FP16, per model config.json)

Inference speed

3.9 tok/s102.4GB/s × 60% eff ÷ 17GB × 95% ctx. Varies with thermal state and power mode.

Memory

3.9GB over17GB weights + 1.1GB KV + 1.5GB OS = 19.9GB of 16GB.

Capability

Preferred Qwen3.5 dense tier with strong coding and reasoning.Based on public benchmarks for Qwen3.5 27B (preferred). Validate on your workload.

get started

pip install, run setup, start chatting.

$pip install open-jet

Contact How it works