Architecture

remnix is a local-first shell history manager. A device writes commands to SQLite, optionally encrypts them into event bundles, and stores those objects on a remote filesystem. There is no remnix server.

TUI / wizard / picker / remnix-attach / shell hooks
        │
   remnix daemon       (control.sock + terminal.sock)
        │
   history.Cache       (RAM projection) + history.Store (SQLite WAL)
        │
   internal/config     (portable YAML; rclone.conf is in the data dir)
        │
   internal/app        (endpoint factory, fan-out sync, lock, callbacks)
        │
   internal/sync/syncer  (events, checkpoints, GC, keys)
        │
   Endpoints: directory | rsync | scp | rclone  (any mix, all mirrors)
        │
   internal/repository.Probe   (empty / unrelated / valid / partial / unsupported)

One long-lived remnix daemon per user session owns SQLite, the interactive history cache, periodic sync/GC, inner PTYs, and overlay TUIs (Ctrl+R search and Ctrl+Space suggest). Terminals attach with the tiny remnix-attach helper (not another copy of the Go binary). Shell widgets trigger overlays over the control socket (search-interactive / suggest-interactive); attach stays a byte pump. A daemon crash cannot reattach existing PTYs; the service manager should restart it and new shells will reconnect.

rclone is a filesystem adapter. remnix never calls rclone sync or rclone bisync on the native path. Merge, encryption, and GC stay in remnix.

The encrypted repository is the endpoint path on the provider (for example gdrive:remnix on Google Drive). This device keeps SQLite history, local.yaml, and rclone.conf under $XDG_DATA_HOME/remnix. A folder such as ~/GoogleDrive/remnix is not the repository unless it is a type: directory endpoint in config.yaml.

A device may have unlimited enabled endpoints (Google Drive, S3, Dropbox, a local folder, …). Each is a mirror of the same encrypted repository. Sync pulls and publishes per endpoint, then equalizes missing objects so a reachable subset still converges the rest.

rsync and scp are session transports (Begin/End staging). rclone talks to the cloud directly.

The exclusive mutator for remote writes is internal/app’s file lock (remnix.lock), used by the daemon, manual sync (via RPC when the daemon is up), GC, and key/device ops. Normal history mutations go through the daemon so SQLite and the RAM cache stay coherent.

Scale test

TestScaleMillionUniqueCommands in internal/history seeds unique commands into a temp SQLite file and reports wall time for the interactive paths:

Skipped unless REMNIX_SCALE=1 is set (go test ./... and -short skip it). Default size is 1 000 000 unique commands; override with REMNIX_SCALE_N.

REMNIX_SCALE=1 go test ./internal/history/ -run TestScaleMillionUniqueCommands -timeout 45m -v

PTY forwarding

When pty_proxy is enabled, remnix-attach is a byte pump between /dev/tty and the daemon’s inner PTY over terminal.sock. Keyboard input is authoritative. Shadow VT parsing is best-effort and must never stall real I/O.

/dev/tty keys
    -> remnix-attach streaming classifier
         consume only explicit emulator replies (focus, CPR, OSC/DCS, query)
         unknown/incomplete sequences are forwarded unchanged
    -> lossless input FIFO (never drop-oldest/drop-newest)
    -> single socket writer
    -> daemon ReadFrame
    -> lossless PTY input FIFO
    -> full PTY write -> child

child PTY output
    -> terminal forwarding (must stay fast)
    -> bounded shadow parse queue (may drop stale backlog; never backpressures)

Invariants:

• Read and socket-frame boundaries have no semantic meaning.
• A timeout may flush an incomplete escape sequence; it must not discard it.
• Write is never assumed to accept the full buffer; attach and daemon retry short writes, EINTR, and EAGAIN.
• Keyboard queues that exhaust their byte budget fail the session instead of dropping keys.
• REMNIX_PTY_DIAG=1 prints attach/daemon counters; it does not log keys.

Further reading: sync protocol, cryptography, rclone, threat model, configuration.