name: site-architecture description: Capsem system architecture -- service daemon, per-VM processes, CLI, MCP server, guest agent, vsock, network proxy. Use when you need to understand the system design to write code, review changes, write documentation, or debug cross-component issues. Covers the service architecture, IPC protocols, vsock ports, storage modes, network policy, MITM proxy, and key source files.

Capsem Architecture

System overview

Capsem sandboxes AI agents in air-gapped Linux VMs on macOS using Apple's Virtualization.framework (with a KVM backend for Linux). It runs as a daemon service (like Docker). The system has these layers:

Host-side:

• capsem-service (daemon): always-running background service. Axum HTTP server over Unix Domain Socket (~/.capsem/run/service.sock). Manages VM lifecycle, routes API calls to per-VM processes.
• capsem-process (per-VM): one process per sandbox. Boots the VM, bridges vsock connections (terminal + control), manages structured jobs (exec, file I/O) via a job store.
• capsem (CLI): user-facing CLI. Everything is ephemeral unless asked otherwise. capsem shell (no args) = temp VM + auto-destroy on exit. capsem create -n <name> = persistent VM (detached). capsem create (no name) = ephemeral VM (detached). capsem shell <id> = attach to existing. Talks to capsem-service over UDS HTTP.
• capsem-mcp (MCP server): stdio-based MCP server for AI agents (Claude Code, Gemini CLI). Bridges MCP tool calls to capsem-service HTTP API.
• capsem-gateway (HTTP gateway): TCP-to-UDS reverse proxy (default port 19222). Bearer token auth, CORS, 10MB body limit. Provides /status (cached 1s), /terminal/{id} (WebSocket relay to per-VM UDS), and transparent fallback proxy to capsem-service. The frontend and tray app connect through the gateway. Writes runtime files to ~/.capsem/run/ (gateway.token, gateway.port, gateway.pid).
• capsem-app (Tauri GUI): thin webview shell. Connects to gateway at http://127.0.0.1:19222. No VM logic, no capsem-core dependency. Only 2 IPC commands: open_url (opens URL in system browser) and check_for_app_update (Tauri updater). Bundles frontend/dist as offline fallback when gateway is unreachable.
• capsem-tray (system tray): menu-bar companion process. Polls the gateway for VM status, shows running/stopped counts, and provides quick actions (open dashboard, quit). Non-standalone: refuses to run without --parent-pid pointing at a live capsem-service, acquires a system-wide singleton lock at ~/.capsem/run/tray.lock (only one tray ever in the menu bar), and self-exits within 500ms when its parent dies. Contract enforced by capsem-guard on the companion side, not the spawner.
• capsem-guard (shared library): parent-watch + singleton primitives used by capsem-tray and capsem-gateway. Provides watch_parent_or_exit, Singleton::try_acquire, and the umbrella install(parent_pid, lock_path). Guarantees companions die with their parent and can't run standalone or as multiple instances -- closes the orphan-accumulation class of bug that kill_on_drop(true) alone cannot cover under SIGKILL/OOM/test-harness termination. See /dev-rust-patterns lesson 18.

Guest-side:

• capsem-init (capsem-init): PID 1, sets up air-gapped networking, mounts filesystems, deploys guest binaries, launches daemons, writes boot timing JSONL
• capsem-pty-agent (capsem-pty-agent): main guest agent -- PTY bridge, control channel, exec, file I/O, shutdown handler (see "Guest agent architecture" below)
• capsem-sysutil (capsem-sysutil): multi-call binary for guest lifecycle commands (shutdown, halt, poweroff, reboot, suspend). Opens its own vsock:5004 connection independently of the agent, so shutdown works even if the agent is hung. Symlinked by capsem-init to /sbin/shutdown, /sbin/halt, /sbin/poweroff, /sbin/reboot, /usr/local/bin/suspend.
• capsem-net-proxy (capsem-net-proxy): redirects HTTPS traffic to host MITM proxy via vsock
• capsem-mcp-server (capsem-mcp-server): in-guest MCP gateway, routes tool calls to external MCP servers via vsock

Service architecture

All VM operations go through a single path. There is no direct VM boot -- every entry point routes through capsem-service to capsem-process.

AI Agent  -> capsem-mcp (stdio)  -> HTTP/UDS -> capsem-service
User      -> capsem CLI          -> HTTP/UDS -> capsem-service
Frontend  -> capsem-gateway (TCP)-> HTTP/UDS -> capsem-service
Tray app  -> capsem-gateway (TCP)-> HTTP/UDS -> capsem-service
                                                     |
                                        capsem-process (per-VM, UDS IPC)
                                                     |
                                         +-----------+-----------+
                                         |           |           |
                                    vsock:5000  vsock:5001  vsock:5005
                                    (control)  (terminal)  (exec output)
                                         |           |           |
                                         +-----guest agent------+

Entry points for exec:

• capsem exec <id> "cmd" -> service HTTP /exec/{id} -> process IPC -> vsock
• capsem run "cmd" -> service HTTP /run -> provision + exec + destroy
• MCP capsem_exec / capsem_run -> service HTTP -> same path

Entry point for interactive shell:

• capsem shell [id] -> UDS IPC directly to capsem-process -> StartTerminalStream -> vsock:5001

IPC protocols

Service HTTP API

MCP tools (capsem-mcp)

21 tools: capsem_create (env + image params), capsem_list, capsem_info, capsem_exec (timeout param), capsem_run, capsem_stop, capsem_resume, capsem_persist, capsem_purge, capsem_read_file, capsem_write_file, capsem_vm_logs (grep + tail), capsem_service_logs (grep + tail), capsem_inspect_schema, capsem_inspect, capsem_delete, capsem_version, capsem_fork, capsem_image_list, capsem_image_inspect, capsem_image_delete.

Host-guest communication

All host-guest communication flows through capsem-process via vsock. There is no direct vsock access from any other host binary.

Interactive shell:  capsem-process -> vsock:5001 <-> Guest PTY (bash)
Exec command:       capsem-process -> vsock:5000 (Exec cmd) -> Guest agent
                    capsem-process <- vsock:5005 (stdout)    <- Guest child process
                    capsem-process <- vsock:5000 (ExecDone)  <- Guest agent
File I/O:           capsem-process -> vsock:5000 (FileWrite/FileRead) <-> Guest agent

Terminal I/O flows through vsock port 5001 (raw PTY bytes). Exec output flows on a dedicated port 5005 connection -- completely separated from the interactive terminal. File I/O uses port 5000 (control channel).

Serial console stays active for kernel boot logs. Terminal I/O switches to vsock once the guest agent sends Ready.

Vsock ports

Guest agent architecture

All guest binaries live in crates/capsem-agent/ and are cross-compiled for aarch64-unknown-linux-musl (and x86_64-unknown-linux-musl). Deployed chmod 555 (read-only) into the initrd at /run/.

capsem-pty-agent (main agent)

Single-threaded, sync Rust binary (no tokio). Launched by capsem-init after filesystems are mounted.

Boot sequence:

1. Connect to host on vsock:5001 (terminal) and vsock:5000 (control)
2. Send GuestToHost::Ready with agent version
3. Boot handshake: receive BootConfig (clock sync), then SetEnv/FileWrite messages, then BootConfigDone
4. Apply env vars, write files, set hostname from CAPSEM_VM_NAME
5. Open PTY pair, fork bash on the slave side
6. Send GuestToHost::BootReady + BootTiming (parsed from capsem-init's JSONL)
7. Enter bridge loop

Runtime -- two loops running concurrently:

• bridge_loop (main thread): polls master PTY, forwards output to vsock:5001. Spawns a dedicated thread for the reverse direction (vsock -> PTY). Pure bidirectional byte bridge with no scanning or filtering.
• control_loop (background thread): reads vsock:5000, handles Resize (set winsize + SIGWINCH), Ping/Pong heartbeat, Exec (spawns background thread for direct child process), FileWrite/FileRead/FileDelete, and Shutdown.

Exec mechanism: spawns bash -c '<cmd> 2>&1' as a direct child process (not via PTY). Connects to host on vsock:5005, sends ExecStarted { id } handshake, then streams child stdout to the exec port. Exit code comes from waitpid, sent as ExecDone { id, exit_code } on vsock:5000. Runs in a background thread so control_loop stays responsive to heartbeats during long commands.

Shutdown handler: sync() -> SIGTERM bash -> wait SHUTDOWN_GRACE_SECS (defined in capsem-proto) -> SIGKILL (interactive bash ignores SIGTERM) -> break. The bridge loop cleanup then sends SIGHUP + waitpid to reap the child.

capsem-sysutil (lifecycle multi-call binary)

Busybox-pattern binary dispatching on argv[0]. Symlinked by capsem-init:

• /sbin/shutdown, /sbin/halt, /sbin/poweroff, /sbin/reboot -> /run/capsem-sysutil
• /usr/local/bin/suspend -> /run/capsem-sysutil

Opens its own vsock:5004 connection (independent of capsem-pty-agent) and sends GuestToHost::ShutdownRequest or SuspendRequest. Shows a countdown (SHUTDOWN_GRACE_SECS + 1 seconds) before sending. Rejects reboot requests with an error.

Shutdown flow (end-to-end):

Guest: shutdown -> capsem-sysutil -> vsock:5004 -> capsem-process
  capsem-process: reads ShutdownRequest -> sends ProcessToService::ShutdownRequested to service
  capsem-process: sends HostToGuest::Shutdown on control channel (vsock:5000)
  capsem-pty-agent: receives Shutdown -> sync + SIGTERM + grace + SIGKILL -> exit
  capsem-process: VM stops, process exits
  capsem-service: child reaper cleans up (ephemeral: destroy session, persistent: preserve)

capsem-net-proxy

Listens on localhost:10443 inside the guest. iptables redirects all port 443 traffic here. Each connection is bridged to host vsock:5002 where the MITM proxy handles TLS termination and policy.

capsem-mcp-server

In-guest MCP gateway. Listens for MCP tool calls and routes them to external MCP servers via vsock:5003 NDJSON passthrough.

Storage modes

Selected by kernel cmdline capsem.storage=virtiofs (default) or absence (block mode).

VirtioFS mode (default):

~/.capsem/sessions/{id}/
  system/rootfs.img    # ext4 loopback (2GB sparse) -- overlayfs upper
  workspace/           # VirtioFS files for /root (host-visible)
  auto_snapshots/      # Rolling ring buffer (12 APFS clones, 5min interval)

Boot sequence: squashfs -> VirtioFS mount -> loopback ext4 -> overlayfs -> bind-mount workspace.

Why ext4 loopback: Apple VZ's VirtioFS doesn't support mknod (whiteout creation), so overlayfs can't use VirtioFS directly as upper.

Block mode (legacy): tmpfs overlay + scratch disk. No host file visibility, no snapshots.

Fork images (user-created templates):

~/.capsem/images/
  image_registry.json       # Image metadata index (JSON)
  {name}/
    system/                  # APFS clone of source VM's rootfs overlay
    workspace/               # APFS clone of workspace files
    session.db               # Telemetry from source VM (checkpointed)

Network architecture

The guest is air-gapped. No real NIC, no real DNS, no direct internet access.

1. capsem-init creates a dummy0 NIC with fake DNS (dnsmasq)
2. iptables redirects all port 443 traffic to capsem-net-proxy on localhost:10443
3. capsem-net-proxy bridges each TCP connection to host vsock port 5002
4. Host MITM proxy terminates TLS using per-domain minted certs (signed by static Capsem CA)
5. Host inspects HTTP request, applies domain + HTTP policy, forwards to real upstream
6. Full telemetry recorded to session DB (domain, method, path, status, headers, body preview)

Network policy

• User config: ~/.capsem/user.toml -- domain allow/block lists + HTTP rules
• Corp config: /etc/capsem/corp.toml -- enterprise lockdown (MDM-distributed)
• Merge: corp overrides user entirely per field; unspecified fields fall through
• HTTP rules: [[network.rules]] with method+path matching per domain

MITM CA

• Static CA: config/capsem-ca.key + config/capsem-ca.crt (ECDSA P-256)
• Baked into rootfs via update-ca-certificates + certifi patch
• Guest trusts it via system store + env vars (REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS, SSL_CERT_FILE)

Ephemeral VM model (invariants)

VirtioFS mode: fresh workspace + sparse rootfs.img per session. Host creates empty dirs, guest formats on first boot.

Block mode: mke2fs runs unconditionally at boot. Overlay upper is always tmpfs.

Everything is ephemeral unless asked otherwise. VMs are temporary by default. Named VMs (capsem create -n <name>) are persistent -- their workspace and rootfs overlay survive stops and can be resumed. Persistent VM data lives in ~/.capsem/run/persistent/. Never make the overlay upper layer persistent for ephemeral VMs. To add packages: edit guest config and just build-assets.

Fork images extend the ephemeral model with reusable templates. capsem fork <vm> <image-name> snapshots a VM (running or stopped) via APFS clonefile. capsem create --image <name> boots from the template. Images have flat genealogy: each depends only on a base squashfs version, never on other images. Deleting any image is always safe; asset cleanup protects referenced squashfs versions.

Installation and service lifecycle

capsem setup is the primary install entry point. On first CLI use, auto-runs non-interactively if ~/.capsem/setup-state.json is missing.

Setup wizard (6 steps): corp config provisioning, background asset download, security preset, AI provider detection, repository access, service installation.

Install layout (~/.capsem/):

• bin/ -- capsem, capsem-service, capsem-process, capsem-mcp, capsem-gateway, capsem-tray
• assets/ -- manifest.json, v{VERSION}/{vmlinuz, initrd.img, rootfs.squashfs}
• run/ -- service.sock, service.pid, gateway.token, gateway.port, gateway.pid, instances/{id}.sock

Service registration: LaunchAgent com.capsem.service (macOS) or systemd user unit capsem.service (Linux). KeepAlive/Restart=always. Service auto-launches gateway and tray as companion processes, passing --parent-pid so companions self-exit when the service dies (see capsem-guard, /dev-rust-patterns lesson 18).

Auto-launch cascade: capsem-service starts -> spawns capsem-gateway (port 19222) + capsem-tray. All three are separate processes.

Self-update: capsem update checks GitHub for new manifest, downloads assets in background. Binary swap deferred. Background update-check cache (update-check.json, 24h TTL) refreshes on every CLI command.

Key source files: crates/capsem/src/setup.rs, paths.rs, service_install.rs, update.rs, uninstall.rs.

Key source files

Read references/key-files.md for the full annotated source map.

Tauri v2 reference

Read references/tauri-v2.md for Tauri v2 patterns. capsem-app is a thin webview shell -- only 2 IPC commands (open_url, check_for_app_update). All VM operations route through the gateway.

Crate architecture

• capsem-core: all shared logic (VM, network, policy, telemetry, config). This is where business logic lives.
• capsem-service: daemon process. Axum HTTP server over UDS, spawns/manages capsem-process children, routes API calls via IPC.
• capsem-process: per-VM process. Boots VM via capsem-core, bridges vsock, manages structured jobs (exec, file I/O) with a job store + oneshot channels.
• capsem: CLI client. HTTP over UDS to service, direct UDS to process for shell.
• capsem-mcp: MCP server (stdio). Uses rmcp crate. Bridges AI agent tool calls to service HTTP API.
• capsem-gateway: TCP-to-UDS HTTP reverse proxy. Axum server on port 19222, Bearer token auth, CORS. Provides /status (cached), /terminal/{id} (WebSocket relay), and transparent fallback to service. Frontend and tray connect through this.
• capsem-app: thin Tauri webview shell. Points at gateway (http://127.0.0.1:19222). No capsem-core dependency. 2 IPC commands: open_url, check_for_app_update.
• capsem-agent: guest binaries crate. Contains four binaries cross-compiled for aarch64/x86_64-linux-musl: capsem-pty-agent (PTY bridge + control + exec + file I/O + shutdown), capsem-sysutil (lifecycle multi-call: shutdown/halt/poweroff/reboot/suspend), capsem-net-proxy (HTTPS -> MITM), capsem-mcp-server (MCP gateway).
• capsem-logger: session DB schema, queries, async writer.
• capsem-proto: shared protocol types. ipc.rs (ServiceToProcess/ProcessToService), lib.rs (HostToGuest/GuestToHost).

Process privilege model

capsem-process is a low-privilege per-VM process. Security invariants:

1. Minimal environment: service uses env_clear() before spawn, then passes only HOME, PATH, USER, TMPDIR, RUST_LOG. API keys and tokens from the user's shell never reach the process.
2. Socket permissions 0600: IPC ({id}.sock) and terminal WS ({id}-ws.sock) sockets are chmod 0600 after bind. Only the owning user can connect.
3. Session directory 0700: created by the service via create_virtiofs_session. Contains workspace/, system/, serial.log (0600), session.db.
4. No guest-triggered process exit: control channel read errors cause break (loop exit), not process::exit(). Guest cannot DoS the host process.
5. Gateway auth layer: external access goes through capsem-gateway (Bearer token, rate limiting, localhost CORS). Per-VM sockets are not exposed to the network.
6. Rootfs read-only: squashfs mounted read-only by Apple VZ. Guest binaries deployed chmod 555.
7. Guest binary security: all injected binaries are read-only. Guest cannot modify its own agent.
8. VirtioFS boundary: only session_dir/guest/ is shared via VirtioFS (contains system/ and workspace/). Host-only files (session.db, serial.log, auto_snapshots/, checkpoint.vzsave) are outside the share. Compat symlinks at session_dir/{system,workspace} point into guest/ so existing code paths work unchanged.

What capsem-process CAN access

• Its own session_dir (read-write)
• Assets dir (read-only: kernel, initrd, rootfs)
• Its own UDS sockets
• Apple VZ framework (requires com.apple.security.virtualization entitlement)

What capsem-process CANNOT access

• Other VMs' session dirs (0700, different path)
• Other VMs' UDS sockets (0600)
• The service's UDS socket (filesystem permission only)
• The persistent registry or other service state
• The user's environment variables (cleared at spawn)

MITM CA key transparency

The MITM proxy CA private key (config/capsem-ca.key) is committed to the repo and embedded at compile time. This is intentional -- capsem's network interception exists for user visibility into what AI agents do, not for secrecy. The CA is only trusted inside capsem's own air-gapped VMs and has zero trust outside them. A public key lets anyone verify there is no hidden interception. Per-installation key generation would reduce transparency.