diff --git a/README.md b/README.md index b16db57..95c8d18 100644 --- a/README.md +++ b/README.md @@ -1,110 +1,127 @@ # Homelab Butler 🤵 -Unified API proxy + infrastructure management for Homelab Pfannkuchen. +Unified API proxy and infrastructure management for Homelab Pfannkuchen. -**Base URL:** `http://10.4.1.116:8888` -**Auth:** `Authorization: Bearer ***` -**Version:** 2.1.0 +- **Base URL:** `http://10.4.1.116:8888` +- **Authentication:** `Authorization: Bearer ` +- **Version:** 2.3.0 +- **Interactive API documentation:** `/docs` -## Service Proxy +## Service proxy -Proxies requests to backend services with automatic authentication: +Requests are proxied as `/{service}/{backend-path}`. Butler adds each backend's authentication automatically and forwards query parameters. -| Service | Backend | Auth | -|---------|---------|------| -| `dockhand` | 10.4.1.116:3000 | Session | -| `sonarr` | 10.2.1.100:8989 | API Key | -| `radarr` | 10.2.1.100:7878 | API Key | -| `seerr` | 10.2.1.100:5055 | API Key | -| `outline` | 10.1.1.100:3000 | Bearer | -| `n8n` | 10.4.1.113:5678 | X-N8N-API-KEY | -| `proxmox` | 10.5.85.11:8006 | PVE Token | -| `homeassistant` | 10.10.1.20:8123 | Bearer | -| `grafana` | 10.1.1.111:3000 | Bearer | -| `uptime` | 159.69.245.190:3001 | Bearer | -| `waha` | 10.4.1.110:3500 | API Key | -| `forgejo` | 10.4.1.116:3001 | Bearer | -| `semaphore` | 10.4.1.116:8090 | Bearer | +| Service | Backend | Authentication | +|---|---|---| +| `dockhand` | `10.4.1.116:3000` | Session | +| `sonarr` / `sonarr1080p` | `10.2.1.100:8989/8990` | API key | +| `radarr` / `radarr1080p` | `10.2.1.100:7878/7879` | API key | +| `seerr` | `10.2.1.100:5055` | API key | +| `outline` | `10.1.1.100:3000` | Bearer | +| `n8n` | `10.4.1.113:5678` | n8n API key | +| `proxmox` | `10.5.85.11:8006` | PVE API token | +| `homeassistant` | `10.10.1.1:8123` | Bearer | +| `grafana` | `10.1.1.111:3000` | Bearer | +| `uptime` | `10.5.85.5:3001` | Web UI only; no supported REST API | +| `waha` | `10.4.1.110:3500` | API key | +| `forgejo` | `10.4.1.116:3001` | Bearer | +| `semaphore` | `10.4.1.116:8090` | Bearer | +| `fileflows` | `10.2.1.104:8268` | Local API, no additional auth | -Usage: `GET/POST/PUT/DELETE /{service}/{path}` +Known secret response fields such as Dockhand's `hawserToken` and `webhookSecret` are redacted before data leaves Butler. -## VM Lifecycle +## Operations | Endpoint | Method | Description | -|----------|--------|-------------| -| `/vm/list` | GET | All VMs across all 7 Proxmox nodes | -| `/vm/create` | POST | Full VM deployment (~10 min): ISO build, VM create, SSH wait, inventory, Ansible | -| `/vm/status/{vmid}` | GET | VM status (CPU, RAM, uptime) | -| `/vm/{vmid}` | DELETE | Destroy VM | +|---|---:|---| +| `/info` | GET | Secret-free machine-readable context | +| `/status` | GET | Concurrent authenticated functional probes with deterministic states | +| `/overview` | GET | Compact overall verdict and ordered findings for lightweight models; `details=true` adds raw data | +| `/audit` | GET | Recent proxied/management calls | +| `/health/all` | GET | SSH reachability and Docker status for inventory hosts | +| `/backup/status` | GET | Concurrent Borgmatic checks with age, state and summary | +| `/disk/usage` | GET | Root filesystem usage per host | +| `/logs/{host}/{container}` | GET | Docker logs, `tail` query supported | +| `/docker/inspect/{host}/{container}` | GET | Sanitized image, runtime, resources, mounts and state | +| `/docker/restart/{host}/{container}` | POST | Restart container; supports `dry_run=true` | +| `/config/reload` | POST | Reload YAML configuration and credential cache | -### POST /vm/create -```json -{"node": 5, "ip": "10.5.1.115", "hostname": "lychee", "cores": 2, "memory": 4096, "disk": 32} -``` -Steps: iso-builder → Proxmox VM → wait SSH → add to pfannkuchen.ini → Ansible base setup (Docker, Borgmatic, Hawser) - -## Ansible / Inventory +## VM lifecycle and inventory | Endpoint | Method | Description | -|----------|--------|-------------| -| `/inventory/host` | POST | Add host to pfannkuchen.ini (idempotent, with group) | -| `/ansible/run` | POST | Run Ansible playbook on host | +|---|---:|---| +| `/vm/list` | GET | All VMs across the seven Proxmox nodes | +| `/vm/create` | POST | VM deployment; supports `dry_run=true` | +| `/vm/status/{vmid}` | GET | CPU, RAM, uptime and state | +| `/vm/destroy/{vmid}` | DELETE | Full lifecycle cleanup; supports `dry_run=true` | +| `/inventory/host` | POST | Create or update inventory host and host vars | +| `/ansible/run` | POST | Run the standard setup for a host | + +Example inventory upsert: -### POST /ansible/run ```json -{"hostname": "lychee"} +{"name":"example","ip":"10.5.1.115","group":"auto","user":"sascha"} ``` -**Post-Run Automation** (after successful Ansible): -1. **Hawser Token Sync** – Reads `/etc/hawser/config` from VM, syncs token to Dockhand environment -2. **SOPS + .env Setup** – If `compose.yaml` exists in `/app-config/github/{hostname}/`: - - Generates secure secrets (admin password, DB password, secret key) - - Creates `.env` file with service-specific variables - - Encrypts to `.env.enc` using SOPS (Age key from automation1) - - Copies both files to VM's git repo directory - - Stores secrets in Butler vault cache for future reference +Node-7 VMs default to SSH user `chris`; Proxmox nodes default to `root`. -### POST /inventory/host -```json -{"name": "lychee", "ip": "10.5.1.115", "group": "auto"} -``` - -## TTS / Speech +## TTS | Endpoint | Method | Description | -|----------|--------|-------------| -| `/tts/speak` | POST | Text-to-speech via Chatterbox | -| `/tts/voices` | GET | Available voices | -| `/tts/health` | GET | Speaker + Chatterbox status | +|---|---:|---| +| `/tts/speak` | POST | Chatterbox/speaker TTS | +| `/tts/voices` | GET | Available Chatterbox voices | +| `/tts/health` | GET | Speaker and Chatterbox status | -### POST /tts/speak -```json -{"text": "Hallo!", "target": "speaker"} -``` -- `"target": "speaker"` → plays on Pi5 speaker (10.10.1.166) -- `"target": "telegram"` → generates OGG on hermes, use `MEDIA:/tmp/trulla_voice.ogg` +The speaker endpoint is `10.5.85.2:10800`; Chatterbox runs at `10.2.1.104:8004`. -## Credentials +## Deployment -Reads from Vaultwarden cache (synced by host cron) with flat-file fallback (`/data/api/`). +Required mounts and settings are defined in `compose.yaml`: -## Stack +- `.env` containing `BUTLER_TOKEN` +- `/app-config/kiro/api/` as flat-file credential fallback +- persistent Vaultwarden cache volume +- SSH key mounted read-only at `/root/.ssh` +- `butler.yaml` mounted read-only at `/data/butler.yaml` -``` -docker compose build && docker compose up -d +Git is the source of truth. Build/recreate the Compose service only after committing and pushing changes. + +## Tests + +```bash +python -m pytest -q tests/test_app.py ``` -Requires: -- `.env` with `BUTLER_TOKEN` -- `/app-config/kiro/api/` flat-file credentials -- SSH key mount (`/home/sascha/.ssh:/root/.ssh:ro`) for VM operations +Integration Compose definition: `tests/compose.integration.yaml` (binds only to `127.0.0.1:8889`). ## Changelog -### v2.1.0 (22.04.2026) -- ✅ **Hawser Token Auto-Sync** – After `/ansible/run`, reads token from VM and updates Dockhand environment -- ✅ **SOPS + .env Automation** – Auto-generates and encrypts environment files for Git-centric deployments -- ✅ **Service Detection** – Recognizes Paperless-ngx and other services from hostname, generates appropriate env vars +### 2.3.0 — 22.07.2026 -### v2.0.0 -- Initial unified API proxy release +- Added `/overview`, a compact schema-versioned verdict for lightweight language models. +- Added explicit `healthy`, `degraded`, `auth_failed`, `misconfigured` and `offline` service states. +- Service probes now use the configured backend credentials and run concurrently. +- Backup checks now run with bounded concurrency and a 12-second per-host timeout. +- Backup results include age and severity (`healthy` up to 30 h, `warning` up to 48 h, then `critical`). +- Host and disk collection now run concurrently. +- Added regression tests for classification, authentication, backup age/concurrency and overview output. + +### 2.2.0 — 17.07.2026 + +- Restored and modernized `/info`, `/health/all`, `/backup/status`, `/disk/usage` and Docker log/restart endpoints. +- Fixed stale Home Assistant, Uptime Kuma and speaker addresses. +- Fixed Forgejo credential fallback by deploying the YAML-driven service config. +- Correct SSH defaults for Proxmox nodes and Node-7 VMs. +- Added recursive secret redaction for proxied JSON. +- Forward query parameters through the generic proxy. +- Added validated inventory upserts. +- Added regression tests and a loopback-only integration Compose setup. + +### 2.1.1 + +- Full VM destruction lifecycle cleanup. + +### 2.1.0 + +- YAML service configuration, status/audit endpoints, dry-run support, Hawser token sync and SOPS automation.