sascha/homelab-butler
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Butler 2.3: deterministische Control Plane für leichte Modelle #1

Merged
sascha merged 6 commits from trulla/butler-2.3-20260722 into main 2026-07-22 10:38:13 +02:00
Conversation 0 Commits 6 Files changed 6 +96 -79
1 changed files with 96 additions and 79 deletions
Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit c9a900390b - Show all commits

175
README.md
View file

@ -1,110 +1,127 @@
# Homelab Butler 🤵 # Homelab Butler 🤵
Unified API proxy + infrastructure management for Homelab Pfannkuchen. Unified API proxy and infrastructure management for Homelab Pfannkuchen.
**Base URL:** `http://10.4.1.116:8888` - **Base URL:** `http://10.4.1.116:8888`
**Auth:** `Authorization: Bearer ***` - **Authentication:** `Authorization: Bearer <BUTLER_TOKEN>`
**Version:** 2.1.0 - **Version:** 2.3.0
- **Interactive API documentation:** `/docs`
## Service Proxy ## Service proxy
Proxies requests to backend services with automatic authentication: Requests are proxied as `/{service}/{backend-path}`. Butler adds each backend's authentication automatically and forwards query parameters.
| Service | Backend | Auth | | Service | Backend | Authentication |
|---------|---------|------| |---|---|---|
| `dockhand` | 10.4.1.116:3000 | Session | | `dockhand` | `10.4.1.116:3000` | Session |
| `sonarr` | 10.2.1.100:8989 | API Key | | `sonarr` / `sonarr1080p` | `10.2.1.100:8989/8990` | API key |
| `radarr` | 10.2.1.100:7878 | API Key | | `radarr` / `radarr1080p` | `10.2.1.100:7878/7879` | API key |
| `seerr` | 10.2.1.100:5055 | API Key | | `seerr` | `10.2.1.100:5055` | API key |
| `outline` | 10.1.1.100:3000 | Bearer | | `outline` | `10.1.1.100:3000` | Bearer |
| `n8n` | 10.4.1.113:5678 | X-N8N-API-KEY | | `n8n` | `10.4.1.113:5678` | n8n API key |
| `proxmox` | 10.5.85.11:8006 | PVE Token | | `proxmox` | `10.5.85.11:8006` | PVE API token |
| `homeassistant` | 10.10.1.20:8123 | Bearer | | `homeassistant` | `10.10.1.1:8123` | Bearer |
| `grafana` | 10.1.1.111:3000 | Bearer | | `grafana` | `10.1.1.111:3000` | Bearer |
| `uptime` | 159.69.245.190:3001 | Bearer | | `uptime` | `10.5.85.5:3001` | Web UI only; no supported REST API |
| `waha` | 10.4.1.110:3500 | API Key | | `waha` | `10.4.1.110:3500` | API key |
| `forgejo` | 10.4.1.116:3001 | Bearer | | `forgejo` | `10.4.1.116:3001` | Bearer |
| `semaphore` | 10.4.1.116:8090 | Bearer | | `semaphore` | `10.4.1.116:8090` | Bearer |
| `fileflows` | `10.2.1.104:8268` | Local API, no additional auth |
Usage: `GET/POST/PUT/DELETE /{service}/{path}` Known secret response fields such as Dockhand's `hawserToken` and `webhookSecret` are redacted before data leaves Butler.
## VM Lifecycle ## Operations
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---:|---|
| `/vm/list` | GET | All VMs across all 7 Proxmox nodes | | `/info` | GET | Secret-free machine-readable context |
| `/vm/create` | POST | Full VM deployment (~10 min): ISO build, VM create, SSH wait, inventory, Ansible | | `/status` | GET | Concurrent authenticated functional probes with deterministic states |
| `/vm/status/{vmid}` | GET | VM status (CPU, RAM, uptime) | | `/overview` | GET | Compact overall verdict and ordered findings for lightweight models; `details=true` adds raw data |
| `/vm/{vmid}` | DELETE | Destroy VM | | `/audit` | GET | Recent proxied/management calls |
| `/health/all` | GET | SSH reachability and Docker status for inventory hosts |
| `/backup/status` | GET | Concurrent Borgmatic checks with age, state and summary |
| `/disk/usage` | GET | Root filesystem usage per host |
| `/logs/{host}/{container}` | GET | Docker logs, `tail` query supported |
| `/docker/inspect/{host}/{container}` | GET | Sanitized image, runtime, resources, mounts and state |
| `/docker/restart/{host}/{container}` | POST | Restart container; supports `dry_run=true` |
| `/config/reload` | POST | Reload YAML configuration and credential cache |
### POST /vm/create ## VM lifecycle and inventory
```json
{"node": 5, "ip": "10.5.1.115", "hostname": "lychee", "cores": 2, "memory": 4096, "disk": 32}
```
Steps: iso-builder → Proxmox VM → wait SSH → add to pfannkuchen.ini → Ansible base setup (Docker, Borgmatic, Hawser)
## Ansible / Inventory
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---:|---|
| `/inventory/host` | POST | Add host to pfannkuchen.ini (idempotent, with group) | | `/vm/list` | GET | All VMs across the seven Proxmox nodes |
| `/ansible/run` | POST | Run Ansible playbook on host | | `/vm/create` | POST | VM deployment; supports `dry_run=true` |
| `/vm/status/{vmid}` | GET | CPU, RAM, uptime and state |
| `/vm/destroy/{vmid}` | DELETE | Full lifecycle cleanup; supports `dry_run=true` |
| `/inventory/host` | POST | Create or update inventory host and host vars |
| `/ansible/run` | POST | Run the standard setup for a host |
Example inventory upsert:
### POST /ansible/run
```json ```json
{"hostname": "lychee"} {"name":"example","ip":"10.5.1.115","group":"auto","user":"sascha"}
``` ```
**Post-Run Automation** (after successful Ansible): Node-7 VMs default to SSH user `chris`; Proxmox nodes default to `root`.
1. **Hawser Token Sync** – Reads `/etc/hawser/config` from VM, syncs token to Dockhand environment
2. **SOPS + .env Setup** – If `compose.yaml` exists in `/app-config/github/{hostname}/`:
- Generates secure secrets (admin password, DB password, secret key)
- Creates `.env` file with service-specific variables
- Encrypts to `.env.enc` using SOPS (Age key from automation1)
- Copies both files to VM's git repo directory
- Stores secrets in Butler vault cache for future reference
### POST /inventory/host ## TTS
```json
{"name": "lychee", "ip": "10.5.1.115", "group": "auto"}
```
## TTS / Speech
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---:|---|
| `/tts/speak` | POST | Text-to-speech via Chatterbox | | `/tts/speak` | POST | Chatterbox/speaker TTS |
| `/tts/voices` | GET | Available voices | | `/tts/voices` | GET | Available Chatterbox voices |
| `/tts/health` | GET | Speaker + Chatterbox status | | `/tts/health` | GET | Speaker and Chatterbox status |
### POST /tts/speak The speaker endpoint is `10.5.85.2:10800`; Chatterbox runs at `10.2.1.104:8004`.
```json
{"text": "Hallo!", "target": "speaker"}
```
- `"target": "speaker"` → plays on Pi5 speaker (10.10.1.166)
- `"target": "telegram"` → generates OGG on hermes, use `MEDIA:/tmp/trulla_voice.ogg`
## Credentials ## Deployment
Reads from Vaultwarden cache (synced by host cron) with flat-file fallback (`/data/api/`). Required mounts and settings are defined in `compose.yaml`:
## Stack - `.env` containing `BUTLER_TOKEN`
- `/app-config/kiro/api/` as flat-file credential fallback
- persistent Vaultwarden cache volume
- SSH key mounted read-only at `/root/.ssh`
- `butler.yaml` mounted read-only at `/data/butler.yaml`
``` Git is the source of truth. Build/recreate the Compose service only after committing and pushing changes.
docker compose build && docker compose up -d
## Tests
```bash
python -m pytest -q tests/test_app.py
``` ```
Requires: Integration Compose definition: `tests/compose.integration.yaml` (binds only to `127.0.0.1:8889`).
- `.env` with `BUTLER_TOKEN`
- `/app-config/kiro/api/` flat-file credentials
- SSH key mount (`/home/sascha/.ssh:/root/.ssh:ro`) for VM operations
## Changelog ## Changelog
### v2.1.0 (22.04.2026) ### 2.3.0 — 22.07.2026
- ✅ **Hawser Token Auto-Sync** – After `/ansible/run`, reads token from VM and updates Dockhand environment
- ✅ **SOPS + .env Automation** – Auto-generates and encrypts environment files for Git-centric deployments
- ✅ **Service Detection** – Recognizes Paperless-ngx and other services from hostname, generates appropriate env vars
### v2.0.0 - Added `/overview`, a compact schema-versioned verdict for lightweight language models.
- Initial unified API proxy release - Added explicit `healthy`, `degraded`, `auth_failed`, `misconfigured` and `offline` service states.
- Service probes now use the configured backend credentials and run concurrently.
- Backup checks now run with bounded concurrency and a 12-second per-host timeout.
- Backup results include age and severity (`healthy` up to 30 h, `warning` up to 48 h, then `critical`).
- Host and disk collection now run concurrently.
- Added regression tests for classification, authentication, backup age/concurrency and overview output.
### 2.2.0 — 17.07.2026
- Restored and modernized `/info`, `/health/all`, `/backup/status`, `/disk/usage` and Docker log/restart endpoints.
- Fixed stale Home Assistant, Uptime Kuma and speaker addresses.
- Fixed Forgejo credential fallback by deploying the YAML-driven service config.
- Correct SSH defaults for Proxmox nodes and Node-7 VMs.
- Added recursive secret redaction for proxied JSON.
- Forward query parameters through the generic proxy.
- Added validated inventory upserts.
- Added regression tests and a loopback-only integration Compose setup.
### 2.1.1
- Full VM destruction lifecycle cleanup.
### 2.1.0
- YAML service configuration, status/audit endpoints, dry-run support, Hawser token sync and SOPS automation.