initial pfannkuchen
This commit is contained in:
sascha
parent
b6dafc7a73
commit
4d305fa19f
99 changed files with 3575 additions and 321 deletions
371
README.md
Normal file
371
README.md
Normal file
|
|
@ -0,0 +1,371 @@
|
|||
# Ansible Playbooks
|
||||
|
||||
Ansible-Setup für Proxmox-VMs, Hetzner-Server und Borg Backup auf Hetzner Storagebox.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# Wrapper-Script nutzen
|
||||
./pfannkuchen.sh setup emby_sascha # VM komplett einrichten
|
||||
./pfannkuchen.sh backup # Borg auf allen Backup-Hosts
|
||||
./pfannkuchen.sh update # Alle Hosts updaten
|
||||
./pfannkuchen.sh gpu tdarr # NVIDIA Setup
|
||||
./pfannkuchen.sh pve # Proxmox Post-Install
|
||||
./pfannkuchen.sh passthrough # GPU Passthrough vorbereiten
|
||||
./pfannkuchen.sh telegraf # Telegraf Monitoring
|
||||
./pfannkuchen.sh hysteria2 node1 # Hysteria2 + WireGuard
|
||||
./pfannkuchen.sh ping # Alle Hosts anpingen
|
||||
./pfannkuchen.sh list # Inventory anzeigen
|
||||
|
||||
# Oder direkt mit Ansible
|
||||
ansible-playbook site.yml -l emby_sascha
|
||||
ansible-playbook borg-backup.yml -l proxmox
|
||||
ansible-playbook update.yml
|
||||
```
|
||||
|
||||
## Infrastruktur
|
||||
|
||||
### Proxmox Hosts
|
||||
| Host | IP | GPU |
|
||||
|-------|--------------|------|
|
||||
| node1 | 10.5.85.11 | – |
|
||||
| node2 | 10.5.85.12 | – |
|
||||
| node3 | 10.5.85.13 | – |
|
||||
| node4 | 10.5.85.14 | – |
|
||||
| node5 | 10.5.85.15 | A400 |
|
||||
| node6 | 10.5.85.16 | A400 |
|
||||
| node7 | 10.5.85.17 | A400 |
|
||||
|
||||
### VMs / LXCs
|
||||
| Name | IP | Gruppe | Zweck |
|
||||
|-------------|--------------|---------------|------------------------|
|
||||
| emby-sascha | 10.6.1.103 | media | Emby (Sascha) |
|
||||
| jellyfin | 10.5.1.112 | media | Jellyfin |
|
||||
| immich | 10.4.1.107 | media | Immich Fotoverwaltung |
|
||||
| emby-chris | 10.7.1.106 | media | Emby (Chris) + SMB |
|
||||
| tdarr | 10.2.1.104 | arr | Tdarr Transcoding |
|
||||
| arrapps | 10.2.1.100 | arr | Sonarr/Radarr etc. |
|
||||
| sabnzbd | 10.2.1.119 | arr | SABnzbd |
|
||||
| dockhand | 10.4.1.116 | docker | Dockhand |
|
||||
| n8n | 10.4.1.113 | auto | n8n Automation |
|
||||
| openclaw | 10.4.1.100 | auto | OpenClaw |
|
||||
| monitoring | 10.1.1.111 | auto | Monitoring Stack |
|
||||
| automation | 10.1.1.115 | auto | Automation Stack |
|
||||
| matrix | 10.4.1.110 | communication | Matrix |
|
||||
|
||||
### Hetzner
|
||||
| Name | IP | Zweck |
|
||||
|-------------|-----------------|--------------------------|
|
||||
| pfannkuchen | 159.69.245.190 | Caddy Reverse Proxy + VW |
|
||||
|
||||
### Hetzner Storagebox
|
||||
- Host: `u457772-sub3.your-storagebox.de`
|
||||
- User: `u457772-sub3`
|
||||
- Port: 23
|
||||
- SSH Key liegt in `roles/borg/files/` und wird als `/root/.ssh/id_borg` deployed
|
||||
|
||||
## Docker Stacks
|
||||
|
||||
Jeder Stack hat ein eigenes Git-Repo unter `~/pfannkuchen/github/` und ein eigenes Docker-Netzwerk.
|
||||
|
||||
| Stack | VM | Netzwerk | Services |
|
||||
|------------|--------------|----------------------|-----------------------------------------------------------------|
|
||||
| pfannkuchen| Hetzner VPS | `proxy_network` | Caddy, Vaultwarden, Homepage |
|
||||
| monitoring | 10.1.1.111 | `monitoring_network` | Teslamate, Postgres, Grafana, Mosquitto, Prometheus, SNMP-Exporter, InfluxDB, Emby-Exporter, Node-Exporter |
|
||||
| automation | 10.1.1.115 | `auto_network` | WAHA, Semaphore UI, Patchmon (DB, Redis, Backend, Frontend) |
|
||||
| n8n | 10.4.1.113 | `n8n_network` | n8n, Postgres |
|
||||
|
||||
### Caddy Reverse Proxy (Hetzner VPS)
|
||||
|
||||
Caddy läuft auf dem Hetzner VPS und proxied alle Services über WireGuard-IPs:
|
||||
|
||||
| Domain | Backend | Bemerkung |
|
||||
|-------------------------|----------------------------|--------------------|
|
||||
| tv.sascha-lutz.de | host.docker.internal:18096 | Emby Sascha |
|
||||
| guck.tv | host.docker.internal:28096 | Emby Chris |
|
||||
| netzflix.org | host.docker.internal:38096 | Emby Jellyfin |
|
||||
| vault.sascha-lutz.de | vaultwarden (Container) | Vaultwarden |
|
||||
| home.sascha-lutz.de | homepage:3000 (Container) | Homepage (Basic Auth) |
|
||||
| grafana.sascha-lutz.de | grafana:3000 (Container) | Grafana |
|
||||
| patchmon.sascha-lutz.de | patchmon-frontend:3000 | Patchmon |
|
||||
| tesla.sascha-lutz.de | teslamate:4000 (Container) | Teslamate (Basic Auth) |
|
||||
| influx.sascha-lutz.de | influxdb:8086 (Container) | InfluxDB |
|
||||
| status.guck.tv | 10.200.200.254:3001 | Uptime Kuma |
|
||||
| n8n.sascha-lutz.de | 10.4.1.113:5678 | n8n |
|
||||
| docker.sascha-lutz.de | 10.4.1.116:3000 | Dockhand |
|
||||
| immich.sascha-lutz.de | 10.4.1.107:2283 | Immich |
|
||||
| dl.guck.tv | 10.2.1.100:5055 | Jellyseerr |
|
||||
| plappern.com | 10.4.1.110:8008 | Matrix Synapse |
|
||||
| web.plappern.com | 10.4.1.110:8080 | Matrix Element |
|
||||
| chat.plappern.com | 10.4.1.110:8090 | Matrix Chat |
|
||||
| monitor.guck.tv | 10.5.1.102:19999 | Netdata (Basic Auth) |
|
||||
|
||||
### Seerr → WhatsApp + Auto-Approve
|
||||
|
||||
Kombinierter n8n Workflow der zwei Dinge macht:
|
||||
|
||||
**1. Auto-Approve (MEDIA_PENDING):**
|
||||
```
|
||||
Jellyseerr → Webhook → n8n → TMDB Genre+FSK Lookup → Kategorisieren → Pfad+Tags setzen → Approve → 20min warten → FHD Radarr/Sonarr Suche
|
||||
```
|
||||
|
||||
Genre-Logik:
|
||||
| Kategorie | Bedingung | Radarr Pfad | Tag |
|
||||
|---|---|---|---|
|
||||
| Horror | Genre "Horror" + FSK ≥ 16 | `/data/UHD/horror` | `horror-4k` |
|
||||
| Kids | Genre "Animation"/"Family" + FSK ≤ 12 | `/data/UHD/kids-video` | `kids-4k` |
|
||||
| Normal | Alles andere | `/data/UHD/video` | `video-4k` |
|
||||
|
||||
Serien:
|
||||
| Kategorie | Bedingung | Sonarr Pfad | Tag |
|
||||
|---|---|---|---|
|
||||
| Kids | Genre "Animation" + FSK ≤ 12 | `/data/UHD/kids-serien` | `kids-4k` |
|
||||
| Streaming | Network ist Netflix/HBO/Amazon/Disney+ etc. | `/data/UHD/serien` | `serien-4k` |
|
||||
| TV | Alles andere | `/data/UHD/tvshows` | `tvshows-4k` |
|
||||
|
||||
**2. WhatsApp Newsletter (MEDIA_AVAILABLE):**
|
||||
```
|
||||
Jellyseerr → Webhook → n8n → WAHA Session Restart → WAHA sendText → WhatsApp Newsletter
|
||||
```
|
||||
|
||||
- Webhook URL: `http://10.4.1.113:5678/webhook/seerr-notify`
|
||||
- WAHA: `http://10.1.1.115:3500` (Dashboard: admin / WAHA_API_KEY)
|
||||
- Newsletter Channel: `120363404705299449@newsletter`
|
||||
- n8n Workflow: `tmp/n8n-seerr-combined.json`
|
||||
- Status: **Funktioniert** (Text + Bild-URL, keine eingebetteten Bilder möglich)
|
||||
|
||||
### Semaphore UI
|
||||
|
||||
Web-UI für Ansible Playbook Management auf `http://10.1.1.115:3000`.
|
||||
|
||||
- Login: `admin` / `SEMAPHORE_ADMIN_PASSWORD` (aus `.env`)
|
||||
- Projekt: Pfannkuchen
|
||||
- Templates: Setup VM, Update All, Borg Backup, Base Setup, NVIDIA GPU
|
||||
- Git Repo: `https://github.com/feldjaeger/ansible.git`
|
||||
|
||||
### Arr-Stack APIs
|
||||
|
||||
| Service | URL | API Key Datei |
|
||||
|---|---|---|
|
||||
| Radarr UHD | `http://10.2.1.100:7878` | `tmp/radarr` |
|
||||
| Radarr FHD | `http://10.2.1.100:7879` | `tmp/radarr1080p` |
|
||||
| Sonarr UHD | `http://10.2.1.100:8989` | `tmp/sonarr` |
|
||||
| Sonarr FHD | `http://10.2.1.100:8990` | `tmp/sonarr1080p` |
|
||||
| Jellyseerr | `http://10.2.1.100:5055` | `tmp/seer` |
|
||||
| TMDB API | `https://api.themoviedb.org/3` | `tmp/tmdb` (nicht im Repo) |
|
||||
|
||||
## ISO Builder
|
||||
|
||||
Baut Custom Debian ISOs mit Preseed für unattended Installation. Siehe `iso-builder/README.md`.
|
||||
|
||||
```bash
|
||||
# ISO bauen + auf Proxmox Node uploaden
|
||||
./iso-builder/build-iso.sh --node 4 --ip 10.4.1.120 --hostname neue-vm
|
||||
|
||||
# ISO bauen + VM erstellen + starten
|
||||
./iso-builder/build-iso.sh --node 4 --ip 10.4.1.120 --hostname neue-vm --create-vm
|
||||
```
|
||||
|
||||
## Netzwerk
|
||||
|
||||
### WireGuard Tunnel (Hysteria2)
|
||||
|
||||
Alle Proxmox Nodes verbinden sich über Hysteria2 (QUIC) zum VPS. Das umgeht CGNAT-Beschränkungen und versteckt den WireGuard-Traffic vor DPI (Sophos Firewall).
|
||||
|
||||
```
|
||||
[Proxmox Node] → Hysteria2-Client → QUIC (UDP:8443) → Hysteria2-Server (VPS) → WireGuard
|
||||
```
|
||||
|
||||
- Hysteria2 nutzt UDP:8443, Caddy behält TCP:443 + UDP:443 (HTTP/3)
|
||||
- TLS: Selbstsigniertes Zertifikat auf dem VPS (10 Jahre gültig), Clients mit `insecure: true`
|
||||
- Bandwidth-Hints konfiguriert für optimale Performance (400 Mbit/s)
|
||||
|
||||
| Node | WG-IP | VM-Subnetz | Bemerkung |
|
||||
|-------|----------------|----------------|------------------|
|
||||
| VPS | 10.200.200.254 | – | WG Server + Hub |
|
||||
| node1 | 10.200.200.2 | 10.11.1.0/24 | |
|
||||
| node2 | 10.200.200.3 | 10.2.1.0/24 | Arr-Stack |
|
||||
| node3 | 10.200.200.113 | 10.3.1.0/24 | |
|
||||
| node4 | 10.200.200.100 | 10.4.1.0/24 | Docker/Auto |
|
||||
| node5 | 10.200.200.101 | 10.5.1.0/24 | Media |
|
||||
| node6 | 10.200.200.116 | 10.6.1.0/24 | A400 |
|
||||
| node7 | 10.200.200.117 | 10.7.1.0/24 | A400 |
|
||||
|
||||
Direkte Peers (kein Hysteria2):
|
||||
- embyproxy (10.200.200.1)
|
||||
- Sascha (10.200.200.5), Chris (10.200.200.6), Sascha Handy (10.200.200.7)
|
||||
- Fassohneboden (10.200.200.4), Marco Minecraft (10.200.200.8)
|
||||
|
||||
### Netzwerk-Segmente
|
||||
|
||||
| Subnetz | Zweck | Node |
|
||||
|--------------|----------------|-------|
|
||||
| 10.5.85.x | Proxmox Hosts | alle |
|
||||
| 10.1.1.x | Monitoring/Auto| node1 |
|
||||
| 10.11.1.x | VMs node1 | node1 |
|
||||
| 10.2.1.x | Arr-Stack | node2 |
|
||||
| 10.3.1.x | VMs node3 | node3 |
|
||||
| 10.4.1.x | Docker/Auto | node4 |
|
||||
| 10.5.1.x | Media VMs | node5 |
|
||||
| 10.6.1.x | VMs node6 | node6 |
|
||||
| 10.7.1.x | VMs node7 | node7 |
|
||||
| 10.200.200.x | WireGuard | VPS |
|
||||
|
||||
## Rollen
|
||||
|
||||
| Rolle | Zweck |
|
||||
|---------------------|--------------------------------------------------------------|
|
||||
| base | Repos, Pakete, Locale, SSH Key, Sudo, QEMU Guest Agent |
|
||||
| docker | Docker CE + Compose Plugin, User sascha → docker Gruppe |
|
||||
| nvidia | CUDA Repo, cuda-drivers, Container Toolkit, Docker nvidia-RT |
|
||||
| borg | SSH Key Deploy, Borg Repo Init, Backup-Script, Systemd Timer |
|
||||
| hawser | Hawser Install + Systemd Service + Token aus Vault |
|
||||
| sysctl | BBR, TCP Tuning, Buffer Sizes für Streaming-VMs |
|
||||
| sysctl_proxmox | Overcommit, File Handles, IP Forward, Bridge-Tuning |
|
||||
| pve_postinstall | Repos (deb822), Enterprise deaktiviert, Nag-Patch, HA aus |
|
||||
| pve_gpu_passthrough | IOMMU, VFIO, Nouveau/NVIDIA Blacklist für GPU Passthrough |
|
||||
| telegraf | InfluxData Repo, Telegraf Config, lm-sensors, Synology SNMP |
|
||||
| hysteria2 | Hysteria2 Client, Bandwidth-Hints, WireGuard Config |
|
||||
| hysteria2_server | Hysteria2 Server, selbstsigniertes TLS, systemd Service |
|
||||
|
||||
## Playbooks
|
||||
|
||||
| Playbook | Wrapper-Befehl | Zweck |
|
||||
|--------------------------|------------------------|------------------------------------------|
|
||||
| `site.yml` | `setup <host>` | VM komplett einrichten |
|
||||
| `base-debian.yml` | `base <host>` | Grundsetup + Docker |
|
||||
| `nvidia-docker.yml` | `gpu <host>` | NVIDIA Treiber + Docker GPU Runtime |
|
||||
| `borg-backup.yml` | `backup [host]` | Borg Backup einrichten |
|
||||
| `hawser.yml` | `hawser <host>` | Hawser installieren |
|
||||
| `update.yml` | `update [host]` | Dist-Upgrade + Autoremove |
|
||||
| `sysctl.yaml` | `tune <host>` | Netzwerk-Tuning für Streaming |
|
||||
| `sysctl-proxmox.yaml` | – | Proxmox-Host-Tuning |
|
||||
| `pve-postinstall.yml` | `pve [host]` | Proxmox Post-Install (Repos, Nag, HA) |
|
||||
| `pve-gpu-passthrough.yml`| `passthrough [host]` | GPU PCI Passthrough vorbereiten |
|
||||
| `telegraf.yml` | `telegraf [host]` | Telegraf Monitoring deployen |
|
||||
| `hysteria2.yml` | – | Hysteria2 Client + WireGuard deployen |
|
||||
| `hysteria2-server.yml` | – | Hysteria2 Server auf VPS deployen |
|
||||
|
||||
## Inventory-Gruppen
|
||||
|
||||
| Gruppe | Hosts | Zweck |
|
||||
|---------------|-------------------------------------------------|------------------------------|
|
||||
| proxmox | node1–node7 | Alle Proxmox Hosts |
|
||||
| proxmox_gpu | node2, node4, node6, node7 | Nodes mit A400 GPU |
|
||||
| nvidia | tdarr, emby-sascha, emby-chris, immich | VMs mit NVIDIA GPU Runtime |
|
||||
| wireguard | node1–node7 | WireGuard Clients |
|
||||
| media | emby-sascha, jellyfin, immich, emby-chris | Media VMs |
|
||||
| arr | tdarr, arrapps, sabnzbd | Arr-Stack |
|
||||
| docker | dockhand | Docker VMs |
|
||||
| auto | n8n, openclaw, monitoring, automation | Automation |
|
||||
| communication | matrix | Kommunikation |
|
||||
| hetzner | pfannkuchen | Hetzner Server |
|
||||
| frp | emby-sascha, emby-chris, jellyfin | FRP Clients |
|
||||
| backup | alle (via children) | Borg Backup |
|
||||
|
||||
## Borg Backup
|
||||
|
||||
Alle Hosts in der Gruppe `backup` bekommen Borg Backup. Die Backup-Quellen sind pro Gruppe konfiguriert:
|
||||
|
||||
| Gruppe | Backup-Quellen | Quelle |
|
||||
|-----------|-----------------------------------------------------------------------------|-------------------------------|
|
||||
| VMs | `/app-config` | `roles/borg/defaults/main.yml`|
|
||||
| Proxmox | `/etc/pve /etc/network /etc/wireguard /etc/crontab /etc/fstab /etc/systemd/system/ /etc/iptables /etc/telegraf` | `group_vars/proxmox/borg.yml` |
|
||||
| Hetzner | `/etc/wireguard /app-config` | `group_vars/hetzner/borg.yml` |
|
||||
|
||||
Backup-Script: `/usr/local/bin/borg-backup.sh`
|
||||
Systemd Timer: täglich 03:00 Uhr (±30min Jitter)
|
||||
Log: `/var/log/borg-backup.log`
|
||||
Kompression: lz4
|
||||
Retention: 7 daily, 4 weekly, 6 monthly
|
||||
|
||||
## Telegraf Monitoring
|
||||
|
||||
Alle Proxmox Hosts bekommen Telegraf mit InfluxDB v2 Output.
|
||||
|
||||
Standard-Inputs: cpu, disk, diskio, kernel, mem, processes, swap, system, nstat, sensors
|
||||
|
||||
Sonderfall node2: Zusätzlich Synology NAS SNMP Monitoring (konfiguriert via `host_vars/node2/telegraf.yml`).
|
||||
|
||||
## Vault
|
||||
|
||||
Vault-Datei: `group_vars/all/vault.yml`
|
||||
|
||||
Enthaltene Variablen:
|
||||
- `vault_hetzner_storage_host` – Storagebox Hostname
|
||||
- `vault_hetzner_storage_user` – Storagebox User
|
||||
- `vault_borg_passphrase` – Borg Verschlüsselungspasswort
|
||||
- `vault_sascha_password` – SSH/Become-Passwort für User sascha
|
||||
- `vault_chris_password` – SSH/Become-Passwort für User chris
|
||||
- `vault_telegraf_influx_token` – InfluxDB v2 Token für Telegraf
|
||||
- `vault_snmp_sec_name` – SNMP v3 Security Name (Synology)
|
||||
- `vault_snmp_auth_password` – SNMP v3 Auth Passwort
|
||||
- `vault_snmp_priv_password` – SNMP v3 Privacy Passwort
|
||||
- `vault_hysteria2_password` – Hysteria2 Auth Passwort
|
||||
- `vault_wireguard_vps_pubkey` – WireGuard Public Key des VPS
|
||||
- `vault_node[1-7]_wg_privkey` – WireGuard Private Keys der Nodes
|
||||
- `vault_hawser_token` – Hawser Agent Token für Dockhand
|
||||
|
||||
```bash
|
||||
# Passwörter eintragen und verschlüsseln
|
||||
ansible-vault edit group_vars/all/vault.yml
|
||||
ansible-vault encrypt group_vars/all/vault.yml
|
||||
```
|
||||
|
||||
Vault-Passwort liegt in `.vault-password` (in `.gitignore`).
|
||||
|
||||
## Verzeichnisstruktur
|
||||
|
||||
```
|
||||
ansible/
|
||||
├── ansible.cfg
|
||||
├── pfannkuchen.ini # Inventory
|
||||
├── pfannkuchen.sh # Wrapper-Script
|
||||
├── site.yml # Master-Playbook
|
||||
├── base-debian.yml
|
||||
├── borg-backup.yml
|
||||
├── nvidia-docker.yml
|
||||
├── hawser.yml
|
||||
├── update.yml
|
||||
├── sysctl.yaml
|
||||
├── sysctl-proxmox.yaml
|
||||
├── pve-postinstall.yml
|
||||
├── pve-gpu-passthrough.yml
|
||||
├── telegraf.yml
|
||||
├── hysteria2.yml
|
||||
├── hysteria2-server.yml
|
||||
├── iso-builder/ # Custom Debian ISO Builder
|
||||
│ ├── build-iso.sh # ISO Build + Upload + VM Create
|
||||
│ ├── preseed.cfg.tpl # Preseed Template
|
||||
│ └── output/ # Gebaute ISOs
|
||||
├── group_vars/
|
||||
│ ├── all/vault.yml # Vault (verschlüsselt)
|
||||
│ ├── proxmox/borg.yml # Proxmox Backup-Pfade
|
||||
│ └── hetzner/borg.yml # Hetzner Backup-Pfade
|
||||
├── host_vars/
|
||||
│ ├── node1/wireguard.yml
|
||||
│ ├── node2/
|
||||
│ │ ├── telegraf.yml # Synology SNMP
|
||||
│ │ └── wireguard.yml
|
||||
│ ├── node3–node7/wireguard.yml
|
||||
│ └── emby_chris/vars.yml # User chris Credentials
|
||||
├── tmp/
|
||||
│ ├── n8n-seerr-combined.json # n8n Workflow: Auto-Approve + WhatsApp
|
||||
│ ├── radarr, sonarr, seer # API Keys
|
||||
│ ├── radarr1080p, sonarr1080p # FHD API Keys
|
||||
│ └── n8n # n8n API Key
|
||||
└── roles/
|
||||
├── base/tasks/main.yml
|
||||
├── docker/tasks/main.yml
|
||||
├── nvidia/{tasks,handlers,defaults}/
|
||||
├── borg/{tasks,defaults,files,templates}/
|
||||
├── hawser/tasks/main.yml
|
||||
├── hawser/handlers/main.yml
|
||||
├── sysctl/tasks/main.yml
|
||||
├── sysctl_proxmox/tasks/main.yml
|
||||
├── pve_postinstall/tasks/main.yml
|
||||
├── pve_gpu_passthrough/{tasks,handlers}/
|
||||
├── telegraf/{tasks,handlers,defaults,templates}/
|
||||
├── hysteria2/{tasks,handlers,defaults}/
|
||||
└── hysteria2_server/{tasks,handlers,defaults}/
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue