docker: containerizzazione tts-server con GPU Vulkan (Dockerfile multi-stage + compose + entrypoint voci .spk/.rvq)

2026-08-13 20:06:55 +02:00
parent 3be42af9bb
commit 8b9361803e
6 changed files with 294 additions and 0 deletions
@@ -5,3 +5,4 @@ models
 docs
 examples
 *.md
+voices
@@ -0,0 +1,52 @@
+# syntax=docker/dockerfile:1
+# ============================================================================
+# tts-server containerizzato (Qwen3-TTS, backend Vulkan)
+# ----------------------------------------------------------------------------
+# GPU supportate:
+#   - AMD/Intel: Mesa RADV via /dev/dri/renderD128 (nessun driver nel container)
+#   - NVIDIA:    ICD Vulkan (nvidia-container-toolkit) o /dev/dri
+# Il driver kernel resta sull'host; il container monta solo il device node.
+#
+# Build:  docker build -t qwentts-tts:vulkan .
+# Run:    docker compose up -d   (vedi docker-compose.yml)
+# ============================================================================
+
+FROM ubuntu:22.04 AS build-vulkan
+
+RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
+        git ca-certificates cmake g++ make wget gnupg \
+    > /dev/null && rm -rf /var/lib/apt/lists/*
+
+# glslc (shader compiler) è impacchettato solo dal repo LunarG su Ubuntu 22.04
+RUN wget -qO- https://packages.lunarg.com/lunarg-signing-key-pub.asc | gpg --dearmor -o /usr/share/keyrings/lunarg.gpg && \
+    echo "deb [signed-by=/usr/share/keyrings/lunarg.gpg] https://packages.lunarg.com/vulkan/1.3.296 jammy main" \
+        > /etc/apt/sources.list.d/lunarg-vulkan.list && \
+    apt-get update -qq && apt-get install -y -qq --no-install-recommends vulkan-sdk \
+    > /dev/null && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /build
+COPY . .
+
+# GGML_NATIVE=OFF: binario portabile tra CPU diverse (niente tuning nativo)
+RUN cmake -B build -DGGML_VULKAN=ON -DGGML_NATIVE=OFF -DCMAKE_BUILD_TYPE=Release && \
+    cmake --build build --config Release -j"$(nproc)"
+
+FROM ubuntu:22.04 AS runtime
+
+RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
+        libgomp1 libvulkan1 mesa-vulkan-drivers curl ca-certificates jq python3 \
+    > /dev/null && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+COPY --from=build-vulkan /build/build/tts-server /build/build/qwen-tts /build/build/qwen-codec /build/build/*.so* ./
+COPY docker/tts-server/entrypoint.sh ./entrypoint.sh
+COPY docker/tts-server/register_voices.py ./register_voices.py
+RUN chmod +x ./entrypoint.sh
+
+# Le librerie ggml copiate fuori dall'albero di build hanno RPATH interno:
+# serve il path esplicito.
+ENV LD_LIBRARY_PATH=/app
+ENV GGML_BACKEND=Vulkan0
+
+EXPOSE 8881
+ENTRYPOINT ["./entrypoint.sh"]
@@ -0,0 +1,79 @@
+# tts-server containerizzato (Qwen3-TTS, backend Vulkan)
+
+Containerizza il server TTS (qwentts.cpp) con accelerazione GPU **Vulkan**,
+portabile su qualsiasi host con GPU AMD/Intel (Mesa RADV) o NVIDIA (ICD Vulkan).
+
+## Architettura
+
+- **Multi-stage build**: builder con Vulkan SDK (glslc da LunarG) → runtime
+  minimale con `mesa-vulkan-drivers` (RADV userspace).
+- **Nessun driver kernel nel container**: l'host possiede l'hardware; il
+  container monta solo `/dev/dri/renderD128` (node render, least-privilege).
+- **`GGML_NATIVE=OFF`**: binario portabile tra CPU diverse.
+- **Voci clonate** (`.spk`/`.rvq`) registrate all'avvio da `register_voices.py`
+  (scansione ricorsiva di `/voices`; ref_text dal `.txt` omonimo o
+  `*_batch_ref.txt`).
+
+## Requisiti host
+
+- Docker Engine + Docker Compose v2
+- GPU con driver Vulkan funzionante: verificare con `vulkaninfo --summary`
+- Node render: `ls -l /dev/dri/renderD128` (su Debian/Ubuntu serve il gruppo
+  `render`; su Fedora è world-writable)
+
+## Build & avvio
+
+```bash
+cd docker/tts-server
+export RENDER_GID=$(getent group render | cut -d: -f3)   # GID gruppo render
+docker compose up -d --build
+```
+
+Il compose monta `../models` e `../voices` (relativi al file compose, cioè la
+root del repo) e pubblica la porta **8881**.
+
+## Verifica
+
+```bash
+curl http://localhost:8881/health          # {"status":"ok"}
+curl http://localhost:8881/v1/audio/voices # elenco voci registrate
+```
+
+## Configurazione (variabili d'ambiente)
+
+| Variabile | Default | Descrizione |
+|---|---|---|
+| `MODEL_PATH` | `/models/qwen-talker-1.7b-base-Q8_0.gguf` | Modello talker |
+| `CODEC_PATH` | `/models/qwen-tokenizer-12hz-Q8_0.gguf` | Codec audio |
+| `TTS_LANG` | `auto` | Lingua (es. `Italian`) |
+| `MODEL_ALIAS` | — | Alias esposto da `/v1/models` |
+| `PORT` | `8881` | Porta di ascolto |
+| `VOICE_DIR` | `/voices` | Directory voci clonate |
+| `GGML_BACKEND` | `Vulkan0` | Backend ggml |
+| `NO_FA` / `CLAMP_FP16` / `MAX_BATCH` / ecc. | — | Flag avanzati tts-server |
+
+## Portabilità su altre macchine
+
+1. **AMD/Intel**: nessuna modifica — il container usa Mesa RADV dal
+   `mesa-vulkan-drivers` dell'immagine. Serve solo il node render.
+2. **NVIDIA**: aggiungere il runtime nvidia al servizio:
+   ```yaml
+   runtime: nvidia
+   environment:
+     NVIDIA_VISIBLE_DEVICES: all
+   ```
+   (richiede `nvidia-container-toolkit` sull'host) — oppure usare il target
+   `cuda` del Dockerfile principale del progetto.
+3. **Modelli**: montare la directory con i `.gguf` (talker + codec) come
+   `/models:ro`.
+4. **Voci**: montare la directory con `.spk`/`.rvq`/`.txt` come `/voices:ro`.
+
+## Troubleshooting
+
+1. Host: `vulkaninfo --summary` deve vedere la GPU.
+2. Container: `docker exec tts-server vulkaninfo --summary` (se `vulkan-tools`
+   installato) o controllare i log: `docker logs tts-server`.
+3. Se Vulkan non vede la GPU: verificare `--device /dev/dri/renderD128` e il
+   `group_add` (GID del gruppo `render` dell'host).
+4. Non bind-mountare librerie host (`/usr/lib`, ICD JSON): causa mismatch
+   loader/driver. L'immagine ha già RADV.
@@ -0,0 +1,41 @@
+# ============================================================================
+# tts-server (Qwen3-TTS, Vulkan) — docker compose
+# ----------------------------------------------------------------------------
+# Uso:
+#   export RENDER_GID=$(getent group render | cut -d: -f3)   # GID gruppo render
+#   docker compose up -d --build
+#
+# GPU: AMD/Intel via Mesa RADV (/dev/dri/renderD128). Su host dove il node
+# è world-writable (es. Fedora) group_add non è necessario ma innocuo.
+# NVIDIA: aggiungere il runtime nvidia (vedi README.md).
+#
+# Variabili utili:
+#   MODELS_DIR / VOICES_DIR : percorsi di modelli e voci (default: ../ dal compose)
+#   PORT                    : porta host (default 8881)
+# ============================================================================
+services:
+  tts-server:
+    build:
+      context: ../..
+      dockerfile: docker/tts-server/Dockerfile
+    image: qwentts-tts:vulkan
+    container_name: tts-server
+    devices:
+      - /dev/dri/renderD128:/dev/dri/renderD128
+    group_add:
+      - "${RENDER_GID:-44}"
+    environment:
+      GGML_BACKEND: Vulkan0
+      MODEL_PATH: /models/qwen-talker-1.7b-base-Q8_0.gguf
+      CODEC_PATH: /models/qwen-tokenizer-12hz-Q8_0.gguf
+      TTS_LANG: Italian
+      MODEL_ALIAS: qwen3-tts
+      HOST: 0.0.0.0
+      PORT: "8881"
+      VOICE_DIR: /voices
+    volumes:
+      - "${MODELS_DIR:-../models}:/models:ro"
+      - "${VOICES_DIR:-../voices}:/voices:ro"
+    ports:
+      - "${PORT:-8881}:8881"
+    restart: unless-stopped
@@ -0,0 +1,49 @@
+#!/bin/bash
+# ============================================================================
+# Entrypoint tts-server: avvia il server, attende /health, poi registra le
+# voci clonate (.spk/.rvq) trovate in VOICE_DIR.
+# Tutti i parametri sono configurabili via variabili d'ambiente.
+# ============================================================================
+set -e
+
+MODEL=${MODEL_PATH:-/models/qwen-talker-1.7b-base-Q8_0.gguf}
+CODEC=${CODEC_PATH:-/models/qwen-tokenizer-12hz-Q8_0.gguf}
+LANG=${TTS_LANG:-auto}
+HOST=${HOST:-0.0.0.0}
+PORT=${PORT:-8881}
+ALIAS=${MODEL_ALIAS:-}
+VOICE_DIR=${VOICE_DIR:-/voices}
+
+extra_args=()
+[ -n "$ALIAS" ] && extra_args+=(--alias "$ALIAS")
+[ -n "$CODEC_CHUNK_DUR" ] && extra_args+=(--codec-chunk-dur "$CODEC_CHUNK_DUR")
+[ -n "$CODEC_LEFT_DUR" ] && extra_args+=(--codec-left-dur "$CODEC_LEFT_DUR")
+[ -n "$MAX_BATCH" ] && extra_args+=(--max-batch "$MAX_BATCH")
+[ -n "$MAX_PREFILL_TOKENS" ] && extra_args+=(--max-prefill-tokens "$MAX_PREFILL_TOKENS")
+[ "$NO_FA" = "1" ] && extra_args+=(--no-fa)
+[ "$CLAMP_FP16" = "1" ] && extra_args+=(--clamp-fp16)
+
+echo "[tts] avvio tts-server (model=$MODEL codec=$CODEC lang=$LANG port=$PORT)"
+/app/tts-server \
+    --model "$MODEL" \
+    --codec "$CODEC" \
+    --lang "$LANG" \
+    --host "$HOST" \
+    --port "$PORT" \
+    "${extra_args[@]}" &
+SERVER_PID=$!
+
+until curl -sf "http://localhost:${PORT}/health" > /dev/null 2>&1; do
+    kill -0 "$SERVER_PID" 2>/dev/null || { echo "[tts] tts-server uscito prima di diventare healthy" >&2; wait "$SERVER_PID"; }
+    sleep 1
+done
+echo "[tts] server pronto su :${PORT}"
+
+if [ -d "$VOICE_DIR" ] && [ -n "$(ls -A "$VOICE_DIR" 2>/dev/null)" ]; then
+    echo "[tts] registrazione voci da $VOICE_DIR ..."
+    VOICE_DIR="$VOICE_DIR" TTS_PORT="$PORT" python3 /app/register_voices.py || echo "[tts] registrazione voci fallita (il server continua)"
+else
+    echo "[tts] nessuna voce in $VOICE_DIR, salto registrazione"
+fi
+
+wait $SERVER_PID
@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+"""Registra le voci clonate (.spk/.rvq) trovate in VOICE_DIR nel tts-server.
+
+Container-friendly: scansiona VOICE_DIR ricorsivamente; per ogni *.spk
+cerca il .rvq omonimo (obbligatorio) e il .txt omonimo (ref_text, opzionale;
+fallback: *_batch_ref.txt nella stessa directory).
+"""
+import base64
+import json
+import os
+import sys
+import urllib.request
+
+PORT = os.environ.get("TTS_PORT", "8881")
+URL = f"http://127.0.0.1:{PORT}/v1/audio/voices"
+VOICE_DIR = os.environ.get("VOICE_DIR", "/voices")
+
+
+def register(name, spk_path, rvq_path, ref_text):
+    spk_b64 = base64.b64encode(open(spk_path, "rb").read()).decode()
+    rvq_b64 = base64.b64encode(open(rvq_path, "rb").read()).decode()
+    body = {"name": name, "spk_b64": spk_b64, "rvq_b64": rvq_b64}
+    if ref_text:
+        body["ref_text"] = ref_text
+    req = urllib.request.Request(
+        URL, data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"}, method="POST",
+    )
+    with urllib.request.urlopen(req, timeout=30) as r:
+        resp = json.loads(r.read().decode())
+        print(f"{name}: {r.status} {resp.get('status')}")
+        return True
+
+
+def main():
+    if not os.path.isdir(VOICE_DIR):
+        print(f"VOICE_DIR {VOICE_DIR} non esiste, salto registrazione")
+        return 0
+    ok = total = 0
+    for root, _dirs, files in os.walk(VOICE_DIR):
+        batch_ref = None
+        for f in files:
+            if f.endswith("_batch_ref.txt"):
+                batch_ref = open(os.path.join(root, f)).read()
+        for f in sorted(files):
+            if not f.endswith(".spk"):
+                continue
+            stem = f[: -len(".spk")]
+            name = stem.replace("_ref", "").replace("_vocals", "")
+            spk_path = os.path.join(root, f)
+            rvq_path = os.path.join(root, stem + ".rvq")
+            if not os.path.isfile(rvq_path):
+                print(f"{name}: rvq mancante ({rvq_path}), salto")
+                continue
+            txt_path = os.path.join(root, stem + ".txt")
+            ref_text = None
+            if os.path.isfile(txt_path):
+                ref_text = open(txt_path).read()
+            elif batch_ref:
+                ref_text = batch_ref
+            total += 1
+            try:
+                if register(name, spk_path, rvq_path, ref_text):
+                    ok += 1
+            except Exception as e:
+                print(f"{name}: ERRORE {e}")
+    print(f"\nRegistrate {ok}/{total} voci")
+    return 0 if ok == total else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())