Skip to content

feat: Local TTS provider support (Supertonic, Index-TTS) #105

Description

@fuleinist

Problem

The current TTS pipeline (generate_voiceover) only supports remote HTTP API providers (bytedance, minimax, 302). Users who want to generate voiceovers locally — without relying on paid cloud APIs, rate limits, or network latency — have no option.

Local TTS is valuable for:

  • Cost: zero per-character charges after the initial model download
  • Latency: no round-trip to external APIs (~200–500ms on CPU vs. several seconds over the network)
  • Privacy: audio never leaves the machine
  • Offline/air-gapped environments
  • Batch/render workflows where hundreds of segments make API costs prohibitive

Proposal

Add a new local TTS provider type alongside the existing remote providers. The implementation should:

  1. Define a local provider interface (model path, voice, language, speed → WAV bytes)
  2. Ship at least one built-in local engine
  3. Allow users to select "Local" as the TTS provider in the web UI sidebar, choosing which local engine to use
  4. Reuse the existing tts_providers.json parameter schema + config.toml [generate_voiceover.providers.*] pattern so local providers are configured the same way as remote ones

Suggested Local Engines

1. Supertonic (supertone-inc/supertonic)

  • Repo: https://github.com/supertone-inc/supertonic
  • Install: pip install supertonic (or pip install 'supertonic[serve]' for HTTP server mode)
  • Model: auto-downloads to ~/.cache/supertonic3/ (~400MB)
  • Languages: 31 (zh, en, ja, ko, fr, de, es, pt, ru, ar, hi, vi, etc.)
  • Voices: 10 built-in styles (M1–M5 male, F1–F5 female), plus zero-shot voice cloning via Voice Builder JSON export
  • Expressions: {{sad}}, {{laugh}}, {{whisper}}, {{shout}}, {{cry}}, etc.
  • Output: 44.1kHz 16-bit WAV, no upsampler needed
  • Hardware: CPU-only, runs on Raspberry Pi / edge devices. No GPU required.
  • Speed: ~200–500ms per sentence on CPU (tested: 10.33s output in 9.61s wall time)
  • SDK: from supertonic import TTS; tts = TTS(); wav, dur = tts.synthesize(text, voice_style=style, total_steps=8)
  • HTTP server mode: supertonic[serve] exposes POST /v1/tts on 127.0.0.1:7788 — can reuse the same HTTP handler pattern as the existing 302/minimax providers

2. Index-TTS (index-tts/index-tts)

  • Repo: https://github.com/index-tts/index-tts
  • Description: Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
  • Install: pip install index-tts
  • Key feature: voice cloning from a reference audio file — provide a 10–30s clip and it replicates the speaker's voice
  • Languages: zh, en, ja, and more
  • Speed: GPU-accelerated (CUDA), with CPU fallback
  • Model: downloads from HuggingFace (IndexTeam/Index-TTS)
  • Use case: ideal for users who want a specific speaker voice without recording new audio

Acceptance Criteria

  • User can select a local TTS engine (Supertonic and/or Index-TTS) from the web UI sidebar
  • Local engine generates WAV output compatible with the existing voiceover pipeline (same voiceover_id / duration / path contract in generate_voiceover.py)
  • Configurable via config.toml under [generate_voiceover.providers.supertonic] and [generate_voiceover.providers.index_tts]
  • Parameter schema defined in resource/tts/tts_providers.json for each local engine
  • Graceful fallback: if local model files are missing, prompt the user to run a download command rather than crashing
  • No breaking changes to existing remote TTS providers

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions