Description
Fish Audio Drama 3 is the latest text-to-speech model from Fish Audio, specifically engineered for multi-character scenes and long-form content. This model supports multi-speaker generation, voice cloning, and natural-language control, allowing users to create dynamic and engaging audio productions. With 35 billion parameters and 3 billion active per token, it utilizes a MoE text backbone with hybrid attention to deliver high-quality speech synthesis.
Read More
The model features a Codebook-MTP audio decoder, which decodes 10 codebooks causally over the entire sequence, ensuring smooth and coherent audio output. It also employs FP8 MoE experts with the remaining weights in bf16, optimizing performance and efficiency. Users can incorporate up to 16 different characters in a single text using speaker tags, providing flexibility in audio storytelling.
Fish Audio Drama 3 automatically chunks long text and streams it per chunk, enhancing its usability for lengthy scripts. It is launched with SGLang Omni, offering seamless integration and deployment. The Drama 3 Cookbook provides detailed instructions for installation, deployment, and API usage, making it accessible for developers and audio producers.
Performance metrics on NVIDIA H200 GPUs demonstrate its efficiency, with a throughput of up to 1,687.3 frames per second at high concurrency levels. The model maintains a low time to first audio (TTFA) and a favorable real-time factor (RTF), ensuring quick and responsive audio generation. The model is licensed under the Fish Audio Community License, promoting open access and collaboration.
- CategoryText to Speech
- Pricing ModelOpen source
- DeploymentCloud
- PlatformsWeb
Fish Audio Drama 3 Highlights
Multi-speaker generation
Voice cloning
Natural-language control
35B parameters
MoE text backbone
Codebook-MTP audio decoder
FP8 MoE experts
Automatic text chunking
Getting Started with Fish Audio Drama 3
Access model: Visit Hugging Face
Authenticate: Use API key
Set up environment: Follow Cookbook
Integrate via API: Use provided examples
Optimize: Adjust parameters for performance
Fish Audio Drama 3's Use Cases
- Audio Productions
- Voice Cloning
- Long-form Narratives
- Interactive Storytelling
- Podcasting
