Description
dots.tts.edit V1.1 is an advanced continuous autoregressive model developed for sophisticated speech editing and zero-shot text-to-speech (TTS) synthesis. This model offers granular control over speech, allowing users to perform a wide range of modifications while maintaining the integrity of the speaker's voice and the surrounding acoustic environment.
Read More
The model supports comprehensive editing operations, including text replacement, insertion, and deletion, which are crucial for refining spoken content. Beyond basic text manipulation, it provides localized control over emotion and allows for free-form style descriptions, enabling nuanced adjustments to the speech delivery. Users can also precisely control pitch and speaking rate, insert or reduce pauses, and apply speech enhancement or background-audio operations. A key capability is the composition of multiple edit operations, offering flexibility for complex editing tasks.
dots.tts.edit V1.1 is multilingual, supporting both English and Mandarin speech editing and zero-shot synthesis. It generates high-quality 48 kHz audio. The core weights of the model utilize BF16 precision, while the speaker encoder and vocoder maintain FP32 weights for optimal performance. The model can be accessed via a command-line interface (CLI) for speech editing or through a Python API for more integrated applications. It also includes an interactive playground for local experimentation.
Installation involves installing the latest main branch of dots.tts and its recommended dependencies. The CLI requires source audio, a tagged instruction, and an output path, with optional source and target transcripts. The Python API supports various structural tags for detailed control over edits. For zero-shot TTS, the DotsTtsEditRuntime provides `generate` and `generate_stream` functions.
Users are advised to use the model responsibly. Generated or edited speech may occasionally contain errors in pronunciation, timing, speaker similarity, or instruction following. The model should not be used for impersonation, misleading listeners, violating consent, or creating unlawful content. Explicit consent from speakers is required for using their voices. Publicly shared audio that could be mistaken for authentic, unmodified recordings must be clearly disclosed as AI-generated or edited. Misuse can be reported through the model repository's Community discussions.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud
- PlatformsWeb
dots.tts.edit Highlights
Continuous autoregressive model for speech editing
Zero-shot text-to-speech synthesis
Text replacement, insertion, and deletion
Localized emotion control and free-form style descriptions
Pitch and speaking-rate control
Pause insertion and reduction
Speech enhancement and background-audio operations
Composition of multiple edit operations
Preserves speaker and acoustic context outside edited regions
Supports English and Mandarin speech editing and synthesis
Produces 48 kHz audio output
Command-line interface (CLI) for speech editing
Python API for integration
Interactive local edit playground available
BF16 core weights with FP32 speaker encoder and vocoder
Getting Started with dots.tts.edit
Install Dependencies: Install the latest main branch of dots.tts and its recommended dependencies.
Prepare Source Audio: Provide source audio for editing operations.
Define Instructions: Create tagged instructions for desired speech edits, including text, emotion, pitch, or rate controls.
Execute CLI Command: Use the speech editing CLI with source audio, tagged instruction, and output path.
Override Transcripts (Optional): Use --source-text and --target-text to override derived transcripts if needed.
Adjust Speaker Guidance (Optional): Control speaker guidance with --use-xvector and --speaker-scale parameters.
Integrate via Python API: Utilize the Python API with structural tags like <del>, <ins>, <emo>, <pitch> for programmatic control.
Generate Zero-Shot TTS: Use `generate` or `generate_stream` functions from DotsTtsEditRuntime for text-to-speech synthesis.
dots.tts.edit's Use Cases
- Speech Content Refinement
- Emotion and Style Adjustment
- Prosody Control
- Audio Enhancement
- Zero-Shot TTS
- Multilingual Speech Production
- Interactive Speech Experimentation







