O_O VC: Synthetic Data-Driven One-to-One Alignment for Any-to-Any Voice Conversion

Voice conversion (VC) aims to transform a source speaker's voice to sound like a target speaker, while preserving the original speech content. Traditional methods typically attempt to separate speaker identity and linguistic information into distinct representations, which are then combined to reconstruct the audio. However, effectively disentangling these factors remains challenging, often leading to information loss during training. In this paper, we propose a new approach that leverages synthetic data generated by a high-quality, pretrained multi-speaker text-to-speech (TTS) model. Specifically, we create synthetic data pairs that share the same linguistic content but differ in speaker identity, using them as input-output pairs for training the voice conversion model. Additionally, we introduce a flexible training strategy for any-to-any voice conversion. This method generalizes well to unseen speakers, improving both adaptability and performance in zero-shot scenarios.