Aggarwal, S.; Uttam, S.; Garg, S.; Garg, S.; Jain, K.; Aggarwal, S. Advancements in End-to-End Audio Style Transformation: A Differentiable Approach for Voice Conversion and Musical Style Transfer. Preprints2024, 2024110396. https://doi.org/10.20944/preprints202411.0396.v1
APA Style
Aggarwal, S., Uttam, S., Garg, S., Garg, S., Jain, K., & Aggarwal, S. (2024). Advancements in End-to-End Audio Style Transformation: A Differentiable Approach for Voice Conversion and Musical Style Transfer. Preprints. https://doi.org/10.20944/preprints202411.0396.v1
Chicago/Turabian Style
Aggarwal, S., Kopal Jain and Swati Aggarwal. 2024 "Advancements in End-to-End Audio Style Transformation: A Differentiable Approach for Voice Conversion and Musical Style Transfer" Preprints. https://doi.org/10.20944/preprints202411.0396.v1
Abstract
We introduce a fully differentiable end-to-end audio transformation network designed to convert the style of one audio sample to another. This method offers three significant advantages: (a) it operates without the need for parallel utterances, transcriptions, or time alignment processes; (b) it utilizes a global conditioning mechanism, making it vocabulary agnostic and capable of transforming audio styles regardless of the target identity; and (c) it performs one-shot audio transformations without intermediate phonetic representations, thus eliminating the necessity for phonetic alignments and speaker-independent ASR networks. We assess our method against existing approaches in voice conversion and musical style transfer tasks. Subjective evaluations demonstrate the superiority of our approach. The network employs an encoder-decoder architecture that integrates neural network models known for their explainability in natural language processing tasks.
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
