Lyric-to-melody generation is an important task in automatic songwriting. Previous lyric-to-melody generation systems usually adopt end-to-end models that directly generate melodies from lyrics, which suffer from several issues: 1) lack of paired lyric-melody training data; 2) lack of control on generated melodies. In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e.g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i.e., the system consists of a lyric-to-template module and a template-to-melody module). TeleMelody has two advantages. First, it is data efficient. The template-to-melody module is trained in a self-supervised way (i.e., the source template is extracted from the target melody) that does not need any lyric-melody paired data. The lyric-to-template module is made up of some rules and a lyric-to-rhythm model, which is trained with paired lyric-rhythm data that is easier to obtain than paired lyric-melody data. Second, it is controllable. The design of template ensures that the generated melodies can be controlled by adjusting the musical elements in template. Both subjective and objective experimental evaluations demonstrate that TeleMelody generates melodies with higher quality, better controllability, and less requirement on paired lyric-melody data than previous generation systems.
翻译:在自动歌曲写作中, 音频到冶金生成是一项重要的任务。 以前的音频到冶金生成系统通常会采用从歌词直接产生旋律的端到端模型, 这些模型存在几个问题:(1) 缺乏配对的音律冶金培训数据;(2) 缺乏对生成的旋律的控制。 在本文中, 我们开发了TeleMeldy, 是一个两阶段的音频到冶金生成系统, 配有音乐模板( 例如, 调味、 曲律流进、 节奏模式和调味) 来弥合歌词和旋律( 即, 系统包括一个调音频到模版的模块和一个模版到冶金模块 。 TeleMeldy有两个优点。 首先, 它具有数据效率。 模版到冶金模块的训练方式是自我监督( e. 来源模板是从目标旋律流流流流流流中提取的), 它不需要任何音律和配对数据的评估。 精度- 精度- 精度到模制的模版模块模块模块中, 将数据制成一个更易数据制成更易的模制成为数据, 。