Self-supervised learning (SSL) methods such as WavLM have shown promising speech separation (SS) results in small-scale simulation-based experiments. In this work, we extend the exploration of the SSL-based SS by massively scaling up both the pre-training data (more than 300K hours) and fine-tuning data (10K hours). We also investigate various techniques to efficiently integrate the pre-trained model with the SS network under a limited computation budget, including a low frame rate SSL model training setup and a fine-tuning scheme using only the part of the pre-trained model. Compared with a supervised baseline and the WavLM-based SS model using feature embeddings obtained with the previously released 94K hours trained WavLM, our proposed model obtains 15.9% and 11.2% of relative word error rate (WER) reductions, respectively, for a simulated far-field speech mixture test set. For conversation transcription on real meeting recordings using continuous speech separation, the proposed model achieves 6.8% and 10.6% of relative WER reductions over the purely supervised baseline on AMI and ICSI evaluation sets, respectively, while reducing the computational cost by 38%.
翻译:在这项工作中,我们通过大规模扩大培训前数据(超过300K小时)和微调数据(10K小时),扩大了基于SSL的SS的探索范围。我们还调查了各种技术,以便在有限的计算预算下,将预先培训的模式与SSS网络有效地结合起来,包括低框架速率SSL模式培训设置和微调计划,仅使用经过培训的模型的一部分,与监督的基线和基于WavLM的SS模型相比,我们的拟议模型使用与先前公布的94K小时培训WavLM的功能嵌入器相比,分别获得了15.9%和11.2%相对字差率(WER)的降幅,用于模拟远处语音混合测试。关于使用连续语音分离的实际会议录音的谈话记录,拟议模型在AMI和ICSI的纯监督基线上分别实现了6.8%和10.6%的相对WER降幅,同时通过计算降低38 %的成本。