Over 800 languages are spoken across West Africa. Despite the obvious diversity among people who speak these languages, one language significantly unifies them all - West African Pidgin English. There are at least 80 million speakers of West African Pidgin English. However, there is no known natural language processing (NLP) work on this language. In this work, we perform the first NLP work on the most popular variant of the language, providing three major contributions. First, the provision of a Pidgin corpus of over 56000 sentences, which is the largest we know of. Secondly, the training of the first ever cross-lingual embedding between Pidgin and English. This aligned embedding will be helpful in the performance of various downstream tasks between English and Pidgin. Thirdly, the training of an Unsupervised Neural Machine Translation model between Pidgin and English which achieves BLEU scores of 7.93 from Pidgin to English, and 5.18 from English to Pidgin. In all, this work greatly reduces the barrier of entry for future NLP works on West African Pidgin English.
翻译：西非有800多种语言。尽管讲这些语言的人有明显的多样性,但一种语言使所有语言 -- -- 西非皮德金英语 -- -- 大大统一了起来。至少有8 000万西非皮德金英语使用者。然而,没有关于这种语言的已知自然语言处理(NLP)工作。在这项工作中,我们在最流行的语言变种方面开展了首部国家语言处理工作,提供了三大贡献。首先,提供了56,000多句的Pidgin文集,这是我们所知最多的。第二,培训了第一次在Pidgin和英语之间跨语言嵌入。这种结合将帮助完成英语和皮德金之间的各种下游任务。第三,培训了皮德金和英语之间无人监督的神经机器翻译模型,从Pidgin到英语之间达到了7.93分,从英语到Pidgin之间达到了5.18分的BLEU分。这项工作极大地减少了未来NLP在西非洲皮德金英语方面工作的进入障碍。
相关内容
微信扫码咨询专知VIP会员