The task of detecting regionalisms (expressions or words used in certain regions) has traditionally relied on the use of questionnaires and surveys, and has also heavily depended on the expertise and intuition of the surveyor. The irruption of Social Media and its microblogging services has produced an unprecedented wealth of content, mainly informal text generated by users, opening new opportunities for linguists to extend their studies of language variation. Previous work on automatic detection of regionalisms depended mostly on word frequencies. In this work, we present a novel metric based on Information Theory that incorporates user frequency. We tested this metric on a corpus of Argentinian Spanish tweets in two ways: via manual annotation of the relevance of the retrieved terms, and also as a feature selection method for geolocation of users. In either case, our metric outperformed other techniques based solely in word frequency, suggesting that measuring the amount of users that produce a word is informative. This tool has helped lexicographers discover several unregistered words of Argentinian Spanish, as well as different meanings assigned to registered words.
翻译:发现区域主义(在某些地区使用的表达或词词)的任务传统上依赖于问卷和调查的使用,也在很大程度上依赖测量员的专门知识和直觉。社会媒体及其微博客服务的破坏产生了前所未有的大量内容,主要是用户产生的非正式文本,为语言学家提供了新的机会,以扩大语言差异的研究。以前自动发现区域主义的工作主要取决于文字频率。在这项工作中,我们根据信息理论提出了一个包含用户频率的新指标。我们用两种方式在阿根廷西班牙语推文中测试了这一指标:人工说明检索到的术语的相关性,同时也作为用户地理位置的特征选择方法。在这两种情况下,我们的衡量标准都超越了仅以文字频率为基础的其他技术,表明衡量生成一个词的用户的数量是信息性的。这一工具帮助词汇学家发现了阿根廷西班牙语的几种未注册词,以及对登记词的不同含义。