效率方法在自然语言处理中的应用：一项综述 (Efficient Methods for Natural Language Processing: A Survey)

Marcos Treviso,Ji-Ung Lee,Tianchu Ji,Betty van Aken,Qingqing Cao,Manuel R. Ciosici,Michael Hassid,Kenneth Heafield,Sara Hooker,Colin Raffel,Pedro H. Martins,André F. T. Martins,Jessica Zosa Forde,Peter Milder,Edwin Simpson,Noam Slonim,Jesse Dodge,Emma Strubell,Niranjan Balasubramanian,Leon Derczynski,Iryna Gurevych,Roy Schwartz

from arxiv, Accepted at TACL, pre publication version

Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources include data, time, storage, or energy, all of which are naturally limited and unevenly distributed. This motivates research into efficient methods that require fewer resources to achieve similar results. This survey synthesizes and relates current methods and findings in efficient NLP. We aim to provide both guidance for conducting NLP under limited resources, and point towards promising research directions for developing more efficient methods.

翻译：近年来，自然语言处理领域取得了诸多成果，其中通过放大模型参数和训练数据可以提高性能。然而，仅仅依赖数据规模的改变来提升性能也意味着消耗更多的资源。这些资源包括数据、时间、存储和能源等，这些资源的分配非常有限。这促使人们研究使用更少资源来实现类似结果的高效方法。这份综述综合了自然语言处理领域中当前的高效方法和研究成果。我们旨在提供通过有限资源限制条件下运行自然语言处理的指导，并指出发展更加高效方法的有前途的研究方向。