Modern machine learning frameworks are complex: they are typically organised in multiple layers each of which is written in a different language and they depend on a number of external libraries, but at their core they mainly consist of tensor operations. As array-oriented languages provide perfect abstractions to implement tensor operations, we consider a minimalistic machine learning framework that is shallowly embedded in an array-oriented language and we study its productivity and performance. We do this by implementing a state of the art Convolutional Neural Network (CNN) and compare it against implementations in TensorFlow and PyTorch --- two state of the art industrial-strength frameworks. It turns out that our implementation is 2 and 3 times faster, even after fine-tuning the TensorFlow and PyTorch to our hardware --- a 64-core GPU-accelerated machine. The size of all three CNN specifications is the same, about 150 lines of code. Our mini framework is 150 lines of highly reusable hardware-agnostic code that does not depend on external libraries. The compiler for a host array language automatically generates parallel code for a chosen architecture. The key to such a balance between performance and portability lies in the design of the array language; in particular, the ability to express rank-polymorphic operations concisely, yet being able to do optimisations across them. This design builds on very few assumptions, and it is readily transferable to other contexts offering a clean approach to high-performance machine learning.
翻译:现代机器学习框架是复杂的:它们通常以多种层次组织起来,每个层次都用不同语言写成,它们取决于一些外部图书馆,但核心却主要由高压操作组成。由于以阵列为导向的语言提供了执行高压操作的完美抽象性,我们认为一个最起码的机器学习框架,它浅地嵌入了以阵列为导向的语言,我们研究其生产率和性能。我们这样做的方法是实施一个先进的革命神经网络(CNN),并将其与TensorFlow和PyTorrch(PyTorrch)中的实施方法进行比较 -- -- 两个艺术工业强度框架的状态。它的两个状态是工业强度框架。结果显示,即使对TensorFlow和PyTorrch的硬件进行微调后,我们的执行速度更快了2和3倍。我们认为一个最起码的机器学习框架是一个硬件 -- -- 一个64个核心的GPUP-加速机。所有三个CNN规格的大小都是一样的,大约150行的代码。我们的小型框架是150行高度可重复的硬件-认知性代码,不依赖外部图书馆。其他主机组阵列语言自动自动生成的自动生成代码,一个可自动生成的自动生成和可移动的代码,在所选的机型结构结构中,一个特殊的操作中, 也是最高级的操作的功能。在特定的设计结构结构结构中进行。