Deep neural networks (DNNs) have produced state-of-the-art results in many benchmarks and problem domains. However, the success of DNNs depends on the proper configuration of its architecture and hyperparameters. Such a configuration is difficult and as a result, DNNs are often not used to their full potential. In addition, DNNs in commercial applications often need to satisfy real-world design constraints such as size or number of parameters. To make configuration easier, automatic machine learning (AutoML) systems for deep learning have been developed, focusing mostly on optimization of hyperparameters. This paper takes AutoML a step further. It introduces an evolutionary AutoML framework called LEAF that not only optimizes hyperparameters but also network architectures and the size of the network. LEAF makes use of both state-of-the-art evolutionary algorithms (EAs) and distributed computing frameworks. Experimental results on medical image classification and natural language analysis show that the framework can be used to achieve state-of-the-art performance. In particular, LEAF demonstrates that architecture optimization provides a significant boost over hyperparameter optimization, and that networks can be minimized at the same time with little drop in performance. LEAF therefore forms a foundation for democratizing and improving AI, as well as making AI practical in future applications.
翻译:深心神经网络(DNN)在许多基准和问题领域产生了最先进的结果。然而,DNN的成功取决于其建筑和超参数的适当配置。这种配置是困难的,因此DNN往往没有充分利用其潜力。此外,商业应用中的DNN往往需要满足现实世界设计限制,如大小或参数数等。为了使配置更加容易,开发了用于深思熟虑的自动机器学习系统(Automal),主要侧重于优化超参数。本文将AutML更进一步。它引入了称为LEAAAF的进化自动ML框架,这个称为LAAF,不仅优化超参数,而且优化网络结构和网络规模。LEAF利用了最新进化算法(EA)和分布式计算框架。医学图像分类和自然语言分析的实验结果显示,该框架可以用来实现最先进的业绩。特别是,LAF表明,建筑的优化大大地推动了超常参数的升级,因此,将来的网络可以作为AIAF的最起码的改进形式。