AutoML与轻量模型大列表

导读

一份高质量(最新的)AutoML工作和轻量级模型的列表,包括神经结构搜索,轻量级结构,模型压缩和加速,超参数优化,自动特征工程。

作者 | guan-yuan 

编译 | Xiaowen 

Github: 

https://github.com/guan-yuan/awesome-AutoML-and-Lightweight-Models

1.) Neural Architecture Search

[论文]

Gradient:

  • ASAP: Architecture Search, Anneal and Prune | [2019/04]

  • Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours | [2019/04]

    • dstamoulis/single-path-nas | [Tensorflow]

  • Automatic Convolutional Neural Architecture Search for Image Classification Under Different Scenes | [IEEE Access 2019]

  • sharpDARTS: Faster and More Accurate Differentiable Architecture Search | [2019/03]

  • Learning Implicitly Recurrent CNNs Through Parameter Sharing | [ICLR 2019]

    • lolemacs/soft-sharing | [Pytorch]

  • Probabilistic Neural Architecture Search | [2019/02]

  • Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation | [2019/01]

  • SNAS: Stochastic Neural Architecture Search | [ICLR 2019]

  • FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search | [2018/12]

  • Neural Architecture Optimization | [NIPS 2018]

    • renqianluo/NAO | [Tensorflow]

  • DARTS: Differentiable Architecture Search | [2018/06]

    • quark0/darts | [Pytorch]

    • khanrc/pt.darts | [Pytorch]

    • dragen1860/DARTS-PyTorch | [Pytorch]

Reinforcement Learning:

  • Template-Based Automatic Search of Compact Semantic Segmentation Architectures | [2019/04]

  • Understanding Neural Architecture Search Techniques | [2019/03]

  • Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search | [2019/01]

    • falsr/FALSR | [Tensorflow]

  • Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search | [2019/01]

    • moremnas/MoreMNAS | [Tensorflow]

  • ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware | [ICLR 2019]

    • MIT-HAN-LAB/ProxylessNAS | [Pytorch, Tensorflow]

  • Transfer Learning with Neural AutoML | [NIPS 2018]

  • Learning Transferable Architectures for Scalable Image Recognition | [2018/07]

    • wandering007/nasnet-pytorch | [Pytorch]

    • tensorflow/models/research/slim/nets/nasnet | [Tensorflow]

  • MnasNet: Platform-Aware Neural Architecture Search for Mobile | [2018/07]

    • AnjieZheng/MnasNet-PyTorch | [Pytorch]

  • Practical Block-wise Neural Network Architecture Generation | [CVPR 2018]

  • Efficient Neural Architecture Search via Parameter Sharing | [ICML 2018]

    • melodyguan/enas | [Tensorflow]

    • carpedm20/ENAS-pytorch | [Pytorch]

  • Efficient Architecture Search by Network Transformation | [AAAI 2018]

Evolutionary Algorithm:

  • Single Path One-Shot Neural Architecture Search with Uniform Sampling | [2019/04]

  • DetNAS: Neural Architecture Search on Object Detection | [2019/03]

  • The Evolved Transformer | [2019/01]

  • Designing neural networks through neuroevolution | [Nature Machine Intelligence 2019]

  • EAT-NAS: Elastic Architecture Transfer for Accelerating Large-scale Neural Architecture Search | [2019/01]

  • Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution | [ICLR 2019]

SMBO:

  • MFAS: Multimodal Fusion Architecture Search | [CVPR 2019]

  • DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures | [ECCV 2018]

  • Progressive Neural Architecture Search | [ECCV 2018]

    • titu1994/progressive-neural-architecture-search | [Keras, Tensorflow]

    • chenxi116/PNASNet.pytorch | [Pytorch]

Random Search:

  • Exploring Randomly Wired Neural Networks for Image Recognition | [2019/04]

  • Searching for Efficient Multi-Scale Architectures for Dense Image Prediction | [NIPS 2018]

Hypernetwork:

  • Graph HyperNetworks for Neural Architecture Search | [ICLR 2019]

Bayesian Optimization:

  • Inductive Transfer for Neural Architecture Optimization | [2019/03]

Partial Order Pruning

  • Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search | [CVPR 2019]

    • lixincn2015/Partial-Order-Pruning | [Caffe]

Knowledge Distillation

  • Improving Neural Architecture Search Image Classifiers via Ensemble Learning | [2019/03]

[项目]

  • Microsoft/nni | [Python]

2.) Lightweight Structures

[论文]

Segmentation:

  • ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network | [2018/11]

    • sacmehta/ESPNetv2 | [Pytorch]

  • ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation | [ECCV 2018]

    • sacmehta/ESPNet | [Pytorch]

  • BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation | [ECCV 2018]

    • ooooverflow/BiSeNet | [Pytorch]

    • ycszen/TorchSeg | [Pytorch]

  • ERFNet: Efficient Residual Factorized ConvNet for Real-time Semantic Segmentation | [T-ITS 2017]

    • Eromera/erfnet_pytorch | [Pytorch]

Object Detection:

  • Pooling Pyramid Network for Object Detection | [2018/09]

    • tensorflow/models | [Tensorflow]

  • Tiny-DSOD: Lightweight Object Detection for Resource-Restricted Usages | [BMVC 2018]

    • lyxok1/Tiny-DSOD | [Caffe]

  • Pelee: A Real-Time Object Detection System on Mobile Devices | [NeurIPS 2018]

    • Robert-JunWang/Pelee | [Caffe]

    • Robert-JunWang/PeleeNet | [Pytorch]

  • Receptive Field Block Net for Accurate and Fast Object Detection | [ECCV 2018]

    • ruinmessi/RFBNet | [Pytorch]

    • ShuangXieIrene/ssds.pytorch | [Pytorch]

    • lzx1413/PytorchSSD | [Pytorch]

  • FSSD: Feature Fusion Single Shot Multibox Detector | [2017/12]

    • ShuangXieIrene/ssds.pytorch | [Pytorch]

    • lzx1413/PytorchSSD | [Pytorch]

    • dlyldxwl/fssd.pytorch | [Pytorch]

  • Feature Pyramid Networks for Object Detection | [CVPR 2017]

    • tensorflow/models | [Tensorflow]

3.) Model Compression & Acceleration

[论文]

Compression:

  • Slimmable Neural Networks | [ICLR 2019]

    • JiahuiYu/slimmable_networks | [Pytorch]

  • AMC: AutoML for Model Compression and Acceleration on Mobile Devices | [ECCV 2018]

    • AutoML for Model Compression (AMC): Trials and Tribulations | [Pytorch]

  • Learning Efficient Convolutional Networks through Network Slimming | [ICCV 2017]

    • foolwood/pytorch-slimming | [Pytorch]

  • Channel Pruning for Accelerating Very Deep Neural Networks | [ICCV 2017]

    • yihui-he/channel-pruning | [Caffe]

  • Pruning Convolutional Neural Networks for Resource Efficient Inference | [ICLR 2017]

    • jacobgil/pytorch-pruning | [Pytorch]

  • Pruning Filters for Efficient ConvNets | [ICLR 2017]

Acceleration:

  • Fast Algorithms for Convolutional Neural Networks | [CVPR 2016]

    • andravin/wincnn | [Python]

[项目]

  • NervanaSystems/distiller | [Pytorch]

  • Tencent/PocketFlow | [Tensorflow]

[教程/博客]

  • Introducing the CVPR 2018 On-Device Visual Intelligence Challenge

4.) Hyperparameter Optimization

[论文]

  • Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly | [2019/03]

    • dragonfly/dragonfly

  • Google vizier: A service for black-box optimization | [SIGKDD 2017]

[项目]

  • Microsoft/nni | [Python]

  • dragonfly/dragonfly | [Python]

[教程/博客]

  • Hyperparameter tuning in Cloud Machine Learning Engine using Bayesian Optimization

  • Overview of Bayesian Optimization

  • Bayesian optimization

    • krasserm/bayesian-machine-learning | [Python]

5.) Automated Feature Engineering

Model Analyzer

  • Netscope CNN Analyzer | [Caffe]

  • sksq96/pytorch-summary | [Pytorch]

  • Lyken17/pytorch-OpCounter | [Pytorch]

References

  • LITERATURE ON NEURAL ARCHITECTURE SEARCH

  • handong1587/handong1587.github.io

  • hibayesian/awesome-automl-papers

  • mrgloom/awesome-semantic-segmentation

  • amusi/awesome-object-detection


-END-

专 · 知

专知,专业可信的人工智能知识分发,让认知协作更快更好!欢迎登录www.zhuanzhi.ai,注册登录专知,获取更多AI知识资料!

欢迎微信扫一扫加入专知人工智能知识星球群,获取最新AI专业干货知识教程视频资料和与专家交流咨询!

请加专知小助手微信(扫一扫如下二维码添加),加入专知人工智能主题群,咨询技术商务合作~

专知《深度学习:算法到实战》课程全部完成!530+位同学在学习,现在报名,限时优惠!网易云课堂人工智能畅销榜首位!

点击“阅读原文”,了解报名专知《深度学习:算法到实战》课程

展开全文
Top
微信扫码咨询专知VIP会员