We describe TensorFlow-Serving, a system to serve machine learning models inside Google which is also available in the cloud and via open-source. It is extremely flexible in terms of the types of ML platforms it supports, and ways to integrate with systems that convey new models and updated versions from training to serving. At the same time, the core code paths around model lookup and inference have been carefully optimized to avoid performance pitfalls observed in naive implementations. Google uses it in many production deployments, including a multi-tenant model hosting service called TFS^2.
翻译:我们描述了TensorFlow-Servicing(TensorFlow-Servicing),这是一个在谷歌内部提供机器学习模型的系统,在云中和通过开放源码也可以找到,在它所支持的ML平台类型和与将新模型和最新版本从培训到服务的系统整合的方式方面极为灵活,同时,模型搜索和推断的核心代码路径得到了仔细优化,以避免在天真的实施过程中观察到的性能缺陷。谷歌在许多生产部署中使用了它,包括一个称为TFS2的多租户模式主机服务。