We describe TensorFlow-Serving, a system to serve machine learning models inside Google which is also available in the cloud and via open-source. It is extremely flexible in terms of the types of ML platforms it supports, and ways to integrate with systems that convey new models and updated versions from training to serving. At the same time, the core code paths around model lookup and inference have been carefully optimized to avoid performance pitfalls observed in naive implementations. Google uses it in many production deployments, including a multi-tenant model hosting service called TFS.
翻译:我们描述了TensorFlow-Servicing(TensorFlow-Servicing),这是一个在谷歌内部提供机器学习模型的系统,也存在于云层和开放源码中;它具有极大的灵活性,它所支持的ML平台类型和如何与从培训到服务的传输新模型和最新版本的系统整合;与此同时,模型搜索和推断的核心代码路径也经过仔细优化,以避免在天真的实施过程中观察到的性能缺陷;谷歌在许多生产部署中使用了它,包括一个称为TFS的多耗时模型托管服务。