This paper introduces a new set of video analytics plugins developed for the FFmpeg framework. Multimedia applications that increasingly utilize the FFmpeg media features for its comprehensive media encoding, decoding, muxing, and demuxing capabilities can now additionally analyze the video content based on AI models. The plugins are thread optimized for best performance overcoming certain FFmpeg threading limitations. The plugins utilize the Intel OpenVINO Toolkit inference engine as the backend. The analytics workloads are accelerated on different platforms such as CPU, GPU, FPGA or specialized analytics accelerators. With our reference implementation, the feature of OpenVINO as inference backend has been pushed into FFmpeg mainstream repository. We plan to submit more patches later.
翻译:本文介绍为 FFmpeg 框架开发的一套新的视频分析插件 。 日益利用 FFmpeg 媒体特性进行综合媒体编码、 解码、 混音和解模能力的多媒体应用程序现在可以另外分析基于 AI 模型的视频内容。 插件优化为最佳性能以克服某些 FFmpeg 线性限制。 插件使用 Intel OpenVINO 工具包推导引擎作为后端。 分析工作量在不同的平台上加速, 如 CPU、 GPU、 FPGA 或专门分析加速器。 在我们的参考执行中, OpenVINO 作为推导出后端的功能被推到了 FFpeg 主流仓库中。 我们计划稍后提交更多补丁 。