Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HR, available at 'http://vipl.ict.ac.cn/view_database.php?id=15'), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the state-of-the-art methods on both the public-domain and our VIPL-HR databases.
翻译:心率(HR)是一个重要的生理信号,反映了一个人的生理和情感状况。传统的HR测量通常依赖于接触显示器,这可能造成不便和不适。最近,有人提议了一些方法,从脸部视频中进行远程HR估算;然而,大多数方法侧重于控制良好的情景,其概括性能力不为人知,进入不太紧张的情景(如头部移动和照明不良)。与此同时,缺乏大规模HR数据库限制了对远方HR估计的深层模型的使用。在本文中,我们提议建立一个终端至终端RhythmNet,从脸部进行远程HR估算。在RyhthmNet中,我们使用空间时空代表将多个ROI数量中的HR信号编码作为其输入。随后,空间时空表达能力被反馈到一个动态网络,用于人力资源估算。我们还考虑到通过Gederate Complia Unit (GRU) 的视频序列中相邻的HR测量关系,以及实现高效的HR测量。此外,我们还在高层次的HRM-modal HR-al 版本数据库上建立一个大规模多版本版本版本版本,作为 Vireal-lievalal-lieval-lievational-lievational lievational listal listalal listal ladeal 。