Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.
翻译:表面行动股(AU)的承认是面部表现分析的一项关键任务,在人工智能和计算机视觉领域引起了广泛的注意; 现有工作的重点要么是设计或学习复杂的区域特征代表,要么研究各种类型的非盟关系模型; 尽管取得了不同程度的进展,但处理复杂情况的现有方法仍然很困难; 在本文件中,我们调查如何将非盟之间的语义关系传播纳入一个深层神经网络框架,以加强面部区域的特征代表性,并提议非盟的语义关系内嵌代表学习框架(SRERL),具体地说,我们通过分析各种面部表现中的共生关系和相互排斥非盟,以结构化知识图表的形式组织面部非盟,并将GGNNN(G)纳入一个多尺度的CNN(GG)框架,通过图表传播节点信息,以加强非盟的代表性; 由于所了解的特征涉及面部特征和非盟关系推理,因此拟议模式更加有力,能够应对更具挑战性的案件,例如,不精确的变化和部分的表面表现,从而展示我们以往的状态。