We propose a data-driven method for automatic deception detection in real-life trial data using visual and verbal cues. Using OpenFace with facial action unit recognition, we analyze the movement of facial features of the witness when posed with questions and the acoustic patterns using OpenSmile. We then perform a lexical analysis on the spoken words, emphasizing the use of pauses and utterance breaks, feeding that to a Support Vector Machine to test deceit or truth prediction. We then try out a method to incorporate utterance-based fusion of visual and lexical analysis, using string based matching.
翻译:我们提出一种数据驱动方法,用视觉和口头提示在真实试验数据中自动检测欺骗。我们使用面部动作单位识别的开放面孔,分析证人面部特征在提出问题时的动向以及使用OpenSmile的声学模式。我们然后对口语进行词汇分析,强调使用暂停和断语,将暂停和断语反馈给支持矢量机,以测试欺骗或真理预测。然后我们尝试一种方法,利用基于字符串的匹配,纳入基于语音的视觉和词汇分析组合。