Self-supervised speech representation and contextual text embedding for match-mismatch classification with EEG recording

Relating speech to EEG holds considerable importance but challenging. In this study, deep convolutional network was employed to extract spatiotemporal features from EEG data. Self-supervised speech representation and contextual text embedding were used as speech features. Contrastive learning was used to related EEG features to speech features. The experimental results demonstrate the benefits of using self-supervised speech representation and contextual text embedding. Through feature fusion and model ensemble, an accuracy of 60.29% was achieved, and the performance was ranked as No.2 in Task1 of the Auditory EEG Challenge (ICASSP 2024).