A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification
In the spatial cognition and emotion recognition tasks based on electroencephalography (EEG), the signal information representation of the single modal is incomplete because of the significant inter-subject differences in the EEG signals, resulting in low generalization performance of classification models. To address this issue, we propose a radial basis deformable residual convolutional neural networks model embedded with local multimodal feature knowledge (RBDRCNN-LMFK). The RBDRCNN leverages Euclidean distance alignment, deformable convolution, depth-separable convolution, and residual connection modules for effective EEG feature extraction. The embedded local feature knowledge algorithm enables the effective combination of multi-modal information. To further validate the effectiveness of the algorithm, we used the left-one-subject-out cross-validation algorithm on the virtual city walking (VCW), SJTU Emotion EEG Dataset IV (SEED IV), and Building block gesture recognition (BBGR) datasets for spatial cognition and emotion recognition tasks. The average accuracy of the VCW was 97.84 % using the kNN classifier, surpassing the Concate fusion method's 96.62 %. The average accuracy for the SEED IV dataset was 87.56 %, higher than the Concate Fusion's 69.97 %. On the BBGR dataset, the kNN classifier achieved an average accuracy of 87.80 %, compared to 85.31 % with the Concate fusion. The results show that the model enhances the recognition of biologically significant features in EEG signals by embedding local features and increases the correlation between different brain regions associated with the task. This work illustrates a promising direction of using deep learning models to discover effective task-related features from highly diverse EEG signals and enhance brain regions' correlation through multi-modal knowledge embedding.
基金:
National Key Research and Develop-ment Program of China [2021YFF1200603, 2023YFF1203702]; Na-tional Natural Science Foundation of China [62276022, 62206014]
第一作者机构:[1]Yanshan Univ, Sch Informat Sci & Engn, Qinhuangdao 066004, Hebei, Peoples R China
通讯作者:
通讯机构:[4]Univ Sci & Technol Beijing, Sch Intelligence Sci & Technol, Beijing 100083, Peoples R China[5]Univ Sci & Technol Beijing, Key Lab Percept & Control Intelligent Bion Unmanne, Minist Educ, Beijing 100083, Peoples R China
推荐引用方式(GB/T 7714):
Li Jingjing,Zhou Yanhong,Liu Tiange,et al.A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification[J].EXPERT SYSTEMS WITH APPLICATIONS.2024,257:doi:10.1016/j.eswa.2024.125089.
APA:
Li, Jingjing,Zhou, Yanhong,Liu, Tiange,Jung, Tzyy-Ping,Wan, Xianglong...&Wen, Dong.(2024).A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification.EXPERT SYSTEMS WITH APPLICATIONS,257,
MLA:
Li, Jingjing,et al."A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification".EXPERT SYSTEMS WITH APPLICATIONS 257.(2024)