ArticleComputational intelligence and neuroscience2017
Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition.
Article in Computational intelligence and neuroscience, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed.
- A Music Interaction System Based on the Emotional Recognition of Older Adults in Smart Homes: Cross-Sectional Experimental Study.JMIR aging · 2026Article
- Facial Emotion Recognition Deficits and Electroencephalogram Alterations in Mild Behavioral Impairment.Neuro-degenerative diseases · 2026Article
- A Comprehensive Review of Multimodal Emotion Recognition: Techniques, Challenges, and Future Directions.Biomimetics (Basel, Switzerland) · 2025Review
- Multimodal fusion for anticipating human decision performance.Scientific reports · 2024Article
- Simultaneous EEG-fNIRS Data Classification Through Selective Channel Representation and Spectrogram Imaging.IEEE journal of translational engineering in health and medicine · 2024Article
- Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG.IEEE open journal of engineering in medicine and biology · 2024Article
- Empowering human-AI teams via Intentional Behavioral Synchrony.Frontiers in neuroergonomics · 2023Article
- A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm.Frontiers in neuroscience · 2022Article
- Realization of Self-Adaptive Higher Teaching Management Based Upon Expression and Speech Multimodal Emotion Recognition.Frontiers in psychology · 2022Article
- Fusion of EEG-Based Activation, Spatial, and Connection Patterns for Fear Emotion Recognition.Computational intelligence and neuroscience · 2022Article
- Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition.Frontiers in psychology · 2022Article
- An ensemble deep learning approach to evaluate haptic delay from a single trial EEG data.Frontiers in robotics and AI · 2022Article
- Affection of facial artifacts caused by micro-expressions on electroencephalography signals.Frontiers in neuroscience · 2022Article
- Combining Wearable Devices and Mobile Surveys to Study Child and Youth Development in Malawi: Implementation Study of a Multimodal Approach.JMIR public health and surveillance · 2021Article
- Advances in Multimodal Emotion Recognition Based on Brain-Computer Interfaces.Brain sciences · 2020Review
- Systematic Analysis of a Military Wearable Device Based on a Multi-Level Fusion Framework: Research Directions.Sensors (Basel, Switzerland) · 2019Review
- Recognition of Emotions Using Multichannel EEG Data and DBN-GC-Based Ensemble Deep Learning Framework.Computational intelligence and neuroscience · 2018Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
This paper proposes two multimodal fusion methods between brain and peripheral signals for emotion recognition. The input signals are electroencephalogram and facial expression. The stimuli are based on a subset of movie clips that correspond to four specific areas of valance-arousal emotional space (happiness, neutral, sadness, and fear). For facial expression detection, four basic emotion states (happiness, neutral, sadness, and fear) are detected by a neural network classifier. For EEG detection, four basic emotion states and three emotion intensity levels (strong, ordinary, and weak) are detected by two support vector machines (SVM) classifiers, respectively. Emotion recognition is based on two decision-level fusion methods of both EEG and facial expression detections by using a sum rule or a production rule. Twenty healthy subjects attended two experiments. The results show that the accuracies of two multimodal fusion detections are 81.25% and 82.75%, respectively, which are both higher than that of facial expression (74.38%) or EEG detection (66.88%). The combination of facial expressions and EEG information for emotion recognition compensates for their defects as single information sources.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.