ArticleSensors (Basel, Switzerland)2023
Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features.
Article in Sensors (Basel, Switzerland), 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed.
- Multimodal emotion recognition using hybrid deep feature fusion under speaker-independent evaluation.Scientific reports · 2026Article
- Comprehensive survey on multimodal emotion detection with advances, challenges, and future directions in pairwise conversations.Cognitive processing · 2026Review
- Enhanced AlexNet with Gabor and Local Binary Pattern Features for Improved Facial Emotion Recognition.Sensors (Basel, Switzerland) · 2025Article
- UMEDNet: a multimodal approach for emotion detection in the Urdu language.PeerJ. Computer science · 2025Article
- Real-Time Emotion Recognition for Improving the Teaching-Learning Process: A Scoping Review.Journal of imaging · 2024Article
- Real-Time Fatigue Detection Algorithms Using Machine Learning for Yawning and Eye State.Sensors (Basel, Switzerland) · 2024Article
- AVaTER: Fusing Audio, Visual, and Textual Modalities Using Cross-Modal Attention for Emotion Recognition.Sensors (Basel, Switzerland) · 2024Article
- Digitalized therapy and the unresolved gap between artificial and human empathy.Frontiers in psychiatry · 2024Article
- Contextual emotion detection in images using deep learning.Frontiers in artificial intelligence · 2024Article
- Audio-Visual Fusion Based on Interactive Attention for Person Verification.Sensors (Basel, Switzerland) · 2023Article
- New Approach for Generating Synthetic Medical Data to Predict Type 2 Diabetes.Bioengineering (Basel, Switzerland) · 2023Article
- Fire Detection and Notification Method in Ship Areas Using Deep Learning and Computer Vision Approaches.Sensors (Basel, Switzerland) · 2023Article
- Enhancing Speech Emotion Recognition Using Dual Feature Extraction Encoders.Sensors (Basel, Switzerland) · 2023Article
- Real-Time Deep Learning-Based Drowsiness Detection: Leveraging Computer-Vision and Eye-Blink Analyses for Enhanced Road Safety.Sensors (Basel, Switzerland) · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Methods for detecting emotions that employ many modalities at the same time have been found to be more accurate and resilient than those that rely on a single sense. This is due to the fact that sentiments may be conveyed in a wide range of modalities, each of which offers a different and complementary window into the thoughts and emotions of the speaker. In this way, a more complete picture of a person's emotional state may emerge through the fusion and analysis of data from several modalities. The research suggests a new attention-based approach to multimodal emotion recognition. This technique integrates facial and speech features that have been extracted by independent encoders in order to pick the aspects that are the most informative. It increases the system's accuracy by processing speech and facial features of various sizes and focuses on the most useful bits of input. A more comprehensive representation of facial expressions is extracted by the use of both low- and high-level facial features. These modalities are combined using a fusion network to create a multimodal feature vector which is then fed to a classification layer for emotion recognition. The developed system is evaluated on two datasets, IEMOCAP and CMU-MOSEI, and shows superior performance compared to existing models, achieving a weighted accuracy WA of 74.6% and an F1 score of 66.1% on the IEMOCAP dataset and a WA of 80.7% and F1 score of 73.7% on the CMU-MOSEI dataset.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.