ArticleSensors (Basel, Switzerland)2021
Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning.
Article in Sensors (Basel, Switzerland), 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Emotion recognition and human-robot interaction: a systematic review of methods and applications.Frontiers in robotics and AI · 2026Pooled it
- Multimodal emotion recognition using hybrid deep feature fusion under speaker-independent evaluation.Scientific reports · 2026Article
- eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio-Visual Data.Biomimetics (Basel, Switzerland) · 2026Article
- Deep learning-based bimodal speech and facial expression recognition of miners' unsafe emotions.PloS one · 2026Article
- Emotion recognition from body movement through interpretable motion-aware sequential modeling.Frontiers in artificial intelligence · 2026Article
- Article
- Emerging trends and clinical challenges in AI-enhanced emotion diagnosis using physiological data.Medical & biological engineering & computing · 2026Review
- Spatiotemporal multimodal emotion recognition using Temporal video sequences and pose features for child emotion classification.Scientific reports · 2025Article
- Multimodal Sensing-Enabled Large Language Models for Automated Emotional Regulation: A Review of Current Technologies, Opportunities, and Challenges.Sensors (Basel, Switzerland) · 2025Review
- Article
- Transfer learning-based English translation text classification in a multimedia network environment.PeerJ. Computer science · 2024Article
- Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features.Sensors (Basel, Switzerland) · 2023Article
- IoT-Enabled WBAN and Machine Learning for Speech Emotion Recognition in Patients.Sensors (Basel, Switzerland) · 2023Article
- Facial Expression Recognition Robust to Occlusion and to Intra-Similarity Problem Using Relevant Subsampling.Sensors (Basel, Switzerland) · 2023Article
- Audio-Visual Stress Classification Using Cascaded RNN-LSTM Networks.Bioengineering (Basel, Switzerland) · 2022Article
- Emotional Speech Recognition Method Based on Word Transcription.Sensors (Basel, Switzerland) · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
Emotion Recognition is attracting the attention of the research community due to the multiple areas where it can be applied, such as in healthcare or in road safety systems. In this paper, we propose a multimodal emotion recognition system that relies on speech and facial information. For the speech-based modality, we evaluated several transfer-learning techniques, more specifically, embedding extraction and Fine-Tuning. The best accuracy results were achieved when we fine-tuned the CNN-14 of the PANNs framework, confirming that the training was more robust when it did not start from scratch and the tasks were similar. Regarding the facial emotion recognizers, we propose a framework that consists of a pre-trained Spatial Transformer Network on saliency maps and facial images followed by a bi-LSTM with an attention mechanism. The error analysis reported that the frame-based systems could present some problems when they were used directly to solve a video-based task despite the domain adaptation, which opens a new line of research to discover new ways to correct this mismatch and take advantage of the embedded knowledge of these pre-trained models. Finally, from the combination of these two modalities with a late fusion strategy, we achieved 80.08% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. The results revealed that these modalities carry relevant information to detect users' emotional state and their combination enables improvement of system performance.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.