Evidence map›Paper›PMID 30505240›Full record

ArticleProceedings of the ... IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP (Conference)2018

DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE.

Yue Gu, Shuhong Chen, Ivan Marsic

Abstract read
In one paragraph

Article in Proceedings of the ... IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP (Conference), 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers.

0numbers the graph read from it
0cells of the map it votes in
9citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

9 citing papers in PubMed.

  1. Article
  2. Multimodal fusion-powered English speaking robot.Frontiers in neurorobotics · 2024
    Article
  3. Article
  4. Article
  5. Mutual Correlation Attentive Factors in Dyadic Fusion Networks for Speech Emotion Recognition.Proceedings of the ... ACM International Conference on Multimedia, with co-located Symposium & Workshops. ACM International Conference on Multimedia · 2019
    Article
  6. Article
  7. Human Conversation Analysis Using Attentive Multimodal Networks with Hierarchical Encoder-Decoder.Proceedings of the ... ACM International Conference on Multimedia, with co-located Symposium & Workshops. ACM International Conference on Multimedia · 2018
    Article
  8. Hybrid Attention based Multimodal Network for Spoken Language Classification.Proceedings of the conference. Association for Computational Linguistics. Meeting · 2018
    Article
  9. Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment.Proceedings of the conference. Association for Computational Linguistics. Meeting · 2018
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Yue GuDepartment of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA.
Shuhong ChenDepartment of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA.
Ivan MarsicDepartment of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA.

Funding

Intention-aware Recommender System for Improving Trauma Resuscitation OutcomesR01LM011834 · NLM · CHILDREN'S RESEARCH INSTITUTE · PI BURD, RANDALL S., MARSIC, IVAN · 2014 to 2023
$4.7M
NLM NIH HHS R01 LM011834
6 · The paper itself

Abstract

In this paper, we present a novel deep multimodal framework to predict human emotions based on sentence-level spoken language. Our architecture has two distinctive characteristics. First, it extracts the high-level features from both text and audio via a hybrid deep multimodal structure, which considers the spatial information from text, temporal information from audio, and high-level associations from low-level handcrafted features. Second, we fuse all features by using a three-layer deep neural network to learn the correlations across modalities and train the feature extraction and fusion modules together, allowing optimal global fine-tuning of the entire structure. We evaluated the proposed framework on the IEMOCAP dataset. Our result shows promising performance, achieving 60.4% in weighted accuracy for five emotion categories.

Indexed as

deep multimodal learningEmotion recognitionspoken language

Identifiers

PMID30505240
PMCPMC6261381

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.