Evidence map›Paper›PMID 36585439›Full record

ArticleScientific reports2022

Explainable automated recognition of emotional states from canine facial expressions: the case of positive anticipation and frustration.

Tali Boneh-Shitrit, Marcelo Feighelstein, Annika Bremhorst, Shir Amir, Tomer Distelfeld, Yaniv Dassa, Sharon Yaroshetsky, Stefanie Riemer, Ilan Shimshoni, Daniel S Mills and 1 more

Open access · goldAbstract read
In one paragraph

Article in Scientific reports, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
22citing papers in PubMed, 1 pooled it
8.8field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

22 citing papers in PubMed, 1 synthesis or guideline pooled it, 50 citations in OpenAlex.

  1. Pooled it
  2. Review
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors at 6 institutions in 3 countries.

Tali Boneh-Shitrit *Computer Science Department, Open University, Raanana, Israel.
Marcelo Feighelstein *Information Systems Department, University of Haifa, Haifa, Israel.
Annika Bremhorst *Dogs & Science-Institute for Canine Science and Applied Cynology, Zurich, Switzerland.
Shir AmirComputer Science Department, Weizmann Institute, Rehovot, Israel.
Tomer DistelfeldFaculty of Electrical Engineering, Technion, Israel Institute of Technology, Haifa, Israel.
Yaniv DassaFaculty of Electrical Engineering, Technion, Israel Institute of Technology, Haifa, Israel.
Sharon YaroshetskyPrimrose, Tel Aviv, Israel.
Stefanie RiemerUniversity of Bern, Bern, Switzerland.
Ilan ShimshoniInformation Systems Department, University of Haifa, Haifa, Israel.
Daniel S MillsJoseph Banks Laboratories, Department of Life Sciences, University of Lincoln, Lincoln, UK.
Anna ZamanskyInformation Systems Department, University of Haifa, Haifa, Israel. annazam@is.haifa.ac.il.
University of Haifa · ILTechnion – Israel Institute of Technology · ILOpen University of Israel · ILUniversity of Bern · CHUniversity of Lincoln · GBWeizmann Institute of Science · IL

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

In animal research, automation of affective states recognition has so far mainly addressed pain in a few species. Emotional states remain uncharted territories, especially in dogs, due to the complexity of their facial morphology and expressions. This study contributes to fill this gap in two aspects. First, it is the first to address dog emotional states using a dataset obtained in a controlled experimental setting, including videos from (n = 29) Labrador Retrievers assumed to be in two experimentally induced emotional states: negative (frustration) and positive (anticipation). The dogs' facial expressions were measured using the Dogs Facial Action Coding System (DogFACS). Two different approaches are compared in relation to our aim: (1) a DogFACS-based approach with a two-step pipeline consisting of (i) a DogFACS variable detector and (ii) a positive/negative state Decision Tree classifier; (2) An approach using deep learning techniques with no intermediate representation. The approaches reach accuracy of above 71% and 89%, respectively, with the deep learning approach performing better. Secondly, this study is also the first to study explainability of AI models in the context of emotion in animals. The DogFACS-based approach provides decision trees, that is a mathematical representation which reflects previous findings by human experts in relation to certain facial expressions (DogFACS variables) being correlates of specific emotional states. The deep learning approach offers a different, visual form of explainability in the form of heatmaps reflecting regions of focus of the network's attention, which in some cases show focus clearly related to the nature of particular DogFACS variables. These heatmaps may hold the key to novel insights on the sensitivity of the network to nuanced pixel patterns reflecting information invisible to the human eye.

Indexed as

Facial RecognitionFrustrationAnimalsAttentionDogsEmotionsFacial ExpressionHumansRecognition, Psychology

Identifiers

PMID36585439
PMCPMC9803655
OpenAlexW4313454936

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.