Evidence map›Paper›PMID 40498658›Full record

ArticleJMIR infodemiology2025

Assessing the Reliability and Validity of Principles for Health-Related Information on Social Media (PRHISM) for Evaluating Breast Cancer Treatment Videos on YouTube: Instrument Validation Study.

Hiroki Kusama, Yoshimitsu Takahashi, Shunichiro Orihara, Kayo Adachi, Yumiko Ishizuka, Ryoko Semba, Hidetaka Shima, Yoshiya Horimoto, Hiroshi Kaise, Masataka Taguri and 3 more

Abstract readValidation Study
PubMed Full text at PMC Publisher
In one paragraph

Article in JMIR infodemiology, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

2 citing papers in PubMed.

  1. Review
  2. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

13 authors.

Hiroki KusamaDepartment of Breast Surgical Oncology, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, 160-0023, Japan, 81 3-3342-6111.ORCID 0000-0002-2163-7989
Yoshimitsu TakahashiDepartment of Implementation Science in Public Health, School of Public Health, Kyoto University, Kyoto, Japan.ORCID 0000-0003-4073-9945
Shunichiro OriharaDepartment of Health Data Science, Tokyo Medical University Hospital, Tokyo, Japan.ORCID 0000-0003-0168-1250
Kayo AdachiDepartment of Breast Surgical Oncology, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, 160-0023, Japan, 81 3-3342-6111.ORCID 0000-0002-1425-2271
Yumiko IshizukaDepartment of Breast Oncology, Faculty of Medicine, Juntendo University, Tokyo, Japan.ORCID 0000-0002-1588-3067
Ryoko SembaDepartment of Breast Oncology, Faculty of Medicine, Juntendo University, Tokyo, Japan.ORCID 0009-0007-8670-4357
Hidetaka ShimaDepartment of Breast Surgery, Douaikai Ozawa Hospital, Kanagawa, Japan.ORCID 0000-0002-4913-1648
Yoshiya HorimotoDepartment of Breast Surgical Oncology, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, 160-0023, Japan, 81 3-3342-6111.ORCID 0000-0001-8935-0768
Hiroshi KaiseDepartment of Breast Surgical Oncology, Tokyo Medical University Ibaraki Medical Center, Ibaraki, Japan.ORCID 0000-0002-6638-783X
Masataka TaguriDepartment of Health Data Science, Tokyo Medical University Hospital, Tokyo, Japan.ORCID 0000-0001-8902-0056
Sho InoueYCU Co-Creation Innovation Center, Yokohama City University, Kanagawa, Japan.ORCID 0009-0003-2434-9899
Takeo NakayamaDepartment of Health Informatics, School of Public Health, Kyoto University, Kyoto, Japan.ORCID 0000-0002-7918-6252
Takashi IshikawaDepartment of Breast Surgical Oncology, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, 160-0023, Japan, 81 3-3342-6111.ORCID 0000-0003-3450-4533

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Background: There is breast cancer-related medical information on social media, but there is no established method for objectively evaluating the quality of this information. Principles for Health-Related Information on Social Media (PRHISM) is a newly developed tool for objectively assessing the quality of health-related information on social media; however, there have been no reports evaluating its reliability and validity. Objective: The purpose of this study was to statistically examine the reliability and validity of PRHISM using videos about breast cancer treatment on YouTube (Google). Methods: In total, 60 YouTube videos were selected on January 5, 2024, with the Japanese words for "breast cancer," "treatment," and "chemotherapy," and assessed by 6 Japanese physicians with expertise in breast cancer. These evaluators independently evaluated the videos using PRHISM and an established tool for assessing the quality of health-related information, DISCERN, as well as through subjective assessments. We calculated interrater and intrarater agreement among evaluators with CIs, measuring agreement using weighted Cohen kappa. Results: The interrater agreement for PRHISM overall quality was κ=0.52 (90% CI 0.49-0.55), indicating that the expected level of agreement, statistically defined by the lower limit of the 90% CI exceeding 0.53, was not achieved. However, PRHISM demonstrated higher agreement compared with DISCERN overall quality, which had a κ=0.45 (90% CI 0.41-0.48). In terms of validity, the intrarater agreement between PRHISM and subjective assessments by breast experts was κ=0.37 (95% CI 0.14-0.60), while DISCERN showed an agreement of κ=0.27 (95% CI 0.07-0.48), indicating fair agreement and no significant difference in validity. Conclusions: PRHISM has demonstrated sufficient reliability and validity for evaluating the quality of health-related information on YouTube, making it a promising new metric. To further enhance objectivity, it is necessary to explore the use of artificial intelligence and other approaches.

Indexed as

Breast NeoplasmsConsumer Health InformationSocial MediaVideo RecordingFemaleHumansJapanReproducibility of Resultsbreast cancer treatmentcancer treatmentinformation qualityinstrument validation studyJapanmedical informationonline health informationPRHISMPrinciples for Health-Related Information on Social Mediareliabilitysocial mediavalidityvideosYouTube

Identifiers

PMID40498658
PMCPMC12175871

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.