Evidence map›Paper›PMID 37707884›Full record

ArticleJMIR medical education2023

The Potential and Concerns of Using AI in Scientific Research: ChatGPT Performance Evaluation.

Zuheir N Khlaif, Allam Mousa, Muayad Kamal Hattab, Jamil Itmazi, Amjad A Hassan, Mageswaran Sanmugam, Abedalkarim Ayyoub

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in JMIR medical education, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 42 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
42citing papers in PubMed, 5 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

42 citing papers in PubMed, 5 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Review
  13. Article
  14. Advances in laboratory medicine · 2025
    Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Zuheir N Khlaif *Faculty of Humanities and Educational Sciences, An-Najah National University, Nablus, Occupied Palestinian Territory.ORCID https://orcid.org/0000-0002-7354-7512
Allam Mousa *Artificial Intelligence and Virtual Reality Research Center, Department of Electrical and Computer Engineering, An Najah National University, Nablus, Occupied Palestinian Territory.ORCID https://orcid.org/0000-0001-6413-1622
Muayad Kamal Hattab *Faculty of Law and Political Sciences, An-Najah National University, Nablus, Occupied Palestinian Territory.ORCID https://orcid.org/0000-0002-1096-1839
Jamil Itmazi *Department of Information Technology, College of Engineering and Information Technology, Palestine Ahliya University, Bethlahem, Occupied Palestinian Territory.ORCID https://orcid.org/0000-0002-4206-7989
Amjad A Hassan *Faculty of Law and Political Sciences, An-Najah National University, Nablus, Occupied Palestinian Territory.ORCID https://orcid.org/0009-0009-6917-4782
Mageswaran Sanmugam *Centre for Instructional Technology and Multimedia, Universiti Sains Malaysia, Penang, Malaysia.ORCID https://orcid.org/0000-0003-3313-4462
Abedalkarim Ayyoub *Faculty of Humanities and Educational Sciences, An-Najah National University, Nablus, Occupied Palestinian Territory.ORCID https://orcid.org/0000-0001-9111-4465

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundArtificial intelligence (AI) has many applications in various aspects of our daily life, including health, criminal, education, civil, business, and liability law. One aspect of AI that has gained significant attention is natural language processing (NLP), which refers to the ability of computers to understand and generate human language.

objectiveThis study aims to examine the potential for, and concerns of, using AI in scientific research. For this purpose, high-impact research articles were generated by analyzing the quality of reports generated by ChatGPT and assessing the application's impact on the research framework, data analysis, and the literature review. The study also explored concerns around ownership and the integrity of research when using AI-generated text.

methodsA total of 4 articles were generated using ChatGPT, and thereafter evaluated by 23 reviewers. The researchers developed an evaluation form to assess the quality of the articles generated. Additionally, 50 abstracts were generated using ChatGPT and their quality was evaluated. The data were subjected to ANOVA and thematic analysis to analyze the qualitative data provided by the reviewers.

resultsWhen using detailed prompts and providing the context of the study, ChatGPT would generate high-quality research that could be published in high-impact journals. However, ChatGPT had a minor impact on developing the research framework and data analysis. The primary area needing improvement was the development of the literature review. Moreover, reviewers expressed concerns around ownership and the integrity of the research when using AI-generated text. Nonetheless, ChatGPT has a strong potential to increase human productivity in research and can be used in academic writing.

conclusionsAI-generated text has the potential to improve the quality of high-impact research articles. The findings of this study suggest that decision makers and researchers should focus more on the methodology part of the research, which includes research design, developing research tools, and analyzing data in depth, to draw strong theoretical and practical implications, thereby establishing a revolution in scientific research in the era of AI. The practical implications of this study can be used in different fields such as medical education to deliver materials to develop the basic competencies for both medicine students and faculty members.

Indexed as

AIartificial intelligenceChatGPTresearch ethicsscientific research

Identifiers

PMID37707884
PMCPMC10636627

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.