ArticlePLoS computational biology2023
Evaluating a large language model's ability to solve programming exercises from an introductory bioinformatics course.
Article in PLoS computational biology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- A pretrained biomedical large language model for Persian biomedical text mining.Scientific reports · 2026Article
- Teaching artificial intelligence through drug-drug interaction clustering analysis: Integrating project-based learning and large language models.PLoS computational biology · 2026Article
- Understanding practical barriers to the global adoption of environmental DNA (eDNA) methods, tools, and standards.BMC research notes · 2026Article
- Benchmarking large language models for identifying transcription factor regulatory interactions.Bioinformatics (Oxford, England) · 2026Article
- REDAC: RNA-seq expression data analysis chatbot.Bioinformatics advances · 2026Article
- BioAgents: Bridging the gap in bioinformatics analysis with multi-agent systems.Scientific reports · 2025Article
- ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research.Journal of medical Internet research · 2025Article
- Translating short-form Python exercises to other programming languages using diverse prompting strategies.GigaScience · 2025Article
- Opportunities and considerations for using artificial intelligence in bioinformatics education.Bioinformatics advances · 2025Article
- Multiple Confabulations Found in Bioinformatics Tasks Carried Out by Several Free Large Language Models.Current genomics · 2025Article
- Article
- Bioinformatics and biomedical informatics with ChatGPT: Year one review.Quantitative biology (Beijing, China) · 2024Article
- Steering veridical large language model analyses by correcting and enriching generated database queries: first steps toward ChatGPT bioinformatics.Briefings in bioinformatics · 2024Article
- An AI Agent for Fully Automated Multi-Omic Analyses.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2024Article
- The transformative impact of large language models on medical writing and publishing: current applications, challenges and future directions.The Korean journal of physiology & pharmacology : official journal of the Korean Physiological Society and the Korean Society of Pharmacology · 2024Review
- Integrating Large Language Models in Bioinformatics Education for Medical Students: Opportunities and Challenges.Annals of biomedical engineering · 2024Article
- Article
- Data science through natural language with ChatGPT's Code Interpreter.Translational and clinical pharmacology · 2024Article
- On the Responsible Use of Chatbots in Bioinformatics.Genomics, proteomics & bioinformatics · 2024Article
- Scientific figures interpreted by ChatGPT: strengths in plot recognition and limits in color perception.NPJ precision oncology · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Computer programming is a fundamental tool for life scientists, allowing them to carry out essential research tasks. However, despite various educational efforts, learning to write code can be a challenging endeavor for students and researchers in life-sciences disciplines. Recent advances in artificial intelligence have made it possible to translate human-language prompts to functional code, raising questions about whether these technologies can aid (or replace) life scientists' efforts to write code. Using 184 programming exercises from an introductory-bioinformatics course, we evaluated the extent to which one such tool-OpenAI's ChatGPT-could successfully complete programming tasks. ChatGPT solved 139 (75.5%) of the exercises on its first attempt. For the remaining exercises, we provided natural-language feedback to the model, prompting it to try different approaches. Within 7 or fewer attempts, ChatGPT solved 179 (97.3%) of the exercises. These findings have implications for life-sciences education and research. Instructors may need to adapt their pedagogical approaches and assessment techniques to account for these new capabilities that are available to the general public. For some programming tasks, researchers may be able to work in collaboration with machine-learning models to produce functional code.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.