Evidence map›Paper›PMID 42185331›Full record

ArticleScientific data2026

A dataset of human and AI-generated rubric evaluations for formative programming assessment.

Pedro C Mendonça, Filipe Quintal, Karolina Baras, Fábio Mendonça

Abstract readDataset
In one paragraph

Article in Scientific data, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors.

Pedro C MendonçaFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082, Funchal, Portugal. pedro.mendonca@staff.uma.pt.
Filipe QuintalFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082, Funchal, Portugal.
Karolina BarasFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082, Funchal, Portugal.
Fábio MendonçaFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082, Funchal, Portugal.

Funding

Fundação para a Ciência e a Tecnologia 10.54499/LA/P/0083/2020; 10.54499/UIDP/50009/2020; 10.54499/UIDB/50009/2020
6 · The paper itself

Abstract

Formative assessment in education is often constrained by the time required for rubric design and manual evaluation, limiting the frequency of feedback provided to students. The HARMOGEN-R dataset was created to support research on automated rubric generation and evaluation using artificial intelligence under teacher supervision. It contains evaluations from a Data Structures and Algorithms programming course in which 770 student responses were assessed with five rubrics: one human-created and four generated by large language models (LLMs) using reasoning-enhanced structured and free-form approaches. Each response was evaluated by both human instructors and LLMs, with complete reasoning traces and generated feedback reports recorded to document assessment logic and communication with learners. The dataset also incorporates a variance-based quality control system that flags evaluations exceeding predefined consistency thresholds. This parallel evaluation design enables comparisons of rubric performance, evaluator agreement, and assessment reliability. The dataset is intended for reuse in studies of rubric generation, feedback quality, and automated formative evaluation.

Identifiers

PMID42185331
PMCPMC13482975

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.