ArticleEducational and psychological measurement2019
Exploring the Combined Effects of Rater Misfit and Differential Rater Functioning in Performance Assessments.
Article in Educational and psychological measurement, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed.
- Linking essay-writing tests using many-facet models and neural automated essay scoring.Behavior research methods · 2024Article
- Measuring Clinical Preparedness After Residency Training: Development of a New Instrument.Family medicine · 2024Review
- Article
- A Bayesian many-facet Rasch model with Markov modeling for rater severity drift.Behavior research methods · 2023Article
- Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model.Educational and psychological measurement · 2023Article
- The Effects of Rating Designs on Rater Classification Accuracy and Rater Measurement Precision in Large-Scale Mixed-Format Assessments.Applied psychological measurement · 2023Article
- Detecting Differential Rater Functioning in Severity and Centrality: The Dual DRF Facets Model.Educational and psychological measurement · 2022Article
- Academic performance under COVID-19: The role of online learning readiness and emotional competence.Current psychology (New Brunswick, N.J.) · 2022Article
- Detecting Rater Biases in Sparse Rater-Mediated Assessment Networks.Educational and psychological measurement · 2021Article
- Accuracy of performance-test linking based on a many-facet Rasch model.Behavior research methods · 2021Article
- An Iterative Parametric Bootstrap Approach to Evaluating Rater Fit.Applied psychological measurement · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Rater effects, or raters' tendencies to assign ratings to performances that are different from the ratings that the performances warranted, are well documented in rater-mediated assessments across a variety of disciplines. In many real-data studies of rater effects, researchers have reported that raters exhibit more than one effect, such as a combination of misfit and systematic biases related to student subgroups (i.e., differential rater functioning [DRF]). However, researchers who conduct simulation studies of rater effects usually focus on the effects in isolation. The purpose of this study was to explore the degree to which rater effect indicators are sensitive to rater effects when raters exhibit more than one type of effect, and to explore the degree to which this sensitivity changes under different data collection designs. We used a simulation study to explore combinations of DRF and rater misfit. Overall, our findings suggested that it is possible to use common numeric and graphical indicators of DRF and rater misfit when raters exhibit both these effects, but that these effects may be difficult to distinguish using only numeric indicators. We also observed that combinations of rater effects are easier to identify when complete rating designs are used. We discuss implications of our findings as they result to research and practice.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.