Context-aware KazRoBERTa with transformer-based candidate re-ranking for Kazakh spelling correction.

Automatic spelling correction for Kazakh is difficult because productive morphology, vowel harmony, and Cyrillic keyboard substitutions generate many orthographically plausible alternatives. This study proposes Context-Aware KazRoBERTa, a three-stage framework that combines token-level error detection, linguistically constrained candidate generation, and masked-language-model candidate re-ranking with a shared Transformer encoder. Candidate generation integrates inverse ka2ru mapping, vowel-harmony restoration, Optimal String Alignment distance, and lexicon filtering; contextual probability, orthographic similarity, and lexical frequency are then combined for selection. Evaluation was performed on a synthetic corpus of 2,615,771 parallel sentence pairs derived from 922,798 clean sentences. On the held-out synthetic test set, the framework achieved 97.7% detection F1-score and 98.8% detection accuracy, while sentence correction accuracy reached 85.8%, word correction accuracy reached 95.6%, and character error rate decreased to 1.86%. The 95% Wilson interval for sentence correction accuracy was 85.36-86.23%. Relative to the reported two-stage baseline, the 5.6-point sentence-accuracy margin implies a conservative continuity-corrected McNemar lower bound of χ