- Identifier
- arXiv:2211.05166
- Published
- Apr 29, 2023
- Authors
- 6
- Type
- Survey
- Area
- Practical evaluations
In one paragraph
A comprehensive survey of grammatical error correction. It covers the linguistic challenges, datasets in English and other languages, modeling approaches including synthetic error generation, and how far evaluation metrics can be trusted.
- published in Computational Linguistics, 2023
- CL 49(3)
- published in Computational Linguistics, 2023
FIG. 02: Mechanism
How it works
Classify the errors
Errors are replacements, omissions or insertions, subdivided by part of speech and morphology. ERRANT standardises them into 25 main types.
Collect annotated data
Benchmarks such as CoNLL-2014, BEA-2019 and JFLEG pair learner sentences with corrections. The largest clean public training sets hold roughly 50k sentence pairs.
Generate artificial errors
To get more training data, clean text is corrupted with rules, with error patterns taken from learner corpora, or with back-translation and round-trip translation.
Correct as translation
Most current systems treat correction as translating errorful text into corrected text. Edit-based models that tag each token are the main alternative.
Score the edits
System edits are matched against reference edits and summarised as F0.5, which weights precision twice as much as recall. Metrics disagree with each other and with human judgements, so the survey recommends a fixed metric for each test set.
FIG. 03: Interactive figure
Accept an edit, watch F0.5 move
Toggle a system's corrections to see precision, recall and F0.5 change, then watch a valid correction score zero against a single reference.
Loading interactive demo…
FIG. 04: Notes
Summary
Grammatical error correction (GEC) is the task of automatically detecting and correcting errors in text. This survey, published in Computational Linguistics, covers:
- the linguistic challenges that make GEC hard
- the main datasets for English and other languages
- modeling approaches, from rule-based systems to neural machine translation, which is now dominant
- data augmentation through artificial error generation
- evaluation, with a close look at metric reliability and agreement with human judgments
Reference
@article{bryant2023grammatical,
title={Grammatical Error Correction: A Survey of the State of the Art},
author={Christopher Bryant and Zheng Yuan and Muhammad Reza Qorib and Hannan Cao and Hwee Tou Ng and Ted Briscoe},
journal={Computational Linguistics},
volume={49},
number={3},
pages={643--701},
year={2023},
doi={10.1162/coli_a_00478}
}