GEC survey

Grammatical Error Correction: A Survey of the State of the Art

Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng, Ted Briscoe

FIG. 01 Classify the errors → Collect annotated data → Generate artificial errors → Correct as translation → Score the edits
Identifier
arXiv:2211.05166
Published
Apr 29, 2023
Authors
6
Type
Survey
Area
Practical evaluations

In one paragraph

A comprehensive survey of grammatical error correction. It covers the linguistic challenges, datasets in English and other languages, modeling approaches including synthetic error generation, and how far evaluation metrics can be trusted.

published in Computational Linguistics, 2023
CL 49(3)
published in Computational Linguistics, 2023

FIG. 02: Mechanism

How it works

  1. Classify the errors

    Errors are replacements, omissions or insertions, subdivided by part of speech and morphology. ERRANT standardises them into 25 main types.

  2. Collect annotated data

    Benchmarks such as CoNLL-2014, BEA-2019 and JFLEG pair learner sentences with corrections. The largest clean public training sets hold roughly 50k sentence pairs.

  3. Generate artificial errors

    To get more training data, clean text is corrupted with rules, with error patterns taken from learner corpora, or with back-translation and round-trip translation.

  4. Correct as translation

    Most current systems treat correction as translating errorful text into corrected text. Edit-based models that tag each token are the main alternative.

  5. Score the edits

    System edits are matched against reference edits and summarised as F0.5, which weights precision twice as much as recall. Metrics disagree with each other and with human judgements, so the survey recommends a fixed metric for each test set.

FIG. 03: Interactive figure

Accept an edit, watch F0.5 move

Toggle a system's corrections to see precision, recall and F0.5 change, then watch a valid correction score zero against a single reference.

Loading interactive demo…

FIG. 04: Notes

Summary

Grammatical error correction (GEC) is the task of automatically detecting and correcting errors in text. This survey, published in Computational Linguistics, covers:

  • the linguistic challenges that make GEC hard
  • the main datasets for English and other languages
  • modeling approaches, from rule-based systems to neural machine translation, which is now dominant
  • data augmentation through artificial error generation
  • evaluation, with a close look at metric reliability and agreement with human judgments

Reference

Cite this work
@article{bryant2023grammatical,
  title={Grammatical Error Correction: A Survey of the State of the Art},
  author={Christopher Bryant and Zheng Yuan and Muhammad Reza Qorib and Hannan Cao and Hwee Tou Ng and Ted Briscoe},
  journal={Computational Linguistics},
  volume={49},
  number={3},
  pages={643--701},
  year={2023},
  doi={10.1162/coli_a_00478}
}