Fighting Evaluation Inflation: Concentrated Datasets for Grammatical Error Correction Task - Vladimir Starchenko; Darya Kharlamova; Elizaveta Klykova; Anastasia Shavrina; Aleksey Starchenko; Olga Vinogradova; Olga Lyashevskaya (Journal of Language and Education)
Background: Grammatical error correction (GEC) systems have greatly developed over the recent decade. According to common metrics, they often reach the level of or surpass human experts. Nevertheless, they perform poorly on several kinds of errors that are effortlessly corrected by humans. Thus, reaching the resolution limit, evaluation algorithms and datasets do not allow for further enhancement of GEC systems.