TCFLE-8: a Corpus of Learner Written Productions for French as a Foreign Language and its Application to Automated Essay Scoring (2023.emnlp-main)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) aims to automatically assess the quality of essays. |
| Approach: | They propose to use a corpus of 6.5k essays collected in the context of the Test de Connaissance du Français (TCF) certification exam to foster the development of AES for French. |
| Outcome: | The proposed system can assess the quality of essays in a language certification exam using a corpus of 6.5k essays collected in the TCFLE-8 exam. |
Similar Papers
Language Proficiency Scoring (2020.lrec-1)
Copied to clipboard
| Challenge: | a new paper evaluates and extends the results of an automated proficiency classification system for different languages. |
| Approach: | They propose to extend an automated essay scoring system proposed by CEFR . they compare results with those from previous paper and add a new corpus for english . |
| Outcome: | The proposed approach does not scale well with the added English corpus. |
Beyond the Gold Standard in Analytic Automated Essay Scoring (2025.acl-srw)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) is a new approach to assessing writing practice . traditional holistic scoring methods are not reliable and lack formative feedback in the classroom. |
| Approach: | They propose to combine analytic and holistic AES to create a system that learns from individual raters instead of gold standard labels. |
| Outcome: | The proposed system learns from individual raters instead of gold standard labels. |
LEAF: Language Learners’ English Essays and Feedback Corpus (2024.naacl-short)
Copied to clipboard
| Challenge: | Current automated essay scoring models lack the granularity desired by learners and instructors seeking more detailed insights. |
| Approach: | They present a corpus of English essays and their corresponding feedback from the “essayforum” website. |
| Outcome: | The LEAF corpus provides valuable feedback for students and teachers . it provides insights on argumentative aspects and organizational coherence . |
Automated Essay Scoring: A Reflection on the State of the Art (2024.emnlp-main)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is a key application of natural language processing . it is based on a holistic score that summarizes the essay's overall quality . |
| Approach: | aaron carroll: automated essay scoring is one of the most important applications in NLP . carroll says the task is still far from being solved, but it's still progressing steadily . he says it'll be interesting to see how researchers can improve performance numbers . |
| Outcome: | a new neural model can beat existing models on a standard evaluation dataset, authors say . authors: the current model is not enough to improve performance numbers . they say it could spark discussion among researchers on how to move forward . |
Can Large Language Models Automatically Score Proficiency of Written Essays? (2024.lrec-main)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is one of the earliest research problems in natural language processing. |
| Approach: | They propose to use large language models to analyze and score written essays using four different prompts. |
| Outcome: | The proposed models show comparable performance on four different prompts and a slight advantage over the state-of-the-art models. |
Beyond the Score: Uncertainty-Calibrated LLMs for Automated Essay Assessment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) systems attain near–human agreement on some public benchmarks, but real-world adoption is limited. |
| Approach: | They propose a distribution-free wrapper that equips any classifier with set-valued outputs enjoying formal coverage guarantees. |
| Outcome: | The proposed model achieves coverage targets while keeping prediction sets compact. |
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
Jiamin Su, Yibo Yan, Fangteng Fu, Zhang Han, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu
| Challenge: | Automated Essay Scoring (AES) systems face three major challenges: reliance on handcrafted features that limit generalizability, difficulty in capturing fine-grained traits like coherence and argumentation, and inability to handle multimodal contexts. |
| Approach: | They propose a multimodal benchmark to evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
| Outcome: | The proposed system can evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on sentence prediction tasks uses shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss. |
| Approach: | They propose to use a pre-trained language model to learn text representations first and then to constrain the scores with regression loss or ranking loss. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Automated Student Assessment Prize dataset. |
REPROLANG 2020: Automatic Proficiency Scoring of Czech, English, German, Italian, and Spanish Learner Essays (2020.lrec-1)
Copied to clipboard
| Challenge: | a new paper aims to reproduce the work described in Vajjala & Rama (2018) . the paper focuses on features-based and neural approaches to essay scoring in Czech, German and Italian . |
| Approach: | They propose to replicate the work described in Vajjala & Rama 2018, ‘Experiments with universal CEFR classification’, as part of REPROLANG 2020. |
| Outcome: | The proposed methods perform better than feature-based models for large text datasets, though neural network modifications do bring performance closer to the best feature-driven models. |
Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing Automated Essay Scoring (AES) methods focus on sentence-level features, whereas Large Language Models (LLMs) are sensitive to conventions & accuracy, language complexity, and organization. |
| Approach: | They propose to use large language models to aid in decision-making . they propose to analyze the reasoning of neural models by analyzing sentence-level features. |
| Outcome: | The proposed method improves understanding of neural approaches to Automated Essay Scoring (AES) and can also apply to other domains seeking transparency in model-driven decisions. |