Automatic Essay Scoring Incorporating Rating Schema via Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Existing systems for automatic essay scoring are trained to predict the score of each essay at a time without considering rating schema. |
| Approach: | They propose a reinforcement learning framework that incorporates quadratic weighted kappa as guidance to optimize the scoring system. |
| Outcome: | Experiments on benchmark datasets show the proposed framework is effective. |
Similar Papers
Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing reinforcement learning (RL) applications in AES are limited to classification models despite associated performance degradation. |
| Approach: | They propose to integrate actual evaluation schemes into the training process by designing QWK-based rewards with a mean-squared error penalty for multi-trait AES. |
| Outcome: | The proposed scoring-aware multi-reward reinforcement learning integrates actual evaluation schemes into the training process. |
Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on sentence prediction tasks uses shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss. |
| Approach: | They propose to use a pre-trained language model to learn text representations first and then to constrain the scores with regression loss or ranking loss. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Automated Student Assessment Prize dataset. |
Multi-task Learning for Automated Essay Scoring with Sentiment Analysis (2020.aacl-srw)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) is a process that aims to alleviate the workload of graders and improve the feedback cycle in educational systems. |
| Approach: | They propose to combine two tasks, sentiment analysis and AES by utilizing multi-task learning to combine sentiment features extracted from opinion expressions. |
| Outcome: | The proposed model produces a QWK of 0.763 on the Automated StudentAssessment Prize (ASAP) benchmark. |
Automated Essay Scoring via Pairwise Contrastive Regression (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to automate essay scoring use regression or ranking objectives . a novel neural pairwise ranking model is developed to optimize both objectives based on the same loss . |
| Approach: | They propose a novel Neural Pairwise Contrastive Regression model that optimizes both objectives simultaneously as a single loss. |
| Outcome: | The proposed model outperforms previous methods on the public Automated Student Assessment Prize dataset. |
Automated Essay Scoring: A Reflection on the State of the Art (2024.emnlp-main)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is a key application of natural language processing . it is based on a holistic score that summarizes the essay's overall quality . |
| Approach: | aaron carroll: automated essay scoring is one of the most important applications in NLP . carroll says the task is still far from being solved, but it's still progressing steadily . he says it'll be interesting to see how researchers can improve performance numbers . |
| Outcome: | a new neural model can beat existing models on a standard evaluation dataset, authors say . authors: the current model is not enough to improve performance numbers . they say it could spark discussion among researchers on how to move forward . |
Automated Essay Scoring System for Nonnative Japanese Learners (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing systems only provide a holistic score that summarizes the quality of an essay, which provides little feedback for a language learner. |
| Approach: | They developed an automated essay scoring system for Japanese as a second language learners using an essay dataset with annotations for a holistic score and multiple trait scores. |
| Outcome: | The proposed system achieves the highest accuracy in various natural language processing tasks. |
Beyond the Gold Standard in Analytic Automated Essay Scoring (2025.acl-srw)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) is a new approach to assessing writing practice . traditional holistic scoring methods are not reliable and lack formative feedback in the classroom. |
| Approach: | They propose to combine analytic and holistic AES to create a system that learns from individual raters instead of gold standard labels. |
| Outcome: | The proposed system learns from individual raters instead of gold standard labels. |
Automated Chinese Essay Scoring from Multiple Traits (2022.coling-1)
Copied to clipboard
| Challenge: | Current research on AES focuses on scoring the overall quality or single trait of prompt-specific essays. |
| Approach: | They propose a hierarchical multi-task trait scorer to evaluate quality of writing . they propose an inter-sequence attention mechanism to enhance information interaction . |
| Outcome: | The proposed model outperforms several strong models on ACEA and outperformed other models. |
Analytic Automated Essay Scoring Based on Deep Neural Networks Integrating Multidimensional Item Response Theory (2022.coling-1)
Copied to clipboard
| Challenge: | Essay exams have two drawbacks in that grading them is expensive and raises questions about fairness. |
| Approach: | They propose to use a multidimensional item response theory model to improve interpretability while maintaining scoring accuracy. |
| Outcome: | The proposed model improves interpretability while maintaining accuracy while preserving cost and accuracy. |
Graph-Based Multi-Trait Essay Scoring (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on Automated Essay Scoring (AES) models essay as word sequence, but new approach uses graph-attention network approach to model essay traits. |
| Approach: | They propose a graph-attention network approach to automate essay scoring that models interactions among essay traits as a graphical graph. |
| Outcome: | The proposed approach outperforms competing approaches on the ASAP++ dataset . it allows for multiple-task scoring, allowing for more detailed feedback on essays . |