Papers with MCScript
BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations (D19-60)
Copied to clipboard
| Challenge: | Experimental results show that our system achieves significant improvements over the baseline systems with 84.2% accuracy on the official test dataset. |
| Approach: | They propose a system to inject more external knowledge into everyday narrations . they use a pre-trained BERT model to fine-tune on a machine reading comprehension dataset . |
| Outcome: | The proposed system achieves significant improvements over baseline systems with 84.2% accuracy on the official test dataset. |
ABSEval: An Agent-based Framework for Script Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on script evaluation of large language models (LLMs) have not evaluated scripts generated by LLMs due to their logical structure, sequential organization, and open-ended nature. |
| Approach: | They propose to use a script evaluation dataset to evaluate LLM scripts . they propose to develop an agent-based script evaluation framework ABSEval to evaluate scripts. |
| Outcome: | The proposed framework provides superior accuracy and relevance, aligning closely with human evaluation. |