PictureStories: Predicting the Task Adherence of Language Learner Answers to a Picture Story-Based Writing Task (2026.eacl-long)
Copied to clipboard
| Challenge: | a lack of suitable training and evaluation data limits the evaluation of language learning tasks to language proficiency only. |
| Approach: | They develop a marking rubric that covers task adherence with respect to form and content. |
| Outcome: | The proposed model can predict the adherence of learners to written tasks using picture stories. |
Similar Papers
Analyzing Text Representations by Measuring Task Alignment (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in text classification have shown that pre-trained representations are key for text classification. |
| Approach: | They propose a task alignment score that measures alignment at different levels of granularity. |
| Outcome: | The proposed score shows that task alignment can explain the performance of a given representation. |
Image Description Dataset for Language Learners (2022.lrec-1)
Copied to clipboard
| Challenge: | Language learners are limited by the number of texts or speech they are asked to answer . automatic assessment of image descriptions requires a system that depends on both the learner's native language and the target language. |
| Approach: | They propose a dataset that consists of images, their descriptions, and assessment annotations . they propose 'automatic error correction' task that encodes multimodal information from a learner sentence with an image and accurately decodes a corrected sentence. |
| Outcome: | The proposed model can revise errors that cannot be revised without an image. |
Can language models learn from explanations in context? (2022.findings-emnlp)
Copied to clipboard
Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, Felix Hill
| Challenge: | Language Models can adapt to a few in-context examples, but without training. |
| Approach: | They examine how explanations of few-shot examples can help Language Models (LMs) explanations can improve performance even without tuning, they find . |
| Outcome: | The proposed explanations outperform hand-tuned explanations on small validation sets. |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
It is not a piece of cake for GPT: Explaining Textual Entailment Recognition in the presence of Figurative Language (2025.coling-main)
Copied to clipboard
| Challenge: | Figure-based language is used to convey opinions, ideas, or emotions in texts and dialogues. |
| Approach: | They evaluate the capabilities of Large Language Models to address TER and generate textual explanations of TER predictions. |
| Outcome: | The proposed model outperforms the open-source models in Zero- and Few-Shot Learning settings and shows significant performance improvements. |
Instruction Induction: From Few Examples to Natural Language Task Descriptions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models can perform unseen tasks by conditioning on a few input-output demonstrations, but task inference is implicit and the ability of models to explicitly reason about it remains unexplored. |
| Approach: | They propose an instruction induction challenge in which a model is asked to generate a natural language instruction that fits a set of labeled examples. |
| Outcome: | The proposed model achieves 65.7% of human performance while the original model only reaches 9.8% of human performances. |
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to few-shot text classification require domain expertise and an understanding of the language model's abilities to define the mapping between words and labels. |
| Approach: | They propose a method that converts textual inputs to cloze questions that contain some form of task description and processes them with a pretrained language model to map the predicted words to labels. |
| Outcome: | The proposed approach performs almost as well as hand-crafted label-to-word mappings for a number of tasks with small amounts of training data. |
Language Models for Text Classification: Is In-Context Learning Enough? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research on text classification models with prompts is limited in scale and lacks understanding of how these methods compare to more established methods. |
| Approach: | They compare the performance of large and smaller language models with prompts to achieve state-of-the-art performance in many NLP tasks. |
| Outcome: | The proposed models outperform the more standard approaches in binary, multiclass, and multilabel tasks in a large scale evaluation of 16 text classification datasets. |
PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for phrase-similarity compare phrases alone (without context) and phrases with context (with or without context). |
| Approach: | They propose to use a dataset of 28K noun phrases accompanied by their contextual Wikipedia pages to train machine phrase embeddings. |
| Outcome: | The proposed dataset improves ranking-models’ accuracy and pushes span selection models near human accuracy, which is 95% Exact Match (EM) on semantic search given a query phrase and a passage. |
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning (2023.acl-long)
Copied to clipboard
Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, Sang-Woo Lee
| Challenge: | Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. |
| Approach: | They propose to use user behavior sequences as plain text to represent rich information in any domain or system without losing generality. |
| Outcome: | The proposed frameworks achieve excellent results on diverse recommendation tasks and can be used on unseen domains and services. |