Papers with ML
A New Approach to Animacy Detection (C18-1)
Copied to clipboard
| Challenge: | Animacy is a property for a referent to be an agent, and prior work has classified words as either animate or inanimate. |
| Approach: | They propose a method that uses supervised machine learning and hand-built rules to classify the animacy of co-reference chains. |
| Outcome: | The proposed method achieves state-of-the-art performance on a 142-text dataset . it leverages word embeddings over referring expressions, parts of speech, and grammatical and semantic roles . |
Towards Reproducible Machine Learning Research in Natural Language Processing (2022.acl-tutorials)
Copied to clipboard
Ana Lucic, Maurits Bleeker, Samarth Bhargav, Jessica Forde, Koustuv Sinha, Jesse Dodge, Sasha Luccioni, Robert Stojnic
| Challenge: | a tutorial on reproducibility in ML addresses the problem of research results that are not reproducible. |
| Approach: | They propose a tutorial to ensure reproducible research in ML with an emphasis on computational linguistics and NLP. |
| Outcome: | The proposed tutorial focuses on computational linguistics and NLP . it provides a framework for using reproducibility as a teaching tool in university-level computer science programs. |
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements (2022.emnlp-demos)
Copied to clipboard
Leandro Von Werra, Lewis Tunstall, Abhishek Thakur, Sasha Luccioni, Tristan Thrush, Aleksandra Piktus, Felix Marty, Nazneen Rajani, Victor Mustar, Helen Ngo
| Challenge: | Evaluation is a key part of machine learning, yet there is neo-tooling to support it . auxiliary techniques such as testing for significance, measuring statistical power, and auxiliary methods are not available in ML. |
| Approach: | They propose a set of tools to facilitate the evaluation of models and datasets in machine learning . they propose 'evaluation on the Hub' platform that enables large-scale evaluation of over 75,000 models . |
| Outcome: | The proposed tools can be used to evaluate models and datasets on the Hugging Face Hub. |
PeerQA: A Scientific Question Answering Dataset from Peer Reviews (2025.naacl-long)
Copied to clipboard
| Challenge: | a dataset of 579 QA pairs from 208 scientific articles contains answers that reviewers raised while thoroughly examining the scientific article. |
| Approach: | They propose a dataset that contains questions that reviewers raised while thoroughly examining the scientific article. |
| Outcome: | The proposed dataset contains 579 QA pairs from 208 academic articles . the results show that decontextualization approaches improve retrieval performance . |
Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms (2025.acl-long)
Copied to clipboard
| Challenge: | Social media platforms use machine learning and artificial intelligence to maximize user engagement, but can indirectly cause exposure to harmful content. |
| Approach: | They propose a re-ranking approach using Large Language Models to assess and rerank content sequences using large annotated data sets. |
| Outcome: | The proposed method significantly outperforms existing proprietary moderation methods on three datasets, three models and across three configurations. |
Vote’n’Rank: Revision of Benchmarking with Social Choice Theory (2023.eacl-main)
Copied to clipboard
Mark Rofin, Vladislav Mikhailov, Mikhail Florinsky, Andrey Kravchenko, Tatiana Shavrina, Elena Tutubalina, Daniel Karabekyan, Ekaterina Artemova
| Challenge: | ML benchmarks have been criticized for their construct validity, fragility of the design and task choices. |
| Approach: | They propose a framework for ranking systems in multi-task benchmarks under the principles of the social choice theory and propose 'vote'n'rank' procedures are more robust than the mean average while being able to handle missing performance scores and determine conditions under which the system becomes the winner. |
| Outcome: | The proposed framework can be utilised to draw new insights on benchmarking in several ML sub-fields and identify the best-performing systems in research and development case studies. |
TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation (P18-1)
Copied to clipboard
Alexander Fabbri, Irene Li, Prawat Trairatvorakul, Yijiao He, Weitai Ting, Robert Tung, Caitlin Westerfield, Dragomir Radev
| Challenge: | TutorialBank is a publicly available dataset that aims to facilitate NLP education and research . a google search of "Natural Language Processing" returns over 100 million hits with papers, tutorials, 1 http://aan.how blog posts, codebases and other related online resources. |
| Approach: | They have manually collected and categorized over 5,600 resources on NLP . they have created a search engine and command-line tool to search the corpus . |
| Outcome: | The tutorial bank dataset is the largest manually-picked corpus of resources intended for NLP education . it includes lists of research topics, relevant resources for each topic, prerequisite relations among topics . |
PuzzLing Machines: A Challenge on Learning From Small Data (2020.acl-main)
Copied to clipboard
| Challenge: | a benchmark dataset of 81 languages is released to test deep neural models' human-like reasoning and generalization skills. |
| Approach: | They propose a challenge on learning from small data using Rosetta Stone puzzles from Linguistic Olympiads for high school students. |
| Outcome: | The proposed benchmark consists of Rosetta Stone puzzles from Linguistic Olympiads for high school students. |
Unmasking the Myth of Effortless Big Data - Making an Open Source Multi-lingual Infrastructure and Building Language Resources from Scratch (2022.lrec-1)
Copied to clipboard
Linda Wiechetek, Katri Hiovain-Asikainen, Inga Lill Sigga Mikkelsen, Sjur Moshagen, Flammie Pirinen, Trond Trosterud, Børre Gaup
| Challenge: | During the last two decades, machine learning approaches have dominated the field of natural language processing (NLP) weak literary traditions give rise to corpora too unreliable to function as a model for NLP tools. |
| Approach: | They propose an alternative to corpus-based language technology that can provide language technology solutions for minority languages. |
| Outcome: | The proposed approach can provide language technology solutions for minority languages outside the reach of corpus-based language technology. |
Does My Rebuttal Matter? Insights from a Major NLP Conference (N19-1)
Copied to clipboard
| Challenge: | Peer review is a core element of the scientific process, but few studies have evaluated its properties empirically. |
| Approach: | They propose to use peer review to assess the effectiveness of rebuttal phase in NLP conferences. |
| Outcome: | The proposed task predicts after-rebuttal scores from initial reviews and author responses. |
MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to automating ML are time-consuming and difficult to understand for human developers. |
| Approach: | They propose a framework that leverages large language models to develop ML solutions for novel tasks. |
| Outcome: | The proposed framework bridges the gap between machine intelligence and human knowledge by exploiting state-of-the-art large language models. |
Understanding “Democratization” in NLP and ML Research (2024.emnlp-main)
Copied to clipboard
| Challenge: | a large number of NLP and ML papers mention terms related to democracy . authors find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
| Approach: | They analyze papers using the term "democra*" to clarify how it is understood in NLP and ML . they find that democratization is most frequently used to convey (ease of) access to or use of technologies . |
| Outcome: | The authors analyze papers using the term "democra*" they find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating longitudinal multimodal EHRs are limited due to privacy concerns. |
| Approach: | They propose to generate longitudinal multimodal EHRs by unconditional generation or longitudinal inference . existing methods generate single-modal E HRs by conditional generation or by longitudinal inferment . |
| Outcome: | The proposed method is more flexible and controllable than existing methods and is more cost-effective than existing ones. |
Building Question-Answer Data Using Web Register Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in web register (genre) identification have created a shortage of QA datasets for English and Finnish. |
| Approach: | They propose a machine learning-based method for extracting QA pairs from web-scale data using XLM-R and a multilingual CORE web register corpus . they then develop a NER-style token classifier to identify the QA text spans within these documents. |
| Outcome: | The proposed method is adaptable to any language given the availability of language models and extensive web data, but it is limited to English and Finnish. |
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: A Case Study on Hateful Memes (2024.lrec-main)
Copied to clipboard
| Challenge: | Causal analyses define semantics, while gradient-based methods are essential to eXplainable AI (XAI), interpreting the model’s ‘black box’. |
| Approach: | They propose to integrate causal analysis and XAI to integrate a model's mechanisms into their analysis by integrating a dataset of hateful meme detection models. |
| Outcome: | The proposed model can detect hateful memes using intersectionality principles and summarized attention scores highlight distinct behaviors of three Transformer models. |
How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text adversarial attacks are impractical in real-world scenarios where humans are involved. |
| Approach: | They have surveyed 378 human participants about the perceptibility of text adversarial examples produced by state-of-the-art methods. |
| Outcome: | The proposed methods ignore the property of imperceptibility or study it under limited conditions. |
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories (2024.emnlp-main)
Copied to clipboard
Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, Tushar Khot
| Challenge: | Large Language Models (LLMs) have made significant progress in writing code, but can they be used to reproduce results from research repositories? |
| Approach: | They propose a benchmark to evaluate the capability of Large Language Models to reproduce results from research repositories. |
| Outcome: | The benchmark aims to capture the realistic challenges faced by researchers working with machine learning and natural language processing repositories. |
The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation (2022.emnlp-main)
Copied to clipboard
| Challenge: | a paper argues that human label variation impacts all stages of the ML pipeline . human label variations are often considered noise due to disagreement, subjectivity in annotation or multiple plausible answers. |
| Approach: | They propose to reconcile different notions of human label variation and propose a repository of publicly-available datasets with un-aggregated labels. |
| Outcome: | The proposed approaches are compared with publicly available datasets with un-aggregated labels and identify gaps. |
CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing threat models underestimate subset-training privacy risks because of the scale of modern datasets. |
| Approach: | They propose a unified framework for analyzing privacy leakage in subset selection based on side-channel metadata from the subset process or via the outputs of the target model. |
| Outcome: | The proposed framework analyzes privacy leakage in subset selection based on two different scenarios . |
MLAlgo-Bench: Can Machines Implement Machine Learning Algorithms? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, the top-performing models achieve a 48.8% task completion rate on realizing machine learning algorithms . |
| Approach: | They propose a benchmark to test machine learning's ability to generate ML code for humans . they propose an automatic evaluation framework with metrics such as task pass rate and time overhead . |
| Outcome: | The proposed benchmark is unique in its focus on interpreting complex human instructions and producing multi-step, high-complexity code. |
LitSearch: A Retrieval Benchmark for Scientific Literature Search (2024.emnlp-main)
Copied to clipboard
| Challenge: | Literature search questions pose significant challenges for modern retrieval systems . a lack of domain expertise and reasoning through lengthy papers is a challenge . |
| Approach: | They propose a retrieval benchmark for literature search queries using inline citations from papers and questions about recently published papers. |
| Outcome: | The proposed retrieval benchmarks outperform state-of-the-art retrieval models and reranking pipelines. |