Papers by Dong-Ho Lee
Prediction-Augmented Generation for Automatic Diagnosis Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) adopt autoregressive architecture, predicting the next word token based on the preceding context. |
| Approach: | They propose a method that integrates task-specific predictive models as external tools to improve model generation quality and accuracy. |
| Outcome: | The proposed method improves the generation quality and predictive accuracy of large language models in inference-driven tasks. |
Perhaps PTLMs Should Go to School – A Task to Assess Open Book and Closed Book QA (2021.emnlp-main)
Copied to clipboard
| Challenge: | Taking the exam closed book, but having read the textbook, yields at best minor improvement (56%), suggesting that the PTLM may not have “understood” the textbook (or perhaps misundersttoo the questions). |
| Approach: | They propose to use pre-trained language models to answer questions from introductory college textbooks and hundreds of true/false statements based on review questions written by the authors. |
| Outcome: | The proposed task includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History). |
Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, human communication models fail to explicitly model common ground (CG) . less than half of the responses in current data is rated as high quality . |
| Approach: | They propose a dataset that annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
| Outcome: | The proposed dataset annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER (2022.acl-long)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates. |
| Approach: | They propose a demonstration-based learning method which lets the input be prefaced by task demonstrations for in-context learning. |
| Outcome: | The proposed method improves on in-domain learning and domain adaptation in low-resource settings. |
Analyzing Norm Violations in Live-Stream Chat (2023.emnlp-main)
Copied to clipboard
Jihyung Moon, Dong-Ho Lee, Hyundong Cho, Woojeong Jin, Chan Park, Minwoo Kim, Jonathan May, Jay Pujara, Sungjoon Park
| Challenge: | Existing methods for detecting toxic language and norm violations are limited to live-streaming platforms . existing methods are less effective when applied to live streaming platforms based on a limited time frame . |
| Approach: | They propose to use contextual information to automatically moderate toxic content on live streaming platforms. |
| Outcome: | The proposed model improves on live-streaming platforms by 35%. |
RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge (2021.findings-acl)
Copied to clipboard
| Challenge: | a riddle-style commonsense questions require complex commonsensense reasoning and figurative language skills . there is currently no dataset aimed at testing these abilities . authors propose a new multiple-choice question answering task . |
| Approach: | They propose a new multiple-choice question answering task that uses a large dataset for riddlestyle commonsense questions. |
| Outcome: | The proposed task comes with the first large dataset for answering riddlestyle commonsense questions. |
LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)
Copied to clipboard
Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren
| Challenge: | Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited. |
| Approach: | They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision. |
| Outcome: | The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances. |
Machine-Assisted Script Curation (2021.naacl-demos)
Copied to clipboard
Manuel Ciosici, Joseph Cummings, Mitchell DeHaven, Alex Hedges, Yash Kankanampati, Dong-Ho Lee, Ralph Weischedel, Marjorie Freedman
| Challenge: | Scripts have been of interest for encoding procedural knowledge and understanding stories for over 40 years . narrative descriptions often omit common knowledge . |
| Approach: | They propose a machine-aided script creator that automates script creation with suggestions for event types, links to Wikidata, and sub-events that may have been forgotten. |
| Outcome: | The proposed system automates portions of the script creation process with suggestions for event types, links to Wikidata, and sub-events that may have been forgotten. |
TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental information extraction task that focuses on extracting entities from a given text and classifying them using pre-defined categories. |
| Approach: | They propose to use “entity triggers” to facilitate label-efficient learning of NER models. |
| Outcome: | The proposed model is significantly more cost-effective than the traditional neural NER frameworks. |
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models lack visual knowledge of common objects due to reporting bias. |
| Approach: | They investigate whether integrating visual knowledge into a language model can fill the gap . they use captions and images to transfer visual knowledge to 5 downstream tasks . |
| Outcome: | The proposed model can improve performance on 5 tasks that may need visual knowledge to solve the problem. |
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal knowledge graphs (TKGs) are used to represent real-world facts in a structured way. |
| Approach: | They propose to use in-context learning with large language models for TKG forecasting . they compare naive LLMs to state-of-the-art (SOTA) supervised models . |
| Outcome: | The proposed approach performs well against pre-trained large language models . the proposed approach is based on simple heuristics and state-of-the-art models compared with pre-trainers . |
Making Large Language Models Better Data Creators (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced the field of NLP significantly, but deploying them for downstream applications is still challenging due to cost, responsiveness, control, or concerns around privacy and security. |
| Approach: | They propose a unified data creation pipeline that requires only a single formatting example. |
| Outcome: | The proposed pipeline can generate data with a single formatting example. |
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023.acl-demo)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Existing models are susceptible to learning spurious biases that do not reflect the underlying task. |
| Approach: | They propose an open-source framework for explanation-based model debugging that allows users to provide various forms of feedback on model explanations. |
| Outcome: | The proposed framework improves model’s OOD performance on text classification tasks by up to 18%. |
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning has advanced large language models (LLMs) but its application in lower-resource languages faces challenges due to the imbalanced foundational abilities of LLMs across different languages. |
| Approach: | They propose a pivot language guided generation approach that utilizes a high-resource language as the pivot to enhance instruction tuning in lower-resourced languages. |
| Outcome: | The proposed approach improves instruction-following abilities of LLMs by 29% on average compared to directly responding in the target language alone. |
AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (P19-3)
Copied to clipboard
| Challenge: | Existing sequence annotation tools focus on improving user interfaces and user interface. |
| Approach: | They propose an open-source web-based data annotation framework for sequence tagging tasks . the framework is based on active learning and automatic crowd consolidation . |
| Outcome: | The proposed framework is a comprehensive solution for sequence labeling tasks . it can be deployed in downstream systems while new annotations are being made . |
Evaluating Very Long-Term Conversational Memory of LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on long-term open-domain dialogues focus on evaluating responses within contexts spanning no more than five chat sessions. |
| Approach: | They propose a machine-human pipeline to generate very long-term dialogues by leveraging LLMs and retrieval augmented generation techniques. |
| Outcome: | The proposed pipeline generates very long-term dialogues using LLMs and RAGs . the generated conversations are verified and edited by human annotators for long-range consistency and grounding to the event graphs. |
Improving Text Auto-Completion with Next Phrase Prediction (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Language models such as GPT-2 require considerable training effort to adapt to specific writing domains (e.g., medical). |
| Approach: | They propose an intermediate training strategy that encourages language models to complete partial queries with enriched phrases and eventually improve their text auto-completion performance. |
| Outcome: | The proposed approach outperforms baselines in auto-completion tasks for email and academic-writing domains with only around 1.2B tokens. |
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automated forecasting studies rely on structured data to predict future events. |
| Approach: | They propose a question-answering task that limits access to unstructured text data . they use a crowdsourced dataset to form a restricted-domain, multiple-choice, question-announcement task . |
| Outcome: | The proposed model achieves 61.0% accuracy on the dataset, which still lags behind human performance by about 19%. |