Papers by Mingyu Lee
LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)
Copied to clipboard
Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models lack commonsense knowledge . |
| Approach: | They propose a contextualized knowledge prompting scheme to guide the contextualization of structured commonsense knowledge based on large language models. |
| Outcome: | The proposed approach outperforms the state-of-the-art technique by an average of 5.8%. |
Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inefficient transformers have been a challenge for many years, requiring computational costs that scale quadratically with the length of the input sequence. |
| Approach: | They propose a token reduction approach that dynamically routes tokens within layers to ensure that all tokens remain accessible in subsequent layers. |
| Outcome: | The proposed approach achieves up to 25x faster inference time without significant loss in accuracy. |
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches for language models have overlooked the difficulty of training examples. |
| Approach: | They propose a framework that controls difficulty of training examples during pre-training by a tutor network. |
| Outcome: | The proposed framework outperforms state-of-the-art KD methods with student models on the GLUE benchmark. |
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chemical representation learning often lead to overfitting and limited scalability due to early convergence. |
| Approach: | They propose a framework to train Transformers on SMILES sequences to learn from structural examples and integrate external materials embedding to enrich molecular representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on molecular property prediction tasks. |
Curriculum Debiasing: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases (2025.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) addresses the memory footprint issue of full fine- tuning by modifying only a subset of model parameters. |
| Approach: | They propose a framework that debiases models in a biased-to-unbiased order and uses only a subset of parameters to modify model parameters. |
| Outcome: | The proposed framework accelerates convergence on unbiased examples by approximately twofold and improves ID and OOD performance by 1.2% and 8.0%, respectively. |
Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to commonsense inference lack coverage and expressive diversity of commonsensense knowledge graphs. |
| Approach: | They propose a framework that contrasts sets of semantically similar and dissimilar events . they propose 'solar' framework that can be used to learn commonsense inference . |
| Outcome: | The proposed framework outperforms the state-of-the-art commonsense transformer on commonsensense inference by 1.84% on average among 8 metrics. |
“Going to a trap house” conveys more fear than “Going to a mall”: Benchmarking Emotion Context Sensitivity for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
| Approach: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
| Outcome: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
Improving Bias Mitigation through Bias Experts in Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to mitigate the detrimental effect of bias on the network include debiasing methods that down-weight the biased examples identified by an auxiliary model, which is trained with explicit bias labels. |
| Approach: | They propose a framework that introduces binary classifiers between the auxiliary model and main model, coined bias experts, to reduce the detrimental effect of bias on the network. |
| Outcome: | The proposed approach outperforms the state-of-the-art on various datasets while achieving high performance on in-distribution data. |
Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) has been widely used for pre-training effective bidirectional representations but comes at a substantial training cost. |
| Approach: | They propose a concept-based curriculum masking method that evaluates the MLM difficulty of each token based on a carefully-designed linguistic difficulty criterion. |
| Outcome: | The proposed method significantly improves pre-training efficiency with the original BERT model at half the training cost. |
Connecting the Knowledge Dots: Retrieval-augmented Knowledge Connection for Commonsense Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models exhibit a limited understanding of commonsense reasoning due to the necessity of implicit knowledge that is rarely expressed in text. |
| Approach: | They propose a retrieval-augmented knowledge connection framework that transforms indirectly relevant documents into a direct explanation to answer a given question. |
| Outcome: | The proposed framework outperforms state-of-the-art (SOTA) benchmarks and achieves +2.0% and +4.6% average accuracy on in-domain (ID) and out-of domain (OOD) benchmark. |