Papers by Junmo Kang
Let Me Know What to Ask: Interrogative-Word-Aware Question Generation (D19-58)
Copied to clipboard
| Challenge: | Existing models focus on generating questions based on text and the answer to the generated question. |
| Approach: | They propose a pipelined system that predicts the type of interrogative word to be generated . they also propose qg models that can be used to generate questions based on text . |
| Outcome: | The proposed system improves on the task of QG in SQuAD, improving from 46.58 to 47.69 in BLEU-1, 17.55 to 18.53 in blu-4, 21.24 to 22.33 in METEOR, and 44.53 to 46.94 in ROUGE-L. |
MATE: Meet At The Embedding - Connecting Images with Long Texts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Vision Language Models (VLMs) focus on aligning images with short descriptive captions. |
| Approach: | They propose a method that combines VLMs with Large Language Models to efficiently align images with long texts without additional text pairs. |
| Outcome: | The proposed method bridges the gap between VLM and LLM without additional image-long text pairs. |
Exploiting Numerical-Contextual Knowledge to Improve Numerical Reasoning in Question Answering (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing numerical reasoning models overly rely on parametric knowledge at inference time . previous studies show that understanding numbers in text improves numerical reasoning accuracy . |
| Approach: | They propose a numerical reasoning model that leverages parametric knowledge to alleviate this over-reliance on parametric information. |
| Outcome: | The proposed model improves numerical reasoning accuracy and performance in DROP. |
Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets. |
| Approach: | They propose an unsupervised QA approach that generates QA training data automatically. |
| Outcome: | The proposed method improves unsupervised QA significantly across a number of QA tasks. |
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent surveys of literature highlight the overwhelming growth of Large Language Models (LLMs). |
| Approach: | They propose a semi-automated literature analysis approach that automates literature analysis using LLMs. |
| Outcome: | The proposed approach reduces paper surveying and data extraction by 93% compared to manual methods. |
Have You Seen That Number? Investigating Extrapolation in Question Answering Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Numerical reasoning in machine reading comprehension (MRC) has shown drastic improvements over the past few years. |
| Approach: | They propose an E-digit number form that alleviates the lack of extrapolation in numerical MRC models. |
| Outcome: | The proposed model can't extrapolate to unseen numbers, the authors say . they also show that the model needs to treat numbers differently from regular words . |
Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing retrieval-augmented language models assume query relevance and irrelevance as dichotomy . existing models are highly brittle to the presence of conflicting information in both the fine-tuning and in-context few-shot learning scenarios. |
| Approach: | They propose methods for handling knowledge conflicts by fine-tuning a discriminator or prompting it to elicit its discriminative capability. |
| Outcome: | The proposed approaches significantly enhance model robustness on open-domain QA. |
Self-Specialization: Uncovering Latent Expertise within Large Language Models (2024.findings-acl)
Copied to clipboard
Junmo Kang, Hongyin Luo, Yada Zhu, Jacob Hansen, James Glass, David Cox, Alan Ritter, Rogerio Feris, Leonid Karlinsky
| Challenge: | Recent studies have demonstrated the effectiveness of self-alignment in which a large language model is aligned to follow general instructions using instructional data generated from the model itself. |
| Approach: | They propose to use human-written seeds to align large language models to follow general instructions to achieve cross-task generalization. |
| Outcome: | The proposed model outperforms base models and models that are generally instruction-tuned or have been adapted to the target domain by a large margin. |
Regularization of Distinct Strategies for Unsupervised Question Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Unsupervised question answering (UQA) is a task of answering questions from a context that contains the answer. |
| Approach: | They propose a method to generate higher-quality questions with a teacher-student architecture and a regularization module to avoid bias toward a particular question generation strategy. |
| Outcome: | The proposed method generates higher-quality questions across diverse datasets and tasks and can be used to create a model with few-shot learning. |
Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to information retrieval (IR) and natural language processing (NLP) use contextual language models, which can improve both synonymy and polysemy problems associated with words. |
| Approach: | They propose an ultra-high dimensional representation scheme equipped with directly controllable sparsity and a bucketing method where embeddings from multiple layers of BERT are selected/merged to represent diverse linguistic aspects. |
| Outcome: | The proposed representation scheme outperforms sparse models with MS MARCO and TREC CAR, and shows that it is highly efficient for storage and search. |
Schema-Driven Information Extraction from Heterogeneous Tables (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on information extraction from tables has focused on developing custom pipelines for each table collection. |
| Approach: | They propose a task that transforms tabular data into structured records following a human-authored schema. |
| Outcome: | The proposed task achieves F1 scores ranging from 74.2 to 96.1 while maintaining cost efficiency. |
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models (2023.acl-long)
Copied to clipboard
| Challenge: | Increasing the size of pre-trained models can consistently improve performance on downstream tasks after fine-tuning, as seen in studies based on BERT, RoBERTa, T5 and empirical scaling laws. |
| Approach: | They propose to use knowledge distillation to build a compact model with a fixed budget instead of annotating data and manually labeling it. |
| Outcome: | The proposed approach reduces inference costs by reducing costs by hiring annotators and labelling data. |
Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning (2025.acl-long)
Copied to clipboard
| Challenge: | Results show that supervised fine-tuning and preference finetunation are the most efficient approaches for large language models. |
| Approach: | They propose to use Supervised Finetuning and Preference Finetunes to optimize training data budgets for Large Language Models. |
| Outcome: | The proposed approach improves performance on math tasks by 15% on the most expensive model, 1,000 examples. |
Leveraging Order-Free Tag Relations for Context-Aware Recommendation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to tag recommendation neglect orderlessness and inter-dependency . Empirical results on Instagram and Stack Overflow show that our method is significantly superior to the previous approaches. |
| Approach: | They propose a sequence-oblivious generation method for tag recommendation . the next tag to be generated is independent of the order of the generated tags . they also propose regressive generation methods that take orderlessness into account . |
| Outcome: | Empirical results show that the proposed method is superior to previous approaches . the proposed system is based on two domains, Instagram and Stack Overflow . |
Graph-Induced Transformers for Efficient Multi-Hop Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent MHQA tasks that require inter-paragraph/sentence linkages use graphs to model internal structural information within text. |
| Approach: | They propose a graph-induced transformer that applies graph-derived attention patterns directly into a PLM without external graph modules. |
| Outcome: | The proposed model can replace external graph modules while preserving model performance. |