Papers by Huan Sun

Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)

Challenge:	Existing pre-training methods only harvest learning signals from local contexts of naturally occurring texts . ReasonBert provides a method for reasoning over long-range relations and multiple, possibly hybrid contexts.
Approach:	They propose a method that augments language models with the ability to reason over long-range relations and multiple, possibly hybrid contexts.
Outcome:	The proposed method significantly improves sample efficiency over strong baselines.

Copied to clipboard

Boshi Wang, Xiang Deng, Huan Sun

Learning a Cost-Effective Annotation Policy for Question Answering (2020.emnlp-main)

Challenge:	Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks.
Approach:	They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference.
Outcome:	The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning.

Copied to clipboard

Bernhard Kratzwald, Stefan Feuerriegel, Huan Sun

Natural Language Interfaces

Challenge:	State-of-the-art question answering systems require large amounts of training data for which labeling is time consuming and thus expensive.
Approach:	They propose a framework for annotating QA datasets that entails learning a cost-effective annotation policy and a semi-supervised annotation scheme.
Outcome:	The proposed approach can reduce up to 21.1% of the annotation cost compared with traditional methods . the proposed approach is based on a cost-effective annotation policy and semi-supervised annotation scheme .

Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe (2023.acl-long)

Copied to clipboard

Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, Robert Sim

Ethical NLP

Challenge:	Privacy concerns have increased in data-driven products due to the tendency of machine learning models to memorize sensitive training data.
Approach:	They propose a method for generating useful synthetic text with a formal privacy guarantee by fine-tuning a pretrained generative language model with DP.
Outcome:	The proposed method produces synthetic text competitive in terms of utility with its non-private counterpart, while providing strong protection against potential privacy leakages.

Towards Transparent Interactive Semantic Parsing via Step-by-Step Correction (2022.findings-acl)

Copied to clipboard

Lingbo Mo, Ashley Lewis, Huan Sun, Michael White

Natural Language Interfaces

Challenge:	Existing studies on semantic parsing focus on mapping a natural-language utterance to a logical form (LF) but natural language may contain ambiguity and variability, making this challenge difficult.
Approach:	They propose an interactive semantic parsing framework that explains the predicted LF step by step in natural language and enables the user to make corrections through natural-language feedback for individual steps.
Outcome:	The proposed framework improves parsing accuracy and transparency in a crowdsourced dialogue dataset.

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)

Copied to clipboard

Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun

Low-Resource NLP

Challenge:	Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs).
Approach:	They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations.
Outcome:	The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference.

Joint Semantic and Strategy Matching for Persuasive Dialogue (2023.findings-emnlp)

Copied to clipboard

Chuhao Jin, Yutao Zhu, Lingzhen Kong, Shijie Li, Xiao Zhang, Ruihua Song, Xu Chen, Huan Chen, Yuchong Sun, Yu Chen, Jun Xu

Dialogue Systems & Conversational Agents

Challenge:	Persuasive dialogue models rely on utterance semantic matching and a key aspect has been ignored . compared with utterrance semantics, conversation strategies are high-level concepts, which can be informative and provide complementary information to achieve effective persuation.
Approach:	They propose to model conversation semantics and strategies to match them using a BERT-like module and an auto-regressive predictor.
Outcome:	The proposed model improves state-of-the-art by 5% on a small and 37% on 'large' datasets.

AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)

Copied to clipboard

Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun

Challenge:	generative search engines enhance the reliability of large language model responses by providing cited evidence.
Approach:	They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not .
Outcome:	The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation.

When is Tree Search Useful for LLM Planning? It Depends on the Discriminator (2024.acl-long)

Copied to clipboard

Ziru Chen, Michael White, Ray Mooney, Ali Payani, Yu Su, Huan Sun

Information Retrieval

Challenge:	Existing methods to build language agents that can plan efficiently and accurately have not met the needs of advanced planning methods to achieve such improvements.
Approach:	They propose to use iterative correction and tree search to solve multi-step problems in a language agent framework with three components: a generator, a discriminator, and a planning method.
Outcome:	The proposed methods improve performance on two tasks, text-to-SQL parsing and mathematical reasoning, while using discriminators with 90% accuracy.

Tooling or Not Tooling? The Impact of Tools on Language Agents for Chemistry Problem Solving (2025.findings-naacl)

Copied to clipboard

Botao Yu, Frazier N. Baker, Ziru Chen, Garrett Herb, Boyu Gou, Daniel Adu-Ampratwum, Xia Ning, Huan Sun

Challenge:	Existing evaluations of large language models (LLMs) with tools are limited and qualitative . existing evaluations have been limited and only focus on 14 tasks focusing on compound synthesis.
Approach:	They propose to develop an enhanced chemistry agent over ChemCrow to improve chemistry problem solving by integrating tools into LLMs.
Outcome:	The proposed agent does not consistently outperform its base LLMs without tools on specialized chemistry tasks and general chemistry questions.

GroundCocoa: A Benchmark for Evaluating Compositional & Conditional Reasoning in Language Models (2025.naacl-long)

Copied to clipboard

Harsh Kohli, Sachin Kumar, Huan Sun

Adversarial Training for Code Retrieval with Question-Description Relevance Regularization (2020.findings-emnlp)

Challenge:	Existing LLMs excel and often surpass human performance on benchmarks, but they are known to falter in simple tasks and under seemingly straightforward circumstances.
Approach:	They propose a benchmark to assess compositional and conditional reasoning within a flight booking task.
Outcome:	The proposed model outperforms existing models on the flight booking task with a 67% accuracy rate.

Copied to clipboard

Jie Zhao, Huan Sun

Robustness in NLP

Challenge:	Existing methods for code retrieval are based on question-description relevance . code retrievals are a key task aiming to match natural and programming languages .
Approach:	They propose to use question-description relevance to regularize adversarial learning for code retrieval . they adapt a simple adversarial learning technique to generate difficult code snippets .
Outcome:	The proposed method can improve the performance of state-of-the-art models on large-scale code retrieval datasets of two programming languages.

Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms (2023.acl-long)

Copied to clipboard

Tianshu Zhang, Changchang Liu, Wei-Han Lee, Yu Su, Huan Sun

Semantic Text Processing

Challenge:	Neural semantic parsers have achieved remarkable performance in recent years, but they are data-hungry and require annotators to have intimate knowledge of formal programs.
Approach:	They propose a task where multiple clients collaboratively train one global model without sharing their semantic parsing data.
Outcome:	The proposed model improves performance on three widely adopted FL algorithms (FedAvg, FedOPT and FedProx) and clients with smaller datasets enjoy faster performance.

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (2025.acl-long)

Copied to clipboard

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities (2024.naacl-long)

Challenge:	Recent advances in multimodal large language models have led to progress in tackling complex reasoning tasks that combine textual and visual information.
Approach:	They introduce a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Outcome:	The proposed model performs lower on MMMU-Pro than on the previous benchmark, ranging from 16.8% to 26.9%.

Copied to clipboard

Lingbo Mo, Boshi Wang, Muhao Chen, Huan Sun

An Imitation Game for Learning Semantic Parsers from User Interaction (2020.emnlp-main)

Challenge:	Rapid progress in open-source Large Language Models (LLMs) is driving AI development, but lacks sufficient trustworthiness to detect and mitigate adversarial demonstrations.
Approach:	They propose an extended Chain of Utterances-based (CoU) prompting strategy to attack open-source LLMs.
Outcome:	The proposed attack strategy is based on malicious demonstrations and toxicity tests on open-source models.

Copied to clipboard

Ziyu Yao, Yiqi Tang, Wen-tau Yih, Huan Sun, Yu Su

Challenge:	Existing methods for learning semantic parsers are expensive and tedious . despite the widespread applications, bootstrapping and fine-tuning is tedious a task .
Approach:	They propose an alternative method for learning semantic parsers directly from users . they propose an annotation-efficient imitation learning algorithm that iteratively collects new datasets .
Outcome:	The proposed method is cost-effective and shows promising performance on the text-to-SQL problem.

Structure-Grounded Pretraining for Text-to-SQL (2021.naacl-main)

Copied to clipboard

Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, Matthew Richardson

Multimodality

Challenge:	STRUG is a weakly supervised structure-based pretraining framework for text-to-SQL . it can be used to learn to capture text-table alignment in a given database schema .
Approach:	They propose a weakly supervised structure-grounded pretraining framework for text-to-SQL that can effectively learn to capture text-table alignment based on a parallel text-tab corpus.
Outcome:	The proposed framework outperforms BERTLARGE and BERTLAGE on all text-to-SQL alignment settings.

Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA (2026.acl-industry)

Copied to clipboard

Wenwei Li, Ming Xu, Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Structured Data in NLP

Challenge:	Existing methods for QA in industrial environments are inherently relational and often updated.
Approach:	They propose a framework that optimizes retrieval and generation through two components: Graph-aware Retrieval and evidence-constrained reinforcement learning.
Outcome:	Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, safety, and URL validity.

COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval (2021.emnlp-main)

Copied to clipboard

Xinliang Frederick Zhang, Heming Sun, Xiang Yue, Simon Lin, Huan Sun

Information Retrieval

Challenge:	16K FAQ items scraped from 55 credible websites . 32 human-annotated FAQ items for each query.
Approach:	They present a large, challenging dataset for FAQ retrieval for COVID-19 . they use a FAQ bank, Query Bank and Relevance Set to evaluate the dataset .
Outcome:	The proposed model achieves 48.8 under P@5 and is compared with other datasets.

Text-to-SQL Error Correction with Language Models of Code (2023.acl-short)

Copied to clipboard

Ziru Chen, Shijie Chen, Michael White, Raymond Mooney, Ali Payani, Jayanth Srinivasa, Yu Su, Huan Sun

WebOlympus: An Open Platform for Web Agents on Live Websites (2024.emnlp-demo)

Challenge:	Existing semantic parsers are not accurate enough for use in text-to-SQL parsing tasks.
Approach:	They propose to build clause-level edit models to correct SQL queries instead of token-level ones.
Outcome:	The proposed model improves the exact set match accuracy of different parsers by 2.4-6.5 and obtains up to 4.3 point absolute improvement over two strong baselines.

Copied to clipboard

Boyuan Zheng, Boyu Gou, Scott Salisbury, Zheng Du, Huan Sun, Yu Su

Multimodality

Challenge:	Web agents are emerging as powerful tools for automating tasks in cyberspace . however, there is a lack of standardized and user-friendly tools for research and development .
Approach:	They propose an open platform for web agents operating on live websites with a Chrome extension and a safety monitor module to ensure their trustworthiness.
Outcome:	WebOlympus is an open platform for web agents operating on live websites.

Exploring Chain of Thought Style Prompting for Text-to-SQL (2023.emnlp-main)

Copied to clipboard

Chang-Yu Tai, Ziru Chen, Tianshu Zhang, Xiang Deng, Huan Sun

Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again (2022.findings-emnlp)

Challenge:	In-context learning with large language models (LLMs) has recently caught increasing attention due to its superior few-shot performance on various tasks.
Approach:	They propose a new chain of thought prompting method that enhances LLMs’ reasoning ability through chain of thinking prompting, including the original chain-of-thought prompting and least-to-most prompting.
Outcome:	The proposed method brings 5.2 and 6.5 point absolute gains on the Spider development set and the Spider Realistic set, respectively, compared to the standard prompting method without reasoning steps; 2.4 and 1.5 point absolute gain, versus the least-to-most prompting.

Copied to clipboard

Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun, Yu Su

Low-Resource NLP

Challenge:	Large pre-trained language models (PLMs) such as GPT-3 have shown strong in-context learning capabilities, which are appealing for domains such as biomedicine that feature high and diverse demands of language technologies but also high data annotation costs.
Approach:	They propose to compare the few-shot performance of GPT-3 in-context learning with fine-tuning smaller (i.e., BERT-sized) PLMs on two representative biomedical information extraction tasks: named entity recognition and relation extraction.
Outcome:	The proposed model underperforms on two representative biomedical information extraction tasks.

Automatic Evaluation of Attribution by Large Language Models (2023.findings-emnlp)

Copied to clipboard

Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, Huan Sun

Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training (2026.acl-industry)

Challenge:	Generative large language models (LLMs) incorporate external references to generate and support claims. however, evaluating the attribution remains an open problem.
Approach:	They investigate automatic evaluation of attribution given by large language models . they define different types of attributed errors and then explore two approaches .
Outcome:	The proposed methods highlight promising signals and challenges.

Copied to clipboard

Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate (2023.findings-emnlp)

Challenge:	Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external knowledge, yet traditional singleround retrieval struggles with complex multistep reasoning.
Approach:	They propose a framework that introduces path-centric reward shaping for agentic RAG training.
Outcome:	The proposed framework improves on existing methods with an average accuracy gain of 7.7 points.

Copied to clipboard

Boshi Wang, Xiang Yue, Huan Sun

Global Relation Embedding for Relation Extraction (N18-1)

Challenge:	Large language models (LLMs) have shown impressive performance in complex reasoning tasks, but it is difficult to know whether they are reasoning based on deep understandings of truth and logic or leveraging their vast previously-seen patterns in a relatively shallow way.
Approach:	They propose to test large language models by engaging with them in a debate-like conversation where the user and LLM need to discuss to make the correct decision starting from opposing arguments.
Outcome:	The proposed model can achieve the correct answer on its own, but can also hold and defend its belief instead of blindly believing or getting misled by the user’s (invalid) arguments and critiques.

Copied to clipboard

Yu Su, Honglei Liu, Semih Yavuz, Izzeddin Gür, Huan Sun, Xifeng Yan

Information Extraction & Text Mining

Challenge:	Existing methods to extract textual relations with distant supervision are limited by their reliance on supervised training data.
Approach:	They propose to embed relations with global statistics of relations to combat the wrong labeling problem of distant supervision.
Outcome:	The proposed method is more robust to training noise introduced by distant supervision and improves relation extraction models.

AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientists (2025.emnlp-main)

Copied to clipboard

Yifei Li, Hanane Nour Moussa, Ziru Chen, Shijie Chen, Botao Yu, Mingyi Xue, Benjamin Burns, Tzu-Yao Chiu, Vishal Dey, Zitong Lu, Chen Wei, Qianheng Zhang, Tianyu Zhang, Song Gao, Xuhui Huang, Xia Ning, Nesreen K. Ahmed, Ali Payani, Huan Sun

Synthetic Question Value Estimation for Domain Adaptation of Question Answering (2022.acl-long)

Challenge:	AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery.
Approach:	They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows.
Outcome:	The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages.

Copied to clipboard

Xiang Yue, Ziyu Yao, Huan Sun

TableLlama: Towards Open Large Generalist Models for Tables (2024.naacl-long)

Challenge:	Existing work adapts QA scores to select high-quality questions, but these scores do not improve QA performance on the target domain.
Approach:	They propose to synthesize QA pairs with a question generator on the target domain . they propose to train a Question Value Estimator that estimates usefulness of synthetic questions .
Outcome:	The proposed method improves the performance of the target domain QA model by using synthetic questions and only 15% of the human annotations on the targetdomain.

Copied to clipboard

Tianshu Zhang, Xiang Yue, Yifei Li, Huan Sun

Multimodality

Challenge:	Existing methods for interpreting, augmenting, and querying semi-structured tables require pretraining on tables or special model architecture design.
Approach:	They construct a dataset with a variety of tables and tasks for instruction tuning and evaluating LLMs.
Outcome:	The proposed model achieves comparable or better performance on 7 out of 8 in-domain tasks compared with the base model on 6 out-of-domain datasets.

Reinforced Dynamic Reasoning for Conversational Question Generation (P19-1)

Copied to clipboard

Boyuan Pan, Hao Li, Ziyu Yao, Deng Cai, Huan Sun

Natural Language Interfaces

Challenge:	Empirical results on the recently released CoQA dataset demonstrate the effectiveness of our method . large-scale highquality conversational question answering datasets such as CoQA and QuAC can help train models to answer sequential questions.
Approach:	They propose a task called Conversational Question Generation which generates a question based on a passage and a conversation history to generate the next question.
Outcome:	The proposed method is based on a question-answering style conversation dataset . it can be used to generate meaningful questions on QA and SQuAD datasets .

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study (D19-1)

Copied to clipboard

Ziyu Yao, Yu Su, Huan Sun, Wen-tau Yih

Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset (2020.acl-main)

Challenge:	Existing semantic parsing technologies are not well-suited for use in real-world applications.
Approach:	They propose a model-based intelligent agent that generates a clarification question in natural language . they propose 'interactive semantic parsing' with a human user in the loop .
Outcome:	The proposed approach improves both parsing accuracy and user confidence . it is demonstrated on two text-to-SQL datasets with different state-of-the-art parsers .

Copied to clipboard

Xiang Yue, Bernal Jimenez Gutierrez, Huan Sun

Machine Reading Comprehension

Challenge:	Medical professionals often query over clinical notes to find information that can support their decision making.
Approach:	They propose to use expert-annotated question templates and existing i2b2 annotations to create emrQA, the first large-scale dataset for question answering based on clinical notes.
Outcome:	The proposed system can answer clinical questions without using domain knowledge.

Leveraging 2-hop Distant Supervision from Table Entity Pairs for Relation Extraction (D19-1)

Copied to clipboard

Xiang Deng, Huan Sun

Information Extraction & Text Mining

Challenge:	Existing methods to construct noisy labeled data for relation extraction (RE) are expensive and lacks the labeling capability.
Approach:	They propose a 2-hop DS strategy to enhance distantly supervised relation extraction (RE) by combining sentences that mention entities that are linked to each other.
Outcome:	The proposed method outperforms baselines on a benchmark dataset by a substantial margin.

A Multi-Aspect Framework for Counter Narrative Evaluation using Large Language Models (2024.naacl-short)

Copied to clipboard

Jaylen Jones, Lingbo Mo, Eric Fosler-Lussier, Huan Sun

Semantic Text Processing

Challenge:	Existing methods for counter narrative evaluation lack alignment with human judgment as they rely on superficial reference comparisons instead of incorporating key aspects of counter narrative quality as evaluation criteria.
Approach:	They propose to use 5 defined aspects to generate counter narrative candidates using human-annotated scores and feedback from counter narrative specialized NGOs to assess their effectiveness.
Outcome:	The proposed evaluation framework outperforms existing metrics and achieves strong alignment to human-annotated scores and feedback.

Error Detection for Text-to-SQL Semantic Parsing (2023.findings-emnlp)

Copied to clipboard

Shijie Chen, Ziru Chen, Huan Sun, Yu Su

Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)

Challenge:	Existing text-to-SQL parsers are often over-confident, thus casting doubt on their trustworthiness when deployed for real use.
Approach:	They propose a parser-independent error detection model for text-to-SQL semantic parsing . they use a language model of code as its bedrock and graph neural networks to learn structural features of queries .
Outcome:	The proposed model outperforms parser-dependent uncertainty metrics on three strong parsers . it could improve the performance and usability of text-to-SQL semantic parsing, it is shown .

Copied to clipboard

Muhao Chen, Chaowei Xiao, Huan Sun, Lei Li, Leon Derczynski, Anima Anandkumar, Fei Wang