Papers by Yingbo Zhou
ARM: Alignment with Residual Energy-Based Model (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) acquire a wide range of abilities and abilities, but their behavior does not align with human preferences. |
| Approach: | They propose to minimize a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
| Outcome: | The proposed method can learn an aligned policy by minimizing a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data. |
| Approach: | They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages. |
| Outcome: | The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages. |
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)
Copied to clipboard
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval (2022.findings-acl)
Copied to clipboard
| Challenge: | a new model for parallel sentence retrieval can be used to align parallel sentences in multilingual corpora . a faithful aligner can help narrow down the candidate pool without having to deal with an enormous search space . |
| Approach: | They propose a model that can be trained on only one language pair and transfers to low-resource languages with negligible degradation in performance. |
| Outcome: | The proposed model outperforms the previous model on the Tateoba dataset by 8.0 points in accuracy and using less than 0.6% of their parallel data. |
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)
Copied to clipboard
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan
| Challenge: | Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales. |
| Approach: | They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. |
| Outcome: | The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more. |
CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | coding tasks require generated code to be fully executable and functionally correct . current agentic approaches struggle with multi-stage planning, generating, and debugging . |
| Approach: | They propose a framework for LLM agents to efficiently explore the search space in different stages of the code generation process. |
| Outcome: | The proposed framework achieves top results on 7 code generation benchmarks and a 31.9% solving rate on the SWEBench benchmark. |
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
Best-k Search Algorithm for Neural Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Modern natural language generation paradigms require a decoding strategy to obtain quality sequences out of the model. |
| Approach: | They propose a deterministic search algorithm balancing quality and diversity . they investigate the vanilla best-first search algorithm and propose k-k search algorithm. |
| Outcome: | The proposed algorithm is parameter-free, lightweight, efficient, and easy-to-use. |
Modeling Multi-hop Question Answering as Single Sequence Prediction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing generative question answering models that leverage passage retrieval with a pre-trained transformer are not effective for multihop QA. |
| Approach: | They propose a generative approach that explicitly models the reasoning process to resolve the answer for multi-hop questions by encoding cross-passage interactions. |
| Outcome: | The proposed model improves on two multi-hop QA datasets and is interpretable. |
AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation (2026.acl-long)
Copied to clipboard
Rui Qian, Chuanhang Deng, Qiang Huang, Jian Xiong, Mingxuan Li, Yingbo Zhou, Wei Zhai, Jintao Chen, Dejing Dou
| Challenge: | Existing models rely on a single segmentation token whose hidden state implicitly encodes both semantic reasoning and spatial localization . Existing methods rely only on SEG>, which encodes semantic reasoning, limiting the model's ability to explicitly disentangle what to segment from where to segment. |
| Approach: | They propose a method which reformulates reasoning segmentation as a structured conditional generation process over image tokens conditioned on language grounded query banks. |
| Outcome: | The proposed model bridges token-level predictions and pixel-level supervision by decoupling spatial grounding from semantic reasoning through structured language grounded query banks. |
Focused Attention Improves Document-Grounded Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document grounded generation is the task of using the information provided in a document to improve text generation. |
| Approach: | They propose two new document grounded generation tasks that use information provided in a document to improve text generation. |
| Outcome: | The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document. |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
SharPT: Shared Latent Space Prompt Tuning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prompt tuning is an efficient method for adapting large language models, but it is difficult and expensive to identify the source task that provides optimal prompts. |
| Approach: | They propose to learn a shared latent space which captures a set of basis skills from a mixture of source tasks and then transfer them to target tasks. |
| Outcome: | The proposed method outperforms previous methods on NLI, sentence completion, QA, conference resolution, word sense disambiguation and on various model scales. |
RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing KBQA approaches struggle with generalization of unseen KB schema items . Rank-and-generate approach solves coverage issue with strong generalization . |
| Approach: | They propose a Rank-and-Generate approach for KBQA that uses a generation model to generalize to unseen KB schema items. |
| Outcome: | The proposed approach outperforms the prior state-of-the-art on GrailQA and WebQSP datasets. |
Unsupervised Paraphrasing with Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase generation has benefited from recent advances in the design of training objectives and model architectures, but previous studies focused on supervised methods that require a large amount of labeled data that is costly to collect. |
| Approach: | They propose a transfer learning approach that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. |
| Outcome: | The proposed model performs state-of-the-art on the Quora Question Pair and ParaNMT datasets and is robust to domain shift between the two datasets. |
Few-shot Unified Question Answering: Tuning Models or Prompts? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Question-answering (QA) tasks investigate specific question types, knowledge domains, or reasoning skills, leading to specialized models catering to specific categories of QA tasks. |
| Approach: | They propose to use model and prompt tuning for unified QA in a low-resource setting to overcome drawbacks of unified models. |
| Outcome: | The proposed model and prompt tuning paradigms outperform model tuning in a few-shot setting with a good initialization and achieve a significant performance boost from pre-training in 'low-resource' setting. |
Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches on semantic parsing suffer from exponential growth of logical form candidates and can hardly generalize to unseen data. |
| Approach: | They propose a unified semantic parser for question answering on KB and DB . they define the primitive as the essential element in their framework . |
| Outcome: | The proposed framework can predict logical forms by altering and composing top-ranked primitives with different operations. |
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. |
| Approach: | They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning . |
| Outcome: | The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. |
HPE: Answering Complex Questions over Text by Hybrid Question Parsing and Execution (2023.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end neural networks excel at answering natural language questions but fail on complex ones . a proposed framework for question parsing and execution on textual QA is designed to combine the strengths of neural and symbolic methods. |
| Approach: | They propose a framework for question parsing and execution on textual QA . they parse questions into an intermediate representation and use deterministic rules to translate them . |
| Outcome: | The proposed framework outperforms existing methods in supervised, few-shot, and zero-shot settings while preserving its underlying reasoning process. |
Long Document Summarization with Top-down and Bottom-up Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent models infer latent representations of words or tokens with a transformer encoder, which is bottom-up and thus does not capture long-distance context well. |
| Approach: | They propose a method to infer latent representations of words or tokens in documents . they assume a hierarchical structure of a document where top-level captures long range dependency . |
| Outcome: | The proposed model can summarize an entire book and achieve competitive performance on a wide range of document summarization benchmarks. |
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large distribution shifts among different domains hinder transferability of keyphrase generation models. |
| Approach: | They propose a pipeline which guides KPG models’ learning focus from general syntactical features to domain-related semantics in a data-efficient manner. |
| Outcome: | The proposed pipeline can produce good quality keyphrases in new domains and achieve consistent improvements after adaptation with limited in-domain annotated data. |
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have been shown to be more prone to unfaithful facts . 30% of summaries generated by pre-trained language models suffer from hallucination . |
| Approach: | They propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control . they first compute entity coverage precision and prepend the corresponding control code . a further fine-tuning is performed to unlock zero-shot summarization . |
| Outcome: | The proposed method leads to more faithful and salient abstractive summarization in fine-tuning and zero-shot settings. |