Papers by Paul Bennett
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder (2021.emnlp-main)
Copied to clipboard
Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, Arnold Overwijk
| Challenge: | Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space. |
| Approach: | They propose a self-learning method that pre-trains the autoencoder using a weak decoder to push the encoder to provide better sequence representations. |
| Outcome: | The proposed model significantly boosts the effectiveness and few-shot ability of dense retrieval models on web search, news recommendation, and open domain question answering. |
Say ‘YES’ to Positivity: Detecting Toxic Language in Workplace Communications (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Toxic workplace communication is subtle, hidden or shows human biases . lack of corpus, sparsity of toxicity in enterprise emails hinder study . |
| Approach: | They propose a taxonomy to study toxic language at the workplace and a dataset to study it. |
| Outcome: | The proposed taxonomy provides a general and computationally viable taxonomies for studying toxic language at the workplace and analyzes why offensive language and hate-speech datasets are not suitable to detect workplace toxicity. |
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Dense retrieval (DR) methods first encode texts into a dense embedding space and then conduct text retrieval using efficient nearest neighbor search. |
| Approach: | They propose Momentum adversarial Domain Invariant Representation learning to train a domain classifier that distinguishes source versus target domains and adversarially updates the DR encoder to learn domain invariant representations. |
| Outcome: | The proposed method outperforms baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setting, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. |
Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text (2021.acl-long)
Copied to clipboard
| Challenge: | a novel approach to text simplification learns to balance a reward across three properties: fluency, salience and simplicity. |
| Approach: | They propose a novel algorithm to optimize the reward which proposes several candidate simplifications and a realistic text comprehension task as an evaluation method for text simplification. |
| Outcome: | The proposed model outperforms strong supervised baselines on the English news domain and can help people complete a comprehension task an average of 18% faster while retaining accuracy. |
Leveraging Structured Metadata for Improving Question Answering on the Web (2020.aacl-main)
Copied to clipboard
| Challenge: | Using metadata information from web pages can improve the performance of answer passage selection/reranking models. |
| Approach: | They propose a neural passage selection model that leverages metadata information with a fine-grained encoding strategy to learn the representation for metadata predicates in a hierarchical way. |
| Outcome: | The proposed model outperforms baseline models on the MS MARCO and Recipe-MARCO datasets and shows that it is more accurate than baseline models. |
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using mixture-of-memory augmenting to augment language models improves model generalization but with diminishing return. |
| Approach: | They develop a mechanism that augments language models with mixture-of-memory Augmentation (MoMA) they augment strong T5-based retrievers with the option to "plug in" unseen memory at inference time. |
| Outcome: | The proposed model outperforms methods with larger model sizes on the BEIR benchmark and achieves comparable or even better performance than methods relying on target-specific pretraining. |
Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision (2021.acl-long)
Copied to clipboard
Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett
| Challenge: | Neural information retrieval models have shown advanced results in many ranking scenarios where massive relevance labels or clickthrough data are available. |
| Approach: | They propose a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains. |
| Outcome: | The proposed method improves the few-shot ranking accuracy of Neu-IR models on three TREC benchmarks in the web, news, and biomedical domains. |
Axiomatic Preference Modeling for Longform Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have helped bridge the "alignment gap" between the responses of raw pretrained language models and responses that resonate more closely with human preferences. |
| Approach: | They propose to use a axiomatic framework to generate a rich variety of preference signals to uphold these signals. |
| Outcome: | The proposed model outperforms GPT-4 and ChatGPT in preference scoring. |