Papers by Yingxia Shao
RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieval-oriented language models focus on contextualized embedding of the [CLS] token, but recent study shows that ordinary tokens besides [CLL] may provide extra information, which help to produce a better representation effect. |
| Approach: | They propose a method where all contextualized embeddings of pre-trained model can be jointly pre-trained for retrieval tasks. |
| Outcome: | The proposed method improves the quality of representation where all contextualized embeddings of the pre-trained model can be leveraged. |
Matching-oriented Embedding Quantization For Ad-hoc Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Product quantization (PQ) is a widely used technique for ad-hoc retrieval. |
| Approach: | They propose a match-oriented product quantization with a multinoulli contrastive loss objective. |
| Outcome: | The proposed method maximizes matching probability of query and ground-truth key, compared with previous methods on non-supervised datasets. |
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval are not effective, but there are still challenges. |
| Approach: | They propose a retrieval oriented pre-training paradigm based on Masked Auto-Encoder (MAE) where the sentence embedding is generated from the encoder’s masked input and the original sentence is recovered based upon the sentence embedded and decoded input via mangled language modeling. |
| Outcome: | The proposed model significantly improves the SOTA performance on a wide range of NLP benchmarks, like BEIR and MS MARCO. |
Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval requires discriminative embeddings to represent the semantic relationship between query and document. |
| Approach: | They propose an unsupervised approach that performs unsupervised adaptation of large language models for dense retrieval. |
| Outcome: | The proposed model improves on a variety of dense retrieval benchmarks and is available on github. |
Reinforced IR: A Self-Boosting Framework For Domain-Adapted Information Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval methods struggle with highly specialized situations that require extensive domain expertise. |
| Approach: | They propose a method that integrates additional information from an LLM-based generator to enhance query performance and train the retriever to better discriminate the relevant documents identified by the generator. |
| Outcome: | The proposed method outperforms existing domain adaptation methods by a large margin and leads to substantial improvements in retrieval quality across a wide range of application scenarios. |
Causal Intervention and Counterfactual Reasoning for Multi-modal Fake News Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-modal fake news detection neglect the fact that some label-specific features cannot generalize well to the testing set, thus suffering from the latent data bias. |
| Approach: | They propose a Causal intervention and Counterfactual reasoning based debiasing framework for multi-modal fake news detection that eliminates the image-only bias by deducting the direct effect of the image from the total effect on labels. |
| Outcome: | The proposed framework eliminates the psycholinguistic bias in the text and the bias of inferring news label based on only image features. |
Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Existing commonsense question answering models incur prohibitive computation costs and poor interpretability . |
| Approach: | They propose a parameter efficient tuning network to pair PLMs with external knowledge for commonsense question answering. |
| Outcome: | The proposed adapter integrates entity- and query-related knowledge at a small cost. |