Papers by Rui Shen
Copied to clipboard
| Challenge: | Experimental results show that retrieval-augmented NMT model obtains substantial improvements over strong baselines in the benchmark dataset. |
| Approach: | They propose a retrieval-augmented NMT model that is holistically similar to the source sentence while individually contrastive to each other. |
| Outcome: | The proposed model improves on baselines in the translation task. |
Copied to clipboard
| Challenge: | Current reranking models are optimized on static human annotations in isolation, decoupled from the downstream generation process. |
| Approach: | They propose a reinforcement learning framework that directly aligns reranking with LLM's generation quality. |
| Outcome: | Experiments on knowledge-intensive benchmarks show that RRPO outperforms strong baselines. |
Copied to clipboard
| Challenge: | Existing supervised defense methods rely on labeled malicious agents to train a supervised model of malicious behavior. |
| Approach: | They propose an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors. |
| Outcome: | The proposed method detects diverse attack types across MAS with various communication patterns while maintaining superior generalizability compared to baselines. |
Copied to clipboard
| Challenge: | Decomposed Reward Models extract diverse human preferences from binary comparisons without fine-grained annotations. |
| Approach: | They propose a decomposed reward model that extracts diverse human preferences from binary comparisons without fine-grained annotations. |
| Outcome: | The proposed approach extracts diverse human preferences from binary comparisons without fine-grained annotations. |
Copied to clipboard
| Challenge: | Current Chain-of-Thought based ESC methods often employ rigid, text-only reasoning, limiting adaptability in dynamic, multimodal interactions and introducing reasoning noise that degrades support quality. |
| Approach: | They propose a framework that integrates supervised fine-tuning with reinforcement learning to improve ESC models' response quality. |
| Outcome: | The proposed framework enables models to select contextually relevant thinking aspects: Visual Scene, Emotion, Situation, and Response Strategy. |
Copied to clipboard
| Challenge: | Existing knowledge editing techniques that modify models’ internal knowledge without full model retraining have gained significant attention. |
| Approach: | They propose an enhanced approach that merges value computation processes for facts sharing the same subject to improve editing efficiency. |
| Outcome: | The proposed method maintains a 98% editing success rate on same-subject and distinct-sub subject datasets while the original success rate drops to 46%. |
Copied to clipboard
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
Copied to clipboard
| Challenge: | Existing related work generation models are inflexible and extract sentences from multiple papers to form a related work discussion. |
| Approach: | They propose a Relation-aware Related work generator which generates an abstractive related work from the given multiple scientific papers in the same research area. |
| Outcome: | The proposed model improves over existing models and can be used to familiarize researchers with the state of the art in the field. |
Copied to clipboard
| Challenge: | Existing large language models are limited in understanding, reasoning, calculation, and generation, limiting their performance in complex reasoning and dynamic tasks. |
| Approach: | They propose a plug-and-play framework that integrates a small-scale LLM (as agent) with large-scale large-level LLMs (a as environment) they propose generating prompts that are used to interact with LLM, and a double constraint reward that optimizes correctness and quality of generation. |
| Outcome: | The proposed framework significantly outperforms baseline large-scale large-language models across various tasks. |
Copied to clipboard
| Challenge: | Existing summarization methods read through document only once to generate a document representation, resulting in a sub-optimal representation. |
| Approach: | They propose an iterative model for supervised extractive text summarization which polishes the document representation on many passes through the document. |
| Outcome: | The proposed model outperforms state-of-the-art extractive systems on CNN/DailyMail and DUC2002 datasets. |
Copied to clipboard
| Challenge: | Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales. |
| Approach: | They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. |
| Outcome: | The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more. |
Copied to clipboard
| Challenge: | E-commerce websites have billions of products, so it is impossible to write all copywriting manually. |
| Approach: | They propose a model to generate an AD post using a select network and a MGenNet network to generate a post including selected products. |
| Outcome: | The proposed model achieves impressive performance on a large-scale real-world AD post dataset. |
Copied to clipboard
| Challenge: | Existing approaches to task-oriented dialogue systems require a large number of handcrafted features and labels. |
| Approach: | They propose a "Two-Teacher One-Student" learning framework for task-oriented dialogue . the framework amalgamates knowledge from two teacher networks and provides guidance . |
| Outcome: | The proposed framework outperforms baseline methods on two benchmark datasets . it can retrieve accurate KB entities and generate human-like responses simultaneously . |
Copied to clipboard
| Challenge: | citation graphs can be used to extract scientific papers under different conditions. |
| Approach: | They propose a multi-granularity unsupervised summarization model that fine tunes a pre-trained encoder model on the citation graph by link prediction tasks. |
| Outcome: | The proposed model outperforms baseline models on a public benchmark dataset. |
Copied to clipboard
| Challenge: | Existing approaches to knowledge graph question answering (KGQA) face semantic misalignment and reasoning noise. |
| Approach: | They propose a fine-grained semantic parsing framework for KGQA that maps natural language queries to executable logical forms. |
| Outcome: | The proposed framework achieves 18.5% performance improvement over the SOTA on a multi-hop CWQ dataset. |
Copied to clipboard
| Challenge: | Existing Sequential Recommendation Systems (SRS) rely on collaborative filtering signals and fail to capture real-time user preferences. |
| Approach: | They propose a framework that integrates the long-term preference modeling of SRS with the real-time preference elicitation of CRS. |
| Outcome: | The proposed framework integrates the long-term preference modeling of SRS with the real-time preference elicitation of CRS. |
Copied to clipboard
| Challenge: | Existing methods for embedding knowledge graphs implicitly memorize relation rules to infer missing links, but they are difficult to memorize due to the inherent deficiencies of such implicit memorization strategy. |
| Approach: | They propose a vertical learning paradigm that allows to explicitly copy target information from related factual triples for more accurate prediction. |
| Outcome: | The proposed model improves generalization ability and makes distant link prediction significantly easier. |
Copied to clipboard
| Challenge: | Existing methods for summarizing text have not captured the salient information from an article. |
| Approach: | They propose a table-guided abstractive biography summarization that utilizes factual tables to capture important information and generate a summary of a biography. |
| Outcome: | The proposed method is the first large-scale biography summarization dataset with tables. |
Copied to clipboard
| Challenge: | Existing approaches to improve LLM reliability rely on factual hallucinations . Existing methods rely only on graph traversal, resulting in imprecise retrieval and heavy post-processing burdens. |
| Approach: | They propose a framework that integrates knowledge Graphs as structured, high-fidelity buffers to enhance LLM reliability. |
| Outcome: | The proposed framework allows logical constraints to be dynamically interleaved with graph search while optimizing via reinforcement learning with only final answer feedback eliminates the need for gold program annotations. |
Copied to clipboard
| Challenge: | ATOMIC is a large-scale commonsense knowledge graph (CSKG) containing everyday if-then knowledge triplets, i.e., head event, relation, tail event. |
| Approach: | They propose a CSKG completion method called Rel-CSKGC to predict the relation given the head event and tail event of a triplet and train a model based on existing triplets. |
| Outcome: | The proposed method is based on existing triplets and can be used to complete the missing links in ATOMIC. |
Copied to clipboard
| Challenge: | CharacterEval is a benchmark for comprehensive RPCA assessment in Chinese . authors show that Chinese LLMs exhibit more promising capabilities than GPT-4 in role-playing conversation. |
| Approach: | They propose a Chinese benchmark for comprehensive RPCA assessment . they use a dataset of Chinese role-playing dialogues and character profiles . |
| Outcome: | The proposed benchmark demonstrates that Chinese LLMs exhibit more promising capabilities than GPT-4 in Chinese role-playing conversation. |
Copied to clipboard
| Challenge: | Procedural text summarization task is a popular task in the NLP field because of its long length and complexity. |
| Approach: | They propose a procedural text summarization task with two granularity . they propose an Entity-State Graph-based Summarizer (ESGS) which aggregates contextual information for each procedure. |
| Outcome: | The proposed model can summarize the entire procedural text or give an overview for each step or both . Experiments on two datasets confirm the proposed model's effectiveness. |
Copied to clipboard
| Challenge: | Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive. |
| Approach: | They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C. |
| Outcome: | The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness. |
Copied to clipboard
| Challenge: | Visual commonsense data sets lack visual grounded representations of commonsensense . existing knowledge bases lack visual-based knowledge tied to actual visual scenes . |
| Approach: | They present a large-scale visual commonsense dataset with over 100,000 images and 14 million object-commonsense pairs that integrates both Seen (directly observable) and Unseen (inferrable) commonsens. |
| Outcome: | The proposed model integrates Seen (directly observable) and Unseen (inferrable) commonsense across Property, Action, and Space aspects. |
Copied to clipboard
| Challenge: | Commonsense knowledge graphs are typically represented by short text, resulting in many different nodes representing the same concept. |
| Approach: | They propose a framework based on Contrastive Pretraining and Node Clustering to solve these problems. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two CSKG completion benchmarks. |
Copied to clipboard
| Challenge: | Existing reward models assume a global reward function, limiting personalization and pluralistic alignment. |
| Approach: | They propose a framework that leverages binary preference datasets to enhance personalized preference learning. |
| Outcome: | The proposed framework captures diverse human preferences without fine-grained annotations and significantly improves personalized preference learning on downstream tasks. |
Copied to clipboard
| Challenge: | Existing approaches to encode natural languages without orders are lacking. |
| Approach: | They conduct a comprehensive analysis of the ability of neural models to organize sentences from a bag of words under three typical scenarios. |
| Outcome: | The proposed models can reorder or reconstruct sentences from a bag of words under three typical scenarios. |
Copied to clipboard
| Challenge: | Using sub-linear length normalized log-probabilities (SLLN-LP), we find unequal lengths of sentences in minimal pairs difficult for LMs even up to 32B parameters. |
| Approach: | They propose to use ZhoBLiMP as a linguistic minimal pair benchmark for Chinese language models to mitigate biases. |
| Outcome: | The proposed metric mitigates biases in Chinese language models with over 100 paradigms . Anaphor, Quantifiers, and Ellipsis are difficult for LMs even up to 32B parameters . |
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance. |
| Approach: | They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks. |
| Outcome: | The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude . |
Copied to clipboard
| Challenge: | Existing multimodal Mixture-of-Experts models accurately perceive image content yet fail in subsequent reasoning . Seeing but not thinking phenomenon is a puzzling phenomenon . |
| Approach: | They propose a routing-guided intervention method that enhances domain expert activation. |
| Outcome: | The proposed method achieves consistent improvements on visual reasoning tasks. |
Copied to clipboard
| Challenge: | Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers. |
| Approach: | They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion. |
| Outcome: | The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context. |
Copied to clipboard
| Challenge: | Recent techniques such as Generation-Augmented Retrieval (GAR) and Generative Document Retrieleval (GDR) leverage LLMs to enhance retrieval performance but face key challenges: GAR’s generated content may not always align with the target document corpus, while GDR limits the generative capacity of LLM. |
| Approach: | They propose a Context-Aware Generation-Augmented Retrieval approach which integrates corpus information into their generation process. |
| Outcome: | Experimental results show that CA-GAR outperforms existing methods on seven tasks and four non-English languages. |
Copied to clipboard
| Challenge: | Systematic reviews (SRs) are vital for evidence-based practice in high stakes disciplines, such as healthcare. |
| Approach: | They propose a human-centered interactive AI agent powered by large language models that partitions a large literature corpus based on semantics and employs . |
| Outcome: | InsightAgent improves quality of synthesized SRs by 27.2%, reaching 79.7% of human-written quality. |
Copied to clipboard
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values. |
| Approach: | They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values . |
| Outcome: | The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets. |
Copied to clipboard
| Challenge: | Existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. |
| Approach: | They propose to use supervised fine-tuning to generate long reasoning data from more capable Large Language Models and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. |
| Outcome: | Experiments on four LLMs and five evaluation benchmarks show that the proposed approach is effective in mitigating step length confounding problem. |
Copied to clipboard
| Challenge: | Misaligned large language models can magnify harm by exploiting them to undermine safety . et al., 2022b; Bai e.t., 2023): misalignment, realignment and model-specific resistance are important . |
| Approach: | They evaluate four methods to identify a mechanism asymmetry between attack and defense . they find that ORPO is most effective for misalignment, but DPO excels in realignment . |
| Outcome: | The proposed methods show a mechanism asymmetry between attack and defense . the proposed methods excel in realignment, but at the expense of model utility . |
Copied to clipboard
| Challenge: | Existing knowledge editing methods for large language models struggle to maintain logical consistency when propagating ripple effects to associated facts. |
| Approach: | They propose a framework that synergizes knowledge graph-derived logical rules with LLM logical reasoning capabilities to enable systematic chain updates. |
| Outcome: | The proposed framework improves logical generalization and specificity while maintaining reliability and specificness. |
Copied to clipboard
| Challenge: | Existing fact-checking methods focus on verification of individual facts, overlooking logical dependencies . a recent study shows that text containing logical errors may still be misjudged as factual . |
| Approach: | They propose a content–logic coupled factuality evaluation paradigm that conceptualizes factual dimension along two complementary dimensions: content factualism and logic factuity. |
| Outcome: | The proposed paradigm bridges the gap between factual verification and content factuality . it incorporates the logical dimension and a logic-aware metric to expose and penalize logical fallacies. |
Copied to clipboard
| Challenge: | Existing studies show that multimodal news can significantly improve users' sense of satisfaction for informativeness. |
| Approach: | They propose a task of Video-based Multimodal Summarization with Multimodal Output to solve this problem. |
| Outcome: | The proposed method can generate multimodal summaries with a single input . it can model the temporal dependency of video with semantic meaning of article . |
Copied to clipboard
| Challenge: | Extensive experiments on a large-scale real-world text summarization dataset show that PESG achieves the state-of-the-art performance in terms of both automatic metrics and human evaluations. |
| Approach: | They propose a model that learns summary patterns and prototype facts from a prototype document . they use a fact checker to estimate mutual information between the input document and generated summary . |
| Outcome: | Experiments on a large-scale real-world text summarization dataset show that PESG achieves state-of-the-art performance. |
Copied to clipboard
| Challenge: | Experimental results prove that language models can learn from human feedback better, irrespective of sequence length . emergence of length bias often induces the model to favor longer outputs . |
| Approach: | They propose to separate reward modeling from the influence of sequence length by using the Product-of-Experts technique. |
| Outcome: | The proposed approach shows that language models perform better regardless of sequence length . the main expert is focused on understanding human intents, while the biased expert targets the identification and capture of length bias. |
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis aims to determine the sentiment polarity towards a specific aspect in online reviews. |
| Approach: | They propose a relational graph attention network to encode a tree structure for sentiment prediction. |
| Outcome: | The proposed approach improves the performance of the graph attention network (GAT) on the SemEval 2014 and Twitter datasets. |
Copied to clipboard
| Challenge: | Existing methods to search for information are limited by data scarcity and lack contextual semantic understanding. |
| Approach: | They propose a new post-training paradigm with three self-supervised tasks to initialize the conversational search model to enhance dialogue structure and contextual semantic understanding. |
| Outcome: | The proposed method can initialize the conversational search model to enhance dialogue structure and contextual semantic understanding. |
Copied to clipboard
| Challenge: | Empirical studies for communication topology design often overlook why and when sparse and dense topologies help or hinder collaboration. |
| Approach: | They propose a topology design approach that balances error suppression and beneficial information propagation by fusing connectivity patterns from dense and sparse graphs. |
| Outcome: | The proposed topology design achieves superior performance across tasks with sparse and dense graphs. |