Papers by Tan Yu
Copied to clipboard
| Challenge: | Topic modeling (TM) is a classic unsupervised learning task in the field of natural language processing. |
| Approach: | They propose a new taxonomy that emphasizes the role of LLMs and the design of end-to-end workflows. |
| Outcome: | The proposed taxonomy emphasizes the role of LLMs and the design of end-to-end workflows. |
Copied to clipboard
| Challenge: | Existing studies on multi-modal neural machine translation focus on visual information, but text and image may not match exactly, and visual noise is often ignored. |
| Approach: | They propose a noise-robust multi-modal interactive fusion approach with cross-modal relation-aware mask mechanism for MNMT. |
| Outcome: | The proposed model achieves state-of-the-art scores in all En-De, En-Fr and En-Cs translation tasks. |
Copied to clipboard
| Challenge: | MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements. |
| Approach: | a new system is built to integrate music-related tools and an autonomous workflow . the system is based on large language models (LLMs) that can be used to organize and decompose requests . |
| Outcome: | the proposed system integrates numerous music-related tools and an autonomous workflow to address user requirements. |
Copied to clipboard
| Challenge: | Existing multilingual video corpus moment retrieval methods are based on a two-stream structure. |
| Approach: | They propose a multilingual video corpus moment retrieval task that uses a two-stream structure to generate a query-visual similarity and a subtitle stream exploits the query-subtitle similarity. |
| Outcome: | The proposed method improves accuracy on a large-scale video corpus moment retrieval dataset. |
Copied to clipboard
| Challenge: | Existing closed-source LLMs have a performance gap in text-to-SQL reasoning tasks. |
| Approach: | They propose a SQL-based approach to synthesize reliable data to enhance text-to-SQL reasoning in LLMs. |
| Outcome: | The proposed model achieves state-of-the-art accuracy on the widely recognized Spider and BIRD benchmarks, significantly narrowing the performance gap with closed-source methods. |
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research. |
| Approach: | They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models. |
| Outcome: | The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable. |
Copied to clipboard
| Challenge: | Existing methods handle long-term memory (LTM) and short-term (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. |
| Approach: | They propose a framework that integrates LTM and STM management directly into the agent's policy and propose 'agentic memory' to train such unified behaviors. |
| Outcome: | The proposed framework outperforms strong memory-augmented baselines on five long-horizon benchmarks and achieves higher-quality long-term memory and more efficient context usage. |
Copied to clipboard
| Challenge: | Existing approaches to extract multiple relations from a paragraph require multiple passes over the paragraph. |
| Approach: | They propose a method to extract multiple relations from a paragraph by encoding the paragraph only once. |
| Outcome: | The proposed approach can perform state-of-the-art on the benchmark ACE 2005. |
Copied to clipboard
| Challenge: | Recent large language models have made progress at interpreting and executing instructions. |
| Approach: | They propose a method to decouple general reasoning from specialized knowledge . they propose to use abstract reasoning chains and domain tools to reify each chain . |
| Outcome: | The proposed method outperforms baseline methods on QA and mathematical reasoning domains. |
Copied to clipboard
| Challenge: | Bit-flip errors (BFEs) are hardware faults where individual bits in memory or processing units are unintentionally flipped. |
| Approach: | They propose a novel defense strategy to mitigate bit-flip errors (BFEs) they propose bfe protection and a self-correction mechanism to minimize performance degradation . |
| Outcome: | The proposed defense strategy minimizes performance degradation while significantly improving robustness against BFEs. |
Copied to clipboard
| Challenge: | Existing methods for OOD detection and ID classification tasks require massive amounts of ID labeled data and no OOD labeles. |
| Approach: | They propose to use OOD-resistant Prototypical Network to detect OOD cases with limited in-domain (ID) training data to solve this task. |
| Outcome: | The proposed solution outperforms state-of-the-art methods in zero-shot OOD detection task while maintaining a competitive performance on ID classification task. |
Copied to clipboard
| Challenge: | Existing attentive models attend to all words without prior focus, which results in inaccurate concentration on some dispensable words. |
| Approach: | They propose to use semantic role labeling to provide additional guidance for multi-turn dialogue rewriting models. |
| Outcome: | The proposed model outperforms existing models on multi-turn dialogue rewriting tasks. |
Copied to clipboard
| Challenge: | Existing research on news summarization focuses on single-language single-document (SLSD), single-linguistic multi-document or cross-language multi-doc (CLSD) however, in real-world scenarios, news articles often involve multiple documents in different languages, i.e., mixed-language MLMD. |
| Approach: | They propose a mixed-language multi-document news summarization dataset with four different languages and 10,992 source document cluster and target summary pairs. |
| Outcome: | The proposed dataset contains four different languages and 10,992 source document cluster and target summary pairs. |
Copied to clipboard
| Challenge: | Existing work adopts data augmentation techniques to generate pseudo-annotated sentences . existing methods neither preserve semantic consistency of original sentences nor preserve syntax structure of sentences when expressing relations using seq2seq models, resulting in less diverse augmentations. |
| Approach: | They propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures. |
| Outcome: | The proposed technique can bring 2.0% F1 improvements in three datasets under low-resource setting. |
Copied to clipboard
| Challenge: | This paper examines the pressing need to understand and manage inappropriate language within the evolving human-robot interaction landscape. |
| Approach: | They propose to use data cleaning methods to identify inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |
| Outcome: | The proposed methods identify and mitigate inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |
Copied to clipboard
| Challenge: | Existing approaches to review scientific papers are limited by their content or quality . SEA is a framework for automated scientific review, but its contents are generic or partial. |
| Approach: | They propose a framework for automated scientific review using large language models . they propose to use a standardized review dataset to fine-tune an LLM to generate high-quality reviews. |
| Outcome: | The proposed framework can generate high-quality reviews from standardized datasets and improves on the existing feedback mechanisms. |
Copied to clipboard
| Challenge: | Lipid nanoparticles (LNPs) can deliver cargos to tumor and immune cells . traditional approaches rely on experimental screening and expert judgment . |
| Approach: | They propose a method to generate lipid molecules efficiently and actively using deep learning. |
| Outcome: | The proposed method outperforms baseline methods on multiple cell lines and achieves a 30% improvement over the current methods. |
Copied to clipboard
| Challenge: | Existing methods for long-form story generation rely on rigid outlines or lack macro-level planning, making it difficult to achieve contextual consistency and coherent plot development. |
| Approach: | They propose a Dynamic Hierarchical Outlining with Memory-Enhancement long-form story generation method to generate long-formed story with coherent content and plot. |
| Outcome: | The proposed method significantly improves the fluency, coherence, and overall quality of generated long stories compared to state-of-the-art methods. |
Copied to clipboard
| Challenge: | Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences. |
| Approach: | They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences. |
| Outcome: | The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time. |
Copied to clipboard
| Challenge: | Data-to-text annotations can be costly when dealing with tables with nontrivial structures. |
| Approach: | They propose a procedure for extracting semantic triples from tables that encodes their structures by exploiting table headers and table title. |
| Outcome: | The proposed method exploits the semantic dependencies between table headers and title to extract semantic triples from tables. |
Copied to clipboard
| Challenge: | Multi-agent systems fail to consistently outperform strong single-a agent baselines due to error propagation at inter-aggent message handoffs. |
| Approach: | They propose an edge-level error taxonomy that identifies four main errors in multi-agent interactions as data gaps, signal corruption, referential drift and capacity gaps as primary sources of failure. |
| Outcome: | The proposed module outperforms existing systems on five benchmarks and is architecture-agnostic. |
Copied to clipboard
| Challenge: | a recent study shows that late-interaction methods trade off retrieval accuracy and efficiency by exploiting cross-modal interactions only in the late stage. |
| Approach: | They propose an inflating and shrinking approach to exploit cross-modal interactions . they inflate code inputs and shrink code outputs to exploit interactions progressively . |
| Outcome: | The proposed method exploits cross-modal interactions in the late stage to achieve retrieval speed. |
Copied to clipboard
| Challenge: | generating puns with artificial intelligence techniques requires manual training and templates. |
| Approach: | They propose neural network models for homographic pun generation that can generate puns without requiring any pun data for training. |
| Outcome: | The proposed models generate homographic puns of good readability and quality without training. |
Copied to clipboard
| Challenge: | Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages . |
| Approach: | They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder . |
| Outcome: | The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities. |
Copied to clipboard
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a global audience, so alignment must extend to cultural resonance. |
| Approach: | They propose a framework that frames alignment as a conditional capacity separation problem. |
| Outcome: | The proposed framework outperforms both dense baselines and semantic-only MoEs on three large language models. |
Copied to clipboard
| Challenge: | Full-duplex spoken dialogue systems allow simultaneous bidirectional communication . low latency and natural interactions in full-duplice systems remains a challenge . |
| Approach: | They propose a multi-stage post-training scheme that adapts a text large language model into a speech-text dialogue LLM. |
| Outcome: | The proposed model can model human conversation behaviors with low latency and natural interactions with low delay. |
Copied to clipboard
| Challenge: | Recent advances focus on improving DRL-based dialogue policy optimization. |
| Approach: | They propose to design a graph neural network structure that is better suited for dialogue management. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in 18 tasks of the PyDial benchmark. |
Copied to clipboard
| Challenge: | Existing approaches to safety alignment of large language models rely on costly manual annotations or human review. |
| Approach: | They propose a closed-loop reinforcement learning framework called TriPlay-RL that enables iterative collaboration among three roles with near-zero manual annotation. |
| Outcome: | The proposed framework achieves 20%–50% improvement in adversarial effectiveness while preserving high output diversity while achieving 10%–30% gains in safety performance without degrading general reasoning capability. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) fail to identify information gaps across diverse symptoms. |
| Approach: | They propose a Knowledge Graph-augmented LLM with active in-context learning to generate relevant and important follow-up questions. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 5% - 8% on relevant benchmarks. |
Copied to clipboard
| Challenge: | Existing defense mechanisms lack theoretical robustness guarantees and perform unreliably when the LLM has limited knowledge of the retrieved content. |
| Approach: | They propose a provably robust retrieval aggregation algorithm designed to defend against poisoning attacks on retrieved texts. |
| Outcome: | Experiments show that PRA-RAG reduces the attack success rate to as low as 1% while maintaining an accuracy of 71%, significantly outperforming representative state-of-the-art (SOTA) methods. |
Copied to clipboard
| Challenge: | Existing reward models produce scalar scores and struggle to incorporate critiques in a natural language format. |
| Approach: | They propose a framework that predicts critiques and rewards using self-generated critiques without extra supervision. |
| Outcome: | The proposed framework improves reward modeling accuracy by 3.7%-7.3% compared to standard reward models and LLM judges. |
Copied to clipboard
| Challenge: | In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed . |
| Approach: | They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs. |
| Outcome: | The proposed model outperforms state-of-the-art models on four real-world chat logs. |
Copied to clipboard
| Challenge: | Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities. |
| Approach: | They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC . |
| Outcome: | The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way. |
Copied to clipboard
| Challenge: | Existing approaches to LLM reuse treat LLMs as monolithic artifacts. |
| Approach: | They propose to recompose pretrained large language models as modular building blocks . they propose a chromosome-based architectural encoding and evolutionary optimization . |
| Outcome: | The proposed model can be recomposed as modular building blocks without training data. |
Copied to clipboard
| Challenge: | Existing pruning methods rely on sequential revisions and unreliable critique signals . Existing methods fail to detect the loss of answer-critical data . |
| Approach: | They propose a table pruning framework which transforms table pruning to gold trajectory-supervised parallel search. |
| Outcome: | The proposed framework outperforms the strongest baseline pruning framework by 3.2% on various tabular reasoning tasks. |
Copied to clipboard
| Challenge: | Search agents are a pivotal paradigm for solving open-ended, knowledge-intensive reasoning tasks. |
| Approach: | They propose a search agent simulation environment that bootstraps robust search agents using Reinforcement Learning. |
| Outcome: | The proposed model outperforms the web-enhanced ASearcher model by 10.6%. |
Copied to clipboard
| Challenge: | Current reward models for reinforcement learning (RL) rely on outcome rewards that propagate a single scalar value across all tokens based on final correctness. |
| Approach: | They propose a framework that derives dense token-level supervision from LLMs . they use a multi-granularity calibration mechanism to modulate teacher influence . |
| Outcome: | The proposed framework evaluates teacher reliability across problem-level expertise, trajectory-level discrimination, and token-level confidence. |
Copied to clipboard
| Challenge: | Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. |
| Approach: | They propose a new approach to learn cross-lingual cross-modal representations for matching images and captions in multiple languages using an annotated corpus. |
| Outcome: | The proposed model achieves impressive performance on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions. |
Copied to clipboard
| Challenge: | under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors. |
| Approach: | They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients . |
| Outcome: | The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Large Vision Language Model (LVLMs) exhibit advanced proficiency in language reasoning and comprehension across a wide array of languages. |
| Approach: | They propose to use a multitask language understanding benchmark specifically designed for the Malay language to assess their proficiency. |
| Outcome: | The proposed model performs well in well-resourced languages, but in low-resource languages such as Bahasa Melayu, they are less studied due to a lack of studies and benchmarks. |
Copied to clipboard
| Challenge: | TKGF is a technique that requires experience during testing and relying on a single short-term history. |
| Approach: | They propose a framework that integrates dynamic causal rule mining and dual history augmented generation to enhance event prediction. |
| Outcome: | The proposed framework shows significant performance improvements across diverse datasets and significantly improves Hit@k. |
Copied to clipboard
| Challenge: | Existing approaches focus on textual data and voting records to induce political actors' stances. |
| Approach: | They propose a Political Actor Representation learning framework that leverages social context and expert knowledge to model ideological stances. |
| Outcome: | The proposed framework improves political text understanding and improves roll call vote prediction and political perspective detection. |
Copied to clipboard
| Challenge: | NVInfo AI is a generative AI agent that can be deployed in production without full-scale retraining or infrastructure overhauls. |
| Approach: | They propose to implement a retrieval-augmented generation (RAG)-driven data flywheel in NVInfo AI, a mixture-of-experts knowledge assistant, for 30,000 employees. |
| Outcome: | The proposed system addresses failures in retrieval-augmented generation pipelines and enables continuous learning. |
Copied to clipboard
| Challenge: | Existing LLM-based medical question answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice. |
| Approach: | They propose a framework that facilitates the design and evaluation of LLM citations for medical tasks and a retrieval-citation method that generates high-quality citation. |
| Outcome: | The proposed method achieves superior citation precision and recall improvements compared to strong baseline methods and correlates well with annotation results from professional experts. |
Copied to clipboard
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
Copied to clipboard
| Challenge: | elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored. |
| Approach: | They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency . |
| Outcome: | The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training. |
Copied to clipboard
| Challenge: | Existing approaches perform significantly worse in unseen environments compared to seen ones. |
| Approach: | They propose to use a ‘environmental dropout’ method to generate unseen triplets to generate new paths and instructions to generalize the agent. |
| Outcome: | The proposed agent outperforms the state-of-the-art approaches on the private unseen test set and is ranked top on the leaderboard. |
Copied to clipboard
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
Copied to clipboard
| Challenge: | Word-level alignment in speech-text pretraining models is limited by word-level annotated data . authors propose an iterative training method for USDP that reduces the dependency on scarce annotation resources. |
| Approach: | They propose an Unsupervised Speech-text word-level alignment with Dynamic Programming (USDP) this method uses Dynamic programming principles to iteratively refine temporal alignment predictions . |
| Outcome: | The proposed method significantly improves on speech-text pretraining tasks compared to existing methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) lack robustness in knowledge-intensive tasks due to noisy or irrelevant retrieved data. |
| Approach: | They propose a multi-agent debate-based RAG framework that integrates external knowledge sources into large language models to improve their accuracy. |
| Outcome: | The proposed framework is unsupervised and leverages pretrained LLMs without fine-tuning, making it easily adaptable to various tasks. |
Copied to clipboard
| Challenge: | Pre-trained protein language models have been used in protein engineering, but their adoption is limited due to data collection, task benchmarking, and application challenges. |
| Approach: | They propose a versatile engine that integrates biological data retrieval, standardized task benchmarking, and modular fine-tuning of PLMs. |
| Outcome: | The proposed engine integrates biological data retrieval, task benchmarking, and modular fine-tuning of PLMs. |
Copied to clipboard
| Challenge: | Existing methods for document-level multi-event extraction neglect the fine-grained difference between events in multi-documents, which leads to event confusion and missing. |
| Approach: | They propose an event-specific probe-based method to sniff multiple events by querying each corresponding argument library. |
| Outcome: | The proposed method outperforms the state-of-the-art method in the recall of multi-events. |
Copied to clipboard
| Challenge: | Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization. |
| Approach: | They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree. |
| Outcome: | The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%. |
Copied to clipboard
| Challenge: | Existing methods treat each span token equally important, ignoring significant features. |
| Approach: | They propose a span-based joint extraction framework with attention-based semantic representations that utilizes span-specific and contextual representations. |
| Outcome: | The proposed model outperforms existing models on ACE2005, CoNLL2004 and ADE. |