Papers by Qian Li
Copied to clipboard
| Challenge: | Existing work on robustness in text or code tasks has focused on classification, while robustness for code generation tasks is an uncharted area. |
| Approach: | They propose a robustness evaluation benchmark for code generation models that customizes over 30 transformations specifically for code on docstrings, function and variable names, code syntax, and code format. |
| Outcome: | The proposed model performs better on human annotators and on SOTA models with human annnotators. |
Copied to clipboard
| Challenge: | Existing neural audio codecs are not capable of handling multi-domain audio data . et al., 2023) integrate speech modality with text-based large language models . |
| Approach: | They propose a unified audio codec with a single codebook to support multi-domain audio data . they propose combining a mix-of-experts strategy and a partitioned domain-adaptive codebook method . |
| Outcome: | The proposed codec outperforms existing codecs on acoustic and semantic representation capabilities. |
Copied to clipboard
| Challenge: | Existing systems cannot automatically determine when to switch between modes based on text content. |
| Approach: | They propose a unified framework that implicitly infers vocal modes from text context to pioneer SCS Synthesis. |
| Outcome: | The proposed framework infers vocal modes solely from text context to pioneer SCS Synthesis. |
Copied to clipboard
| Challenge: | emergence of tool agent paradigm has broadened capability boundaries of the Large Language Model (LLM) but effectiveness of tool agents limited due to parameter failure during execution . |
| Approach: | They propose a parameter failure taxonomy to investigate parameter failure . they propose suggestions for standardizing tool return formats and improving error feedback mechanisms . |
| Outcome: | The proposed model is based on a tool agent invocation chain and a mainstream tool agent . it shows that parameter name hallucination failure stems from inherent limitations . |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have catalyzed the development of autonomous agents capable of executing complex, multi-turn tasks. |
| Approach: | They propose a framework for agentic reinforcement learning that integrates turn-level tree search with tree search to address key challenges. |
| Outcome: | The proposed framework addresses key challenges: limited exploration diversity, sparse credit assignment, and misaligned policy optimization. |
Copied to clipboard
| Challenge: | Reinforcement learning (RL) has improved text- and vision-language models, but its application in SDMs is hindered. |
| Approach: | They propose a dual-axis Generative Reward Model that provides semantic quality and interaction timing for SDMs. |
| Outcome: | The proposed model achieves state-of-the-art performance on interaction-quality assessment across a wide spectrum of datasets. |
Copied to clipboard
| Challenge: | Existing RS agents built on general-purpose LLMs are domain-agnostic, resulting in brittle and error-prone workflows. |
| Approach: | They propose a knowledge-enhanced memory evolution mechanism that bootstraps RS agents with pre-distilled domain knowledge and iteratively integrates online experience for robust multi-step tool execution. |
| Outcome: | Experiments show that the new model improves tool-use performance and accuracy . iteratively, iteration of the model integrates online experience for robust multi-step tool execution . |
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks . |
| Approach: | They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency . |
| Outcome: | The proposed benchmark covers 43 programming languages and eight coding tasks. |
Copied to clipboard
| Challenge: | Existing studies on vision-language models aligned with general human objectives have not been successful because people with diversified backgrounds have different cognition even in the same situation. |
| Approach: | They propose to characterize individuals based on the sociological concept of Role-Set and then evaluate their actions to see whether personalized alignment is achieved. |
| Outcome: | The proposed framework constructs a cognition-aware and action-based reward model for personalized alignment. |
Copied to clipboard
| Challenge: | Scheduling logistics associated with meetings can amount to hours each week . a lack of automation can cause a lot of errors and can be costly . |
| Approach: | They propose a benchmark for long-horizon calendar conflict resolution that automates calendar conflict management . they propose 'pearl' that augments the language agent with an external preference memory . |
| Outcome: | a new benchmark for long-horizon calendar conflict resolution shows that existing agents perform poorly with high error rates. |
Copied to clipboard
| Challenge: | Existing methods waiting-and-translating for a fixed duration break speech acoustic units . Existing models waiting-for a set duration and generating partial sentences are not effective . |
| Approach: | They propose a monotonic segmentation module inside an encoder-decoder model to detect proper speech unit boundaries for a streaming speech input. |
| Outcome: | The proposed method outperforms existing methods on a speech translation dataset and achieves the best trade-off between translation quality and latency. |
Copied to clipboard
| Challenge: | Existing frameworks that increase context window do not guarantee robust performance across long input tasks. |
| Approach: | They propose a framework that enables language models to handle extended inputs within limited context windows efficiently. |
| Outcome: | The framework improves performance on long-document question-answering and Needle-in-a-Haystack tasks. |
Copied to clipboard
| Challenge: | Existing multi-agent systems lack agent coordination and rely on predefined procedures . existing systems lack adaptive task coordination when task is big and complex . |
| Approach: | They propose a large-scale autonomous LLM-based multi-agent system that generates agents based on task complexity and enables dynamic task decomposition, parallel execution, efficient communication and comprehensive system monitoring. |
| Outcome: | The proposed system outperforms existing systems in task completion efficiency and scalability. |
Copied to clipboard
| Challenge: | Existing models struggle with complex queries, especially multi-table joins and reasoning. |
| Approach: | They propose to build a model with synthetic training samples and a structure-aware curriculum learning framework for enhancing SQL generation. |
| Outcome: | The proposed model improves on the existing model on the Spider and Bird benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for GMNER fail to address semantic ambiguity caused by polysemy and long-tail distribution of datasets. |
| Approach: | They propose a framework for Grounded Multimodal Named Entity Recognition that leverages a Multimodal Large Language Model to address semantic ambiguity. |
| Outcome: | Extensive experiments show that the proposed framework outperforms existing methods on two benchmark datasets. |
Copied to clipboard
| Challenge: | Keyphrase extraction (KPE) extracts phrases in a document that provide a concise summary of the core content. |
| Approach: | They propose an unsupervised keyphrase extraction method that ranks candidates by similarity between embeddings of source document and masked document. |
| Outcome: | The proposed method outperforms state-of-the-art methods on six benchmarks . it achieves average 3.53 improvement over the existing method . |
Copied to clipboard
| Challenge: | Existing methods for multimodal named entity recognition are limited due to limited resources. |
| Approach: | They propose a Few-shot Multimodal Named Entity Recognition task to address these relation types by constructing a multimodal graph and a new multimodal causal intervention strategy. |
| Outcome: | The proposed model improves on two multimodal named entity recognition datasets. |
Copied to clipboard
| Challenge: | Existing retrieval-augmented code generation methods fail to accurately fetch the knowledge required for code generation for consecutive code fragments. |
| Approach: | They propose a paradigm that enables large language models to Self-express their information needs to enhance retrieval-augmented code generation methods. |
| Outcome: | Experiments show that SelfRACG can retrieve external knowledge that better aligns with the LLM’s own information needs, resulting in superior generation performance compared to vanilla RACG. |
Copied to clipboard
| Challenge: | Recent studies on AMR-to-text generation formalize the task as a sequence-tosequence learning problem . previous approaches only consider the relations between directly connected concepts while ignoring the rich structure in AMR graphs. |
| Approach: | They propose a structure-aware self-attention approach to model the relations between indirectly connected concepts in the seq2seq model. |
| Outcome: | The proposed approach outperforms the state-of-the-art on English AMR benchmarks . it significantly outperformed the state of the art on the benchmarks, with 29.66 and 31.82 BLEU scores . |
Copied to clipboard
| Challenge: | Existing approaches to learning KG triplets ignore ternary propagation patterns and ignore zero-shot, few-shot and synonymity problems. |
| Approach: | They propose a framework for contrastive learning based on ternary propagation patterns among head, relation and tail. |
| Outcome: | Experiments on benchmarks show that TernaryCL is superior to state-of-the-art models. |
Copied to clipboard
| Challenge: | Existing efforts to generate Wikipedia articles for new events fall short of real-world application. |
| Approach: | They propose a benchmark to generate Wikipedia articles for new events under real-world scenarios . they use systematic metrics and LLM-based metrics to assess verifiability, organization, and other aspects aligned with real-life scenarios. |
| Outcome: | The proposed benchmarks show that hierarchical-based methods generate more comprehensive content while fine-tuned methods achieve better verifiability. |
Copied to clipboard
| Challenge: | Existing methods to use table pre-training to boost tabular prediction performance remain open . a bachelor's degree earns less than 50K, and a generative LM can be used to unify tasks via one LM. |
| Approach: | They propose a method that leverages table pre-training to empower tabular prediction models. |
| Outcome: | The proposed method outperforms baseline models on 12 datasets and can be easily combined with various backbone models. |
Copied to clipboard
| Challenge: | Existing text-to-SQL approaches have overlooked the critical aspect of system robustness. |
| Approach: | They propose a robust text-to-SQL solution that integrates with LLMs . their method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% . |
| Outcome: | The proposed solution achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have a tendency to hallucinate and provide creative and fluent responses that are not factually accurate. |
| Approach: | They propose a tool that automatically extracts factual claims from text, gathers evidence from external knowledge sources, evaluates the factuality of each claim, and suggests revisions for identified errors. |
| Outcome: | The proposed tool detects errors in text and evaluates their factuality and suggests revisions based on the collected evidence. |
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) is a critical task that aims to identify equivalent entity pairs across multi-modal knowledge graphs (MMKGs). |
| Approach: | They propose a novel MMEA transformer that hierarchically introduces neighbor features, multi-modal attributes, and entity types to enhance alignment task. |
| Outcome: | The proposed transformer hierarchically introduces neighbor features, multi-modal attributes, and entity types to enhance the alignment task. |
Copied to clipboard
| Challenge: | Document-level event factuality identification is crucial for discourse understanding in NLP . identifying document-level factual of events requires comprehensive understanding of documents . |
| Approach: | They propose to construct a corpus annotated with document- and sentence-level event factuality information on English and Chinese texts. |
| Outcome: | The proposed model outperforms baselines on the constructed corpus. |
Copied to clipboard
| Challenge: | Existing methods focus on manipulating entity features to find pairwise relations, yet neglect the more fundamental structural information that links disparate entity pairs together. |
| Approach: | They propose a Visual Relation Extraction framework that generates relation predictions on entity pairs extracted from scanned images and incorporates global structural knowledge into the representations of the entities. |
| Outcome: | The proposed framework outperforms existing methods in fine-tuning setting and yields stronger data-efficient performance in the low-resource setting. |
Copied to clipboard
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Copied to clipboard
| Challenge: | Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time. |
| Approach: | They propose a framework that reframes language modeling as next-state prediction under interaction. |
| Outcome: | The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics . |
Copied to clipboard
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Copied to clipboard
| Challenge: | Existing approaches to search for images using single-modality are limited by representation space fragmentation. |
| Approach: | They propose a unified representation framework that achieves efficient query-target alignment . they introduce a multi-level Chain-of-Thought prompting strategy that guides MLMs to generate discriminative, semantically compatible captions for target images . |
| Outcome: | The proposed framework achieves efficient query-target alignment through synergistic components. |
Copied to clipboard
| Challenge: | Existing methods for inductive knowledge Graphs are limited by sparsity and implicit transfer. |
| Approach: | They propose a Contrastive Learning framework with graph guided Variational autoencoder on Meta-KGs to capture and transfer entities. |
| Outcome: | The proposed framework outperforms state-of-the-art methods with extensive experiments. |
Copied to clipboard
| Challenge: | Existing medical reasoning datasets are limited in scale and typically rely on incomplete data. |
| Approach: | They propose to use ReasonMed to train medical reasoning models using a multi-agent generation, verification, and refinement pipeline. |
| Outcome: | The largest medical reasoning dataset to date surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. |
Copied to clipboard
| Challenge: | Recent advances leverage large language models (LLMs) for legal reasoning, but they face high computational costs and information degradation when handling long cases. |
| Approach: | They propose a framework that selectively retains legally relevant information while reducing redundant or less informative content, enabling efficient and accurate long-context reasoning. |
| Outcome: | The proposed framework outperforms existing methods on four real-world datasets spanning multiple jurisdictions and languages. |
Copied to clipboard
| Challenge: | Document assistant chatbots are empowered with extensive capabilities by Large Language Models (LLMs) however, they suffer from hallucinations that are difficult to verify in the context of given documents. |
| Approach: | They propose a document assistant chatbot with reliable attribution that enables users to seek relevant information from given documents. |
| Outcome: | The proposed system generates answers with detailed inline citations, which can be attributed to the original document paragraphs, facilitating verification of factual consistency of the generated text. |
Copied to clipboard
| Challenge: | In Natural Language Interfaces to Databases systems, text-to-SQL parsers allow users to query databases by using natural language questions. |
| Approach: | They propose a parser-independent interactive approach that interacts with users using multi-choice questions and can easily work with arbitrary parsers. |
| Outcome: | The proposed approach improves performance with limited interaction turns by using simulation and human evaluation on two cross-domain datasets with five state-of-the-art parsers. |
Copied to clipboard
| Challenge: | Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps. |
| Approach: | They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels . |
| Outcome: | The proposed benchmark aims to bridge symbolic reasoning and factual verification. |
Copied to clipboard
| Challenge: | aaron carroll: the precise localization of non-verbal vocal events remains a critical yet under-explored challenge. carroll says current methods suffer from insufficient task definitions with limited category coverage. carrol: knowing exactly where an event occurred is not enough; knowing exactly what it happened is. |
| Approach: | They propose a taxonomy of 21 vocal events with a new categorization into discrete versus continuous types. |
| Outcome: | The proposed model disentangles ASR errors from event detection while maintaining ASR quality. |
Copied to clipboard
| Challenge: | Existing studies have isolated LLM jailbreak attacks and defenses . a new framework combines attack and defense to protect against malicious queries . |
| Approach: | They propose a framework that combines attack and defense to protect the Large Language Model (LLM) by embedding harmful problems into the safe area. |
| Outcome: | The proposed framework achieves an average jailbreak success rate of 88.85% across three popular LLMs while the defense success rate reaches an average of 84.17%. |
Copied to clipboard
| Challenge: | Existing dialogue systems do not exploit document knowledge effectively enough. |
| Approach: | They propose a Transformer-based architecture for document grounded conversations that incorporates document knowledge into a two-pass decoder to improve context coherence and knowledge correctness. |
| Outcome: | The proposed model outperforms baselines on context coherence and knowledge relevance on a real-world document grounded dataset. |
Copied to clipboard
| Challenge: | Existing evaluation frameworks focus on legal professionals, not legal professionals. |
| Approach: | They propose a public-oriented LegalAI benchmark grounded in legal functionalism and genre analysis to address this gap. |
| Outcome: | The proposed model evaluates 17 large language models on Pub-LawBench using simple prompts and Chain-of-Thought under a vanilla inference setting. |
Copied to clipboard
| Challenge: | Recent work has shown that large pretrained Language Models (LMs) can perform remarkably well on a range of NLP tasks but they have limitations on basic symbolic manipulation tasks such as copy, reverse, and addition. |
| Approach: | They propose to use explicit positional markers, fine-grained computation steps, and LMs with callable programs to teach large pretrained Language Models. |
| Outcome: | The proposed model can perform 100% accuracy in OOD and repeating symbols. |
Copied to clipboard
| Challenge: | a recent study evaluated large audio-language models against jailbreak attacks . a new benchmark is being developed to evaluate LAM safety against jailbreaking attacks based on temporal and semantic nature of speech . |
| Approach: | They propose a benchmark to evaluate LAM jailbreak vulnerabilities in adversarial audio prompts . they use a dataset of 1,495 adversarials to evaluate their performance . |
| Outcome: | The proposed benchmark evaluates state-of-the-art LAMs against jailbreak attacks . it demonstrates that even small, semantically preserved perturbations can reduce safety . |
Copied to clipboard
| Challenge: | Existing methods for continual relation extraction (CRE) are rehearsal-based and need to store samples and thus may encounter privacy and security issues. |
| Approach: | They propose an Ensemble-of-Experts framework for rehearsal-free continual relation extraction that discriminates between experts and augments analogous relations across tasks. |
| Outcome: | The proposed method outperforms existing rehearsal-free methods and is even better than existing methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved satisfactory performance in counterfactual generation, however, there are misalignments between LLMs and humans which hinder LLM from handling complex tasks like relation extraction. |
| Approach: | They propose to mimic the episodic memory retrieval mechanism of human hippocampus to align LLMs’ generation process with that of humans. |
| Outcome: | The proposed framework improves over existing methods in terms of quality of counterfactuals. |
Copied to clipboard
| Challenge: | Structured representations of entity names are useful for many entity-related tasks such as entity normalization and variant generation. |
| Approach: | They propose a framework that combines active learning and weak supervision to solve this problem. |
| Outcome: | The proposed framework enables learning of high-quality models from a dozen labeled examples. |
Copied to clipboard
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
Copied to clipboard
| Challenge: | Existing models for knowledge editing focus on knowledge-level or static visual domains, overlooking dynamic semantics. |
| Approach: | They propose a benchmark for modeling large language models using six representative models . they analyze the strengths and limitations of existing models and identify new directions . |
| Outcome: | The proposed benchmark extends existing models from static modalities to dynamic video scenarios. |
Copied to clipboard
| Challenge: | Existing methods to detect identity fraud are prone to errors and are not based on real data. |
| Approach: | They propose to use a KG constructor and structured dialogue management to detect identity fraud in loan applications to generate questions based on personal information. |
| Outcome: | The proposed system can detect fraudsters and achieve higher recognition accuracy compared with rule-based systems. |
Copied to clipboard
| Challenge: | Existing methods for temporal reasoning are limited and apply a fixed pipeline to all questions. |
| Approach: | They propose an adaptive temporal reasoning method that dynamically executes reasoning steps based on context and task requirements. |
| Outcome: | Experiments on two temporal QA benchmarks show the proposed method works. |
Copied to clipboard
| Challenge: | Existing knowledge editing methods rely on unidirectional, feed-forward pipelines . a minor retrieval error or logical mismatch at an early hop can become a silent failure . |
| Approach: | They propose a framework for closed-loop post-edit reasoning that uses a Critic agent to verify coherence and step-wise correctness. |
| Outcome: | Experiments on MQuAKE-2002 and MQuADE-hard show that CARE effectively mitigates error propagation . a minor retrieval error or logical mismatch at an early hop can become a silent failure . |
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
Copied to clipboard
| Challenge: | Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making. |
| Approach: | They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks. |
| Outcome: | The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks. |
Copied to clipboard
| Challenge: | Document-level Event Factuality Identification (DEFI) is a fundamental and crucial task in NLP. |
| Approach: | They propose a framework for document-level event factuality identification (DEFI) they propose to use Span-Extraction and Multiple-Choice to model DEFI as machine reading comprehension tasks . |
| Outcome: | The proposed model outperforms state-of-the-art models on a document-based event factuality task . it uses Span-Extraction (Ext) and Multiple-Choice (Mch) knowledge to extract knowledge from large-scale MRC corpus . |
Copied to clipboard
| Challenge: | Existing task-oriented dialogue systems cannot guarantee that all user needs are taken into account in the design phase. |
| Approach: | They propose a new incremental learning framework to design task-oriented dialogue systems without pre-defining user needs. |
| Outcome: | The proposed framework is robust to unconsidered user actions and can update itself online with less annotation cost. |
Copied to clipboard
| Challenge: | Existing models rely on a single segmentation token whose hidden state implicitly encodes both semantic reasoning and spatial localization . Existing methods rely only on SEG>, which encodes semantic reasoning, limiting the model's ability to explicitly disentangle what to segment from where to segment. |
| Approach: | They propose a method which reformulates reasoning segmentation as a structured conditional generation process over image tokens conditioned on language grounded query banks. |
| Outcome: | The proposed model bridges token-level predictions and pixel-level supervision by decoupling spatial grounding from semantic reasoning through structured language grounded query banks. |
Copied to clipboard
| Challenge: | Existing methods for multi-modal relation extraction lack useful visual information. |
| Approach: | They propose a novel multi-modal relation extraction framework to capture deeper correlations of text, entity pair, and image/objects. |
| Outcome: | The proposed framework captures the deeper correlations of text, entity pair, and image/objects, and extracts useful information. |
Copied to clipboard
| Challenge: | Recent reinforcement learning (RL) has been widely explored for adapting MCAs to various human-AI interaction scenarios. |
| Approach: | They propose to use a latent action space for reinforcement learning instead of RL to fine-tune MCAs. |
| Outcome: | The proposed method outperforms baselines on two conversation tasks with a novel cycle consistency loss. |
Copied to clipboard
| Challenge: | Existing approaches to retrieval-augmented generation rely on fragment-level retrieval . GraphRAG suffers from inefficiencies in information extraction and costly resource consumption . |
| Approach: | They propose a tag-guided hierarchical knowledge graph RAG framework for efficient global reasoning and scalable graph maintenance. |
| Outcome: | GraphRAG achieves an average win rate of 78.36% on a dataset spanning agriculture, computer science, law, and cross-domain settings compared with baselines . |
Copied to clipboard
| Challenge: | Existing language model agents excel in planning and reasoning, but lack creativity in unfamiliar environments. |
| Approach: | They propose a benchmark suite of room escape game environments to challenge agents with creative reasoning, unconventional tool use and iterative problem-solving to uncover implicit goals. |
| Outcome: | The proposed framework can perform with 40% fewer steps and hints and performs robustly across difficulty levels. |
Copied to clipboard
| Challenge: | Existing studies on outline-conditioned text generation focus on generating text using provided outlines as rough sketches, but lack of clarity and rationality of the rough outlines hampers quality of the generated text. |
| Approach: | They propose a novel task that requires generating stories based on specific, sentence-level outlines. |
| Outcome: | The proposed framework improves the quality of precise outline-conditioned text generation. |
Copied to clipboard
| Challenge: | Existing studies show that the design of Meaning Representation (MR) greatly influences the final model performance of a neural semantic parser. |
| Approach: | They propose a data-aware metric called ISS to measure the final performance of MRs. |
| Outcome: | The proposed metric denoting incremental structural stability (ISS) of MRs can be used as an indicator for MR design to avoid the costly training-testing process. |
Copied to clipboard
| Challenge: | Existing approaches to fine-tuning language models use zeroth-order optimizers to conserve GPU memory. |
| Approach: | They propose a full-parameter fine-tuning strategy which updates a subset of parameters at each training step. |
| Outcome: | The proposed approach reduces the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage. |
Copied to clipboard
| Challenge: | Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information. |
| Approach: | They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format. |
| Outcome: | The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots. |
Copied to clipboard
| Challenge: | Existing home assistants struggle to interpret elliptical commands based on ellipine expressions . current assistants overlook the progressive omission that occurs in human dialogue as context accumulates - limiting their effectiveness in real-world applications . |
| Approach: | They propose a simulated home dataset specifically designed for interpreting progressively elliptical commands in smart homes. |
| Outcome: | The proposed dataset shows that existing home assistants struggle to execute user-intended operations based solely on elliptical commands. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly aligned with human preferences through Reinforcement Learning from Human Feedback (RLHF). |
| Approach: | a new study proposes a domain-informed self-consistency policy optimization extension to GRPO that addresses inter-group imbalance. |
| Outcome: | a new extension of GRPO addresses inter-group imbalance with two key innovations . the proposed method outperforms existing GR PO variants by 5% on Qwen3 models . |
Copied to clipboard
| Challenge: | Experiments on GPT and other 23 LLMs indicate that tokens widely exist while GPT’s vocabulary behaves the worst: more than 23% long Chinese tokens (i.e., a token with more than two Chinese characters) are either porn or online gambling. |
| Approach: | They propose to locate Polluted Chinese (PoC) tokens in LLMs and build a PoC token detector to label them in vocabularies by considering each token’s semantics and related contents from the search engines. |
| Outcome: | The proposed method predicts that the ratio of “*” related webpages in GPT-4o's training data is around 0.5%. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are reshaping recommender systems by leveraging extensive world knowledge and semantic reasoning to interpret user intent. |
| Approach: | They propose a single-agent Trajectory-Aligned Recommender to integrate reasoning capabilities into a model by a multi-agend teacher system. |
| Outcome: | The proposed model surpasses its teacher by 8.7% to 39.5% while eliminating iterative latency. |
Copied to clipboard
| Challenge: | Existing methods for memory management struggle to capture fine-grained semantic relations between queries and documents. |
| Approach: | They propose a framework for reasoning and agentic search that grows fine-grained memory fragments from seed tokens from queries, then retraces and deep refines the memory via a contribution function. |
| Outcome: | Experiments on eight benchmark datasets show that MemSearch-o1 significantly mitigates memory dilution and more effectively activates reasoning potential of diverse LLMs. |
Copied to clipboard
| Challenge: | a survey of large language models (LLMs) aims to ensure outputs adhere to human values, ethical standards, and legal norms. |
| Approach: | They present the first systematic review of TF alignment methods . they categorize them by stages of pre-decoding, in-decoder and post-decoration . |
| Outcome: | The proposed methods are based on training-free (TF) alignment techniques . they are able to be used in open-source and closed-source environments without retraining . |
Copied to clipboard
| Challenge: | Existing locate-and-edit knowledge editing methods suffer from two limitations: they are infeasible for large scale KE in practice and require long run-time. |
| Approach: | They propose to use parametric fine-tuning techniques to update obsolete knowledge and induce new knowledge into LLMs. |
| Outcome: | The proposed methods improve the performance of KE and knowledge update in a temporal dataset with knowledge update and knowledge injection examples. |
Copied to clipboard
| Challenge: | Existing approaches to longterm memory rely on rigid retrieval granularity, accumulation-heavy maintenance strategies, and coarse-grained update mechanisms. |
| Approach: | They propose a framework that leverages coordinated agents to manage memory across multiple granularities. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks while reducing token consumption by approximately 80%. |
Copied to clipboard
| Challenge: | In implicit sentiment analysis, the opinion cues come in an implicit and obscure manner. |
| Approach: | They propose a three-step prompting principle for THOR to step-by-step induce the implicit aspect, opinion and finally the sentiment polarity. |
| Outcome: | The proposed framework pushes the state-of-the-art (SoTA) by over 6% F1 on supervised setup and more strikingly, boosts the SoTA by over 50% F1 with THOR+GPT3. |
Copied to clipboard
| Challenge: | RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues spanning over 12 domains . despite of substantial progress made, there are challenges in creating challenging datasets in terms of size, multiple domains, semantic annotations and complexity. |
| Approach: | They propose a large-scale multi-domain Chinese Wizard-of-Oz dataset with rich semantic annotations that captures discourse phenomena for task-oriented dialogue modeling. |
| Outcome: | The proposed dataset contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues with more than 150K utterances spanning over 12 domains. |
Copied to clipboard
| Challenge: | Existing soft prompt methods focus on designing the input-independent prompts that steer the model to fit the domain of the new dataset. |
| Approach: | They propose a multi-level prompt tuning method that utilizes prompts at task-specific, domain-specific and context-specific levels to enhance the comprehension of input semantics. |
| Outcome: | The proposed method improves on 12 benchmarks on various QA formats and achieves an average improvement of 1.94% over the state-of-the-art methods. |
Copied to clipboard
| Challenge: | Autoregressive sequence modeling has been successful in many domains, but maintaining long-term coherence and structural integrity remains a challenge. |
| Approach: | They propose an ACG paradigm that relies on anchor features from previously generated musical content to guide subsequent generation during the autoregressive process. |
| Outcome: | The proposed framework outperforms existing methods in symbolic music generation tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have made remarkable progress in a wide range of natural language understanding and generation tasks, but their ability to generate counterfactuals has not been examined systematically. |
| Approach: | They propose a framework to evaluate LLMs' ability to generate counterfactuals based on key factors including intrinsic properties and prompt design. |
| Outcome: | The proposed framework examines the strengths and weaknesses of large language models (LLMs) and identifies factors that influence their ability to generate counterfactuals. |
Copied to clipboard
| Challenge: | Existing Large Language Models struggle to reason systematically under cost constraints . Existing approaches lack the knowledge-reasoning capability to reason under cost . |
| Approach: | They propose a knowledge-enhanced framework that leverages large language models to construct MDKGs . they propose three collaborative agents that handle language understanding and generation . |
| Outcome: | GraphDx improves diagnostic success rates from 50–68% to 79–93% while reducing test costs by 20–54%. |
Copied to clipboard
| Challenge: | Existing learned metrics perform unsatisfactory across text generation tasks or require human annotations for training on specific tasks. |
| Approach: | They propose a self-supervised approach to train a model-based metric for text generation evaluation using sentences retrieved from a corpus. |
| Outcome: | The proposed model outperforms all prior unsupervised metrics on four text generation evaluation benchmarks, with an average Kendall improvement of 0.158. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable achievements across various language tasks. |
| Approach: | They propose a scientific literature LLM and a knowledge service system based on it . they collect scientific literature and then pre-train it using autoregressive training . |
| Outcome: | The proposed system provides literature investigation, paper reading, and academic writing functions. |
Copied to clipboard
| Challenge: | Generative world models could be used to enhance agents' cognition . agents are expected to operate in settings where tasks unfold over long horizons and involve intricate chains of interdependent decisions. |
| Approach: | They propose to use vision-language models as external simulators to enhance cognition . they find that agents rarely invoke simulation and misuse predicted rollouts . |
| Outcome: | The proposed model could be used to predict future states rather than short-horizon reasoning . the model could also be used for real-world planning and robotics . |
Copied to clipboard
| Challenge: | Effective EHR representations are key to achieving high performance in healthcare applications. |
| Approach: | They propose a multimodal heterogeneous graph-enhanced representation learning to learn EHR representations using medical ontology and textual notes. |
| Outcome: | The proposed model outperforms baseline models on two real clinical datasets in downstream tasks. |
Copied to clipboard
| Challenge: | Existing evaluation protocols for few-shot natural language understanding (NLU) tasks are inconsistent and hinder fair comparison and measuring progress. |
| Approach: | They propose an evaluation framework that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability. |
| Outcome: | The proposed framework improves evaluation procedures in three key aspects, i.e., performance, dev-test correlation, and stability. |
Copied to clipboard
| Challenge: | Existing models follow a 'one-for-all' scheme where one general large model performs all individual N-way-K-shot tasks, which prevents the model from achieving the optimal point on each task. |
| Approach: | They propose a model generation framework that consists of one general model for all tasks and many tiny task-specific models for each individual task. |
| Outcome: | The proposed framework achieves state-of-the-art performance on two public datasets. |
Copied to clipboard
| Challenge: | Existing work on controllable natural language generation has focused on fine-tuning existing models or using attribute discriminators. |
| Approach: | They propose a lightweight framework for controllable GPT2 generation that utilizes attribute-specific vectors to steer natural language generation. |
| Outcome: | The proposed framework can guide generation towards desired attributes while keeping high linguistic quality. |
Copied to clipboard
| Challenge: | Existing web agents suffer from limited robustness, efficiency and task success due to lack of structural understanding of websites and lack of browsing priors in pre-trained models. |
| Approach: | They propose an agent-oriented sitemap protocol that integrates structured website knowledge into web agents. |
| Outcome: | The proposed agent-oriented sitemap improves robustness, efficiency and effectiveness without extra training. |
Copied to clipboard
| Challenge: | Knowledge Editing (KE) has gained increasing attention, yet current evaluation frameworks do not integrate KE into real-world application scenarios. |
| Approach: | They propose a script-based benchmark which encompasses both counterfactual and temporal edits and integrates token-level and text-level evaluation methods. |
| Outcome: | The proposed method combines token-level and text-level evaluation methods with a new fact-based evaluation framework. |
Copied to clipboard
| Challenge: | Empirical evaluations of large language models demonstrate that they improve performance in a wide range of tasks. |
| Approach: | They propose a label-free method for mitigating selection bias during inference by reformulating debiasing as an optimization task. |
| Outcome: | The proposed method mitigates selection bias and improves performance compared to existing methods. |
Copied to clipboard
| Challenge: | Mainstream speaker diarization systems rely only on acoustic information, making it challenging in complex aural environments. |
| Approach: | They propose a multimodal approach that integrates audio, visual, and semantic cues to enhance speaker diarization. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on multi-party conversations . it integrates audio-visual-semantic cues into the clustering process for acoustic speaker embeddings . |
Copied to clipboard
| Challenge: | Audio-aware large language models (ALLMs) can understand textual and non-textual information in the audio input. |
| Approach: | They use audio-aware large language models (ALLMs) to evaluate the speaking styles of SLMs on two tasks: voice style instruction following and role-playing. |
| Outcome: | The proposed models can understand the textual and non-textual information in the audio input and can be used as a judge to assess the speaking styles of SLMs. |
Copied to clipboard
| Challenge: | Structured pruning can reduce model size but results in significant accuracy degradation . quantization and pruning increase the difficulty of fine-tuning, requiring a more refined quantization scheme. |
| Approach: | They propose a structured pruning framework followed by a layer-wise mixed-precision quantization scheme to reduce model memory consumption during fine-tuning and inference. |
| Outcome: | Experiments on benchmark datasets show that QPruner outperforms existing methods in memory savings while maintaining or improving model performance. |
Copied to clipboard
| Challenge: | Recent studies show that LLMs’ intrinsic self-correction fails without oracle labels as feedback. |
| Approach: | They propose to use one simple task and three complex tasks with state-of-the-art LLMs like ChatGPT, Llama, and DeepSeek to interpret LLM's intrinsic self-correction. |
| Outcome: | The proposed methods reveal the dark side of LLMs’ intrinsic self-correction for different tasks, especially for those failure cases. |
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning (RL) suffer from training instability . existing approaches often ignore token-specific discrepancies in expert assignments . |
| Approach: | They propose to introduce expert-level importance sampling to reduce complexity of RL . they propose to leverage expert-centric granularity to ensure a rigorous alignment between reward signals and policy updates. |
| Outcome: | The proposed method outperforms strong baselines across reasoning tasks. |
Copied to clipboard
| Challenge: | Training conversational question-answering systems requires in-domain data, which is often scarce in practice. |
| Approach: | They propose a bottom-up approach where QA pairs are generated first and combined into a coherent dialogue. |
| Outcome: | The proposed approach produces more realistic and higher-quality dialogues compared to top-down methods. |
Copied to clipboard
| Challenge: | Existing methods focus on transferring in-domain (IND) prior knowledge to out-of-domain data through pre-training and clustering. |
| Approach: | They propose a Pseudo-Label enhanced Prototypical Contrastive Learning model for uniformed intent discovery that integrates supervised and pseudo signals from IND and OOD data. |
| Outcome: | The proposed method has been proven effective in two different settings of discovering new intents. |
Copied to clipboard
| Challenge: | Large language model-based multi-agent systems (MAS) are increasingly used to extend agentic problem solving via role specialization and collaboration. |
| Approach: | They propose a graph-centric framework for orchestrating large language model-based multi-agent systems . they compile a user's natural-language intent into an editable workflow specification and then into an executable graph . |
| Outcome: | The proposed framework compiles natural-language intent into an executable graph and then compile and executes it at runtime. |
Copied to clipboard
| Challenge: | Existing methods focus on graph structure learning or semantic reasoning, lacking the capability to capture the inherent differences between historical and non-historical events. |
| Approach: | They propose a temporal knowledge graph reasoning framework that integrates both structural and semantic information to guide the reasoning process for different events. |
| Outcome: | The proposed framework integrates structural and semantic information to predict future events . it can provide evidence for many downstream tasks, including situation analysis and political decision making . |
Copied to clipboard
| Challenge: | Temporal Knowledge Graph Question Answering (TKGQA) aims to answer temporal questions using knowledge in Temporal knowledge graphs (TKTs). |
| Approach: | They propose a Time-aware retrieve-rewrite-retrieve-rerank framework to integrate temporal knowledge from TKGs into Large Language Models (LLMs) to reduce temporal hallucination, they propose rewrite module to rew questions using background knowledge stored in TKG's, then implement a retrieve-rank module to retrieve semantically and temporally relevant facts from Tkgs and rerank them according to temporal constraints. |
| Outcome: | The proposed approach achieves relative gains of 47.8% and 22.5% on two datasets, underscoring its effectiveness in boosting the temporal reasoning abilities of LLMs. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | Existing methods assume that events appear in sentences without overlaps . overlapping event extraction is a challenging task in natural language understanding . |
| Approach: | They propose a joint learning framework with cascade decoding for overlapping event extraction . they sequentially perform type detection, trigger extraction and argument extraction based on the specific former prediction . |
| Outcome: | The proposed framework improves on a public event extraction benchmark . it sequentially performs type detection, trigger extraction and argument extraction . |
Copied to clipboard
| Challenge: | Diffusion language models (DLMs) offer advantages in parallel generation and bidirectional context modeling, but they face a critical trade-off between inference speed and output quality for tasks with strict structural constraints such as code generation. |
| Approach: | They propose an efficient sampling algorithm that reduces the number of tokens unmasked per step based on the model’s evolving confidence. |
| Outcome: | The proposed method improves Pass@1 accuracy by 1.9% while achieving 251.4% inference speedup. |
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods restrict entities on hyper-ellipsoid surfaces, resulting in suboptimal knowledge graph completion. |
| Approach: | They propose a score function that leverages relation-specific translations between head and tail entities to relax constraints on hyper-ellipsoid surfaces. |
| Outcome: | The proposed method achieves state-of-the-art performance on link prediction and generalizes well to datasets in different domains and scales. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) extend their capabilities through function-calling (FC) however, obtaining and annotating real function-called data is challenging, and synthetic data from existing pipelines suffers from unreliable APIs, limited tool scalability, insufficient diversity, and weak quality control. |
| Approach: | They propose a pipeline for generating FC training data using reliable tools and a multi-agent framework that supports a dialogue generation system that produces conversations spanning diverse scenarios. |
| Outcome: | The proposed pipeline outperforms open-source models in in-domain FC performance and out-of-domain generalization while reaching FC capabilities comparable to some of the latest API-based models. |
Copied to clipboard
| Challenge: | Language models (LMs) are becoming imperative tools for consulting in realworld scenarios. |
| Approach: | They propose a model editing method that initializes and retrofits key-value pairs into MLP blocks to construct a new mapping of a piece of knowledge without damaging irrelevant knowledge. |
| Outcome: | The proposed method outperforms baseline methods on a series of GPT series models on edit success and generalization without influencing specificity. |
Copied to clipboard
| Challenge: | Recent studies have successfully decoded natural language from non-invasive brain signals . current dataset splitting methods suffer from data leakage problem . |
| Approach: | They propose a right cross-subject data splitting criterion without data leakage for decoding fMRI and EEG signal to text. |
| Outcome: | The proposed method overfits and overestimates brain-to-text decoding models. |
Copied to clipboard
| Challenge: | Existing methods to generate annotated dialogues require crowdsourcing, which is expensive and time-consuming. |
| Approach: | They propose a dialogue simulation method based on large language model in-context learning that generates new dialogues and annotations in a controllable way. |
| Outcome: | The proposed method can expand a small set of dialogue data with minimum or zero human involvement and parameter update. |
Copied to clipboard
| Challenge: | FineState-Bench evaluates whether an agent can correctly ground an instruction to the intended UI control and reach the exact target state. |
| Approach: | They propose a benchmark that evaluates whether an agent can correctly ground an instruction to the intended UI control and reach the exact target state. |
| Outcome: | The proposed benchmark evaluates whether an agent can ground an instruction to the intended UI control and reach the exact target state. |
Copied to clipboard
| Challenge: | Positional biases in large language models hinder their ability to process long inputs. |
| Approach: | They propose a benchmark to assess positional bias in large language models involving multiple pieces of relevant information. |
| Outcome: | The proposed benchmark assesses the performance of long-context language models by examining their models with different input lengths and tasks. |
Copied to clipboard
| Challenge: | Existing methods for relation extraction struggle to identify causal terms under the invariant entity constraint. |
| Approach: | They propose a framework to generate commonsense counterfactuals for stable relation extraction by using a knowledge base WordNet and a constituency parser. |
| Outcome: | The proposed framework significantly enhances the stability of relation extraction models. |
Copied to clipboard
| Challenge: | Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges. |
| Approach: | They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task. |
| Outcome: | The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing. |
Copied to clipboard
| Challenge: | Existing studies treat prompts as flat text, overlooking their internal structure, and different components within a prompt contribute unequally to robustness. |
| Approach: | They propose a framework that decomposes prompts into functional components and a method that selectively modifies components to expose component-wise vulnerabilities. |
| Outcome: | The proposed framework exposes component-wise vulnerabilities while ensuring linguistic plausibility through perplexity-based filtering. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have similar value rankings but little is known about how susceptible they are to external influence and how different values are correlated with each other. |
| Approach: | They propose to use 6 different value transformation prompting methods to examine the plasticity of LLM value systems by comparing them with 8 LLMs. |
| Outcome: | The proposed methods are effective on 8 LLMs and 3 families. |
Copied to clipboard
| Challenge: | Document-level Event Factuality Identification (DEFI) is a task for identifying the degree of certainty that a specific event occurs in a document. |
| Approach: | They propose a document-level event factuality inference task to bridge the sentence-level and document- level event factualality semantically. |
| Outcome: | The proposed model outperforms the baselines on the public English and Chinese DLEF datasets. |
Copied to clipboard
| Challenge: | Recent work on feature interactions neglects underlying linguistic information in feature representations. |
| Approach: | They propose a method for modeling feature interactions with hyperbolic spaces using Poincare Explanation. |
| Outcome: | The proposed method is able to model feature interactions with hyperbolic spaces in a time efficient manner. |
Copied to clipboard
| Challenge: | Existing knowledge graph completion models require longer training and inference times as well as increased memory usage. |
| Approach: | They propose to encode textual descriptions into semantic representations before training and integrate structural embedding with pre-encoded semantic description to improve model's prediction performance on 1-N relations. |
| Outcome: | The proposed model increases inference speed by 30x and reduces training memory by approximately 60% on the WN18RR and UMLS datasets. |
Copied to clipboard
| Challenge: | TextReact is a new method to augment predictive chemistry with text descriptions retrieved from the literature. |
| Approach: | They propose a method that directly augments predictive chemistry with texts retrieved from the literature. |
| Outcome: | The proposed method outperforms existing models trained on molecular data. |
Copied to clipboard
| Challenge: | Existing methods do not differentiate question difficulty when designing prompting methods for them. |
| Approach: | They propose an adaptive method to improve large language models for reasoning problems by measuring question difficulty and tailoring demonstration set construction and difficulty-adapted retrieval strategies. |
| Outcome: | The proposed method shows an absolute improvement of up to 5.5% on arithmetic reasoning, 7.4% on symbolic reasoning, and 2.3% on commonsense reasoning. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have brought significant changes to various domains, especially through autonomous agents. |
| Approach: | They propose a framework that lets agents learn shortcuts from their past tasks and use them for future task execution. |
| Outcome: | The proposed framework enables agents to tackle unseen software-developing tasks more effectively. |
Copied to clipboard
| Challenge: | Existing methods to characterize RPM generalization are inadequate . existing methods do not provide a fine-grained diagnosis of distribution shifts . |
| Approach: | They propose a reasoning-based effective mutual information difference (R-EMID) to measure RPM performance degradation in an interpretable way. |
| Outcome: | The proposed model predicts the worst-case generalization performance of RPMs and reveals how shifts contribute to degradation. |
Copied to clipboard
| Challenge: | Existing LegalAI tasks are descriptive or predictive, requiring the users to translate the information into legal reasoning. |
| Approach: | They propose a task to generate a structured defence opinion conditioned jointly on an indictment and the defendant’s stated opinion, which often present conflicting claims. |
| Outcome: | The proposed approach improves on eight large language models (LLMs) and shows that it is more efficient than previous approaches. |
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion (KGC) are limited in generality and scalability due to poor contextual facts. |
| Approach: | They propose a contextual facts collector and contextual facts organizer to enhance the inference ability of GM-based methods for various KGC tasks. |
| Outcome: | The proposed model outperforms state-of-the-art methods in terms of performance. |
Copied to clipboard
| Challenge: | Existing frameworks for semi-supervised text mining with lightweight models are limited by label data scarcity. |
| Approach: | They propose a framework for semi-supervised text mining with lightweight models . it incorporates online distillation to train lightweight student models by imitating the Teacher model . |
| Outcome: | The proposed framework exhibits notable performance enhancements over existing frameworks. |
Copied to clipboard
| Challenge: | Prompt tuning (PT) based on frozen pre-trained language models has shown remarkable performance in few-shot learning . however, it relies heavily on good initialization of the prompt embeddings. |
| Approach: | They propose to use meta prompt tuning to improve cross-task generalization by learning to initialize prompt embeddings from other relevant tasks. |
| Outcome: | The proposed method outperforms PT on classification tasks, but not multi-task learning. |
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. |
| Approach: | They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup. |
| Outcome: | The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer. |
Copied to clipboard
| Challenge: | ChatGPT and GPT-4 are commercial large language models (LLMs) however, they may produce vague responses or incorrect answers in certain specialized domains. |
| Approach: | They propose a token compression scheme that uses summarization and semantic compression to reduce the token size of LLMs. |
| Outcome: | The proposed method reduces token size by doing summarization and semantic compression while reducing token size with only 1.6% accuracy drop. |
Copied to clipboard
| Challenge: | Existing approaches focus on diagnostic reasoning based on internal model knowledge or static knowledge bases. |
| Approach: | They propose a two-stage diagnostic reasoning framework that integrates multi-perspective evidence to generate a diagnostic prediction. |
| Outcome: | The proposed method generates suspected diagnoses and reasoning traces from web search, SOAP-formatted case, and clinical case database. |
Copied to clipboard
| Challenge: | Existing ground VLN agents struggle in aerial VLLN due to the lack of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration. |
| Approach: | They propose a large language model-empowered aerial VLN agent that decomposes the long-horizon task into sub-goals with different semantic levels. |
| Outcome: | The proposed method achieves state-of-the-art performance with significant improvement in continuous city environments. |
Copied to clipboard
| Challenge: | Existing tokenizers fail to explicitly leverage historical tokenization results . large language models (LLMs) have demonstrated remarkable effectiveness across NLP tasks . |
| Approach: | They propose a tokenizer that integrates spiking neurons to explicitly leverage historical tokenization results. |
| Outcome: | The proposed tokenizer leverages historical tokenization results, but does not selectively leverage history based on contextual relevance. |
Copied to clipboard
| Challenge: | Recent advances in text generation have limited applications due to multimodality problem. |
| Approach: | They propose a method which uses latent variables to capture word categorical information and invoke an advanced curriculum learning technique to overcome multi-modality problem. |
| Outcome: | The proposed method outperforms strong baselines without an autoregressive model, which further broadens the application scenarios of the parallel decoding paradigm. |
Copied to clipboard
| Challenge: | **Copyright Detective** is the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. |
| Approach: | They propose a system that detects copyright infringements and visualizes them . they use content recall testing, paraphrase-level similarity analysis and persuasive jailbreak probing . |
| Outcome: | The proposed system detects, analyzes, and visualizes potential copyright risks in LLM outputs. |
Copied to clipboard
| Challenge: | Document-level event factuality identification (DEFI) assesses the veracity degree to which an event mentioned in a document has happened. |
| Approach: | They propose a document-level event factuality identification framework with hallucination features . they propose factualusion corpus that integrates both genuine and hallucinous false information . |
| Outcome: | The proposed framework outperforms baselines in document event factuality identification. |
Copied to clipboard
| Challenge: | Large language model (LLM)-integrated applications face security vulnerabilities from prompt injection (PI) attacks. |
| Approach: | They propose a model enhancement method that synthesizes diverse training data and employs instruction-level chain-of-thought fine-tuning to enable LLMs to effectively identify and reject malicious instructions regardless of their source or position in the context. |
| Outcome: | The proposed method outperforms baselines in three critical dimensions while maintaining utility performance without degradation. |
Copied to clipboard
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
Copied to clipboard
| Challenge: | Existing domain-specific code benchmarks focus on assessing what knowledge LLMs possess rather than how they acquire and apply new knowledge. |
| Approach: | They propose a benchmark to evaluate domain specialization methods in real-world software development. |
| Outcome: | KOCO-bench is a new benchmark for evaluating domain specialization methods in real-world software development. |
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear. |
| Approach: | They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs. |
| Outcome: | The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms. |
Copied to clipboard
| Challenge: | Current models are not satisfactory for solving out-of-vocabulary problems . current models assume that the task ontology is well defined in advance . |
| Approach: | They propose to enhance the interrelation between slots with masked hierarchical attention. |
| Outcome: | The proposed model yields a significant performance gain over current state-of-the-art model and is more robust to out-ofvocabulary problem compared with other methods. |
Copied to clipboard
| Challenge: | Existing studies have studied history-dependent reasoning for question answering . utilizing global conversation history for enhancement is gaining interest . |
| Approach: | They propose to establish long-distance dependency among global utterances in multi-turn conversation. |
| Outcome: | The proposed method improves on QuAC by 1%, yielding the F1 score of 73.7%. |
Copied to clipboard
| Challenge: | Entity resolution (ER) is the task of identifying different representations of the same real-world entities across databases. |
| Approach: | They propose a deep learning-based method that targets low-resource settings for ER by combining transfer learning and active learning. |
| Outcome: | The proposed method achieves comparable, if not better, performance compared to state-of-the-art learning-based methods while using an order of magnitude fewer labels. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable capabilities in understanding complex tasks, but they can only handle graph partitioning tasks that require global perception abilities. |
| Approach: | They propose a pipeline for coarsening, reasoning, and refining to enable LLMs to perform graph partitioning on small-scale graphs. |
| Outcome: | The proposed pipeline can handle graph partitioning tasks on small graphs with coarsening, reasoning, and refining. |
Copied to clipboard
| Challenge: | Named entities are atomic objects of reference and reasoning in many knowledge-centric applications. |
| Approach: | They propose an active-learning based framework that drastically reduces the labeled data required to learn entities' structures. |
| Outcome: | The proposed framework outperforms handwritten programs and supervised learning models in relation extraction and entity resolution tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are typically trained on static datasets, preventing them from integrating real-time updates. |
| Approach: | They propose a dynamic question-answer answering dataset reflecting real-world knowledge updates that are automatically compared between Wikipedia versions and generating question-anchor pairs based on these updates. |
| Outcome: | The proposed framework improves LLMs' performance on time-sensitive question answering by maintaining a dynamic knowledge updating process. |
Copied to clipboard
| Challenge: | Existing methods to address the "lost-in-the-middle" problem suffer from high latency or suboptimal hand-crafted scaling strategies. |
| Approach: | They propose a layer-specific positional embedding scaling method that assigns distinct scaling factors to each layer. |
| Outcome: | Experiments show that the proposed method mitigates positional attention bias and delivers consistent improvements across multiple long-context benchmarks. |
Copied to clipboard
| Challenge: | Existing work shows that pre-trained models can improve in various natural language processing tasks. |
| Approach: | They propose a unified-modal encoder-decoder framework that pre-trains speech-text representations using large-scale unlabeled speech and text data. |
| Outcome: | The proposed framework is superior to existing models on speech-to-text processing tasks. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used for financial decision-making and stock market prediction for years. |
| Approach: | They propose to use Large Language Models to analyze on-chain and off-chain data to provide a comprehensive overview of the cryptocurrency market. |
| Outcome: | The proposed trading agent leverages the transparency and immutability of on-chain data, as well as the timeliness and influence of off-chain signals, providing a comprehensive overview of the cryptocurrency market. |
Copied to clipboard
| Challenge: | Existing models for text-to-SQL do not explicitly introduce common knowledge to address comparison relations. |
| Approach: | They propose to leverage adjective-noun phrasing knowledge mined from the web to predict comparison relations in text-to-SQL. |
| Outcome: | The proposed approach improves on the original and re-split Spider datasets on comparison relation prediction. |
Copied to clipboard
| Challenge: | Existing code translation datasets focus on a single pair of programming languages . early software systems are developed using programming languages such as Fortran and COBOL . |
| Approach: | They propose a large-scale comprehensive benchmark that supports the largest variety of programming languages for code translation. |
| Outcome: | The proposed framework supports translations between multiple programming languages and a cross-framework dataset for deep learning code across different frameworks. |
Copied to clipboard
| Challenge: | emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies. |
| Approach: | They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals. |
| Outcome: | The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures . |
Copied to clipboard
| Challenge: | Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts. |
| Approach: | They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing. |
| Outcome: | The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). |
Copied to clipboard
| Challenge: | Bilingual lexicon induction (BLI) is the task of inducing word translations with a learned mapping function that aligns monolingual word embedding spaces in two different languages. |
| Approach: | They propose a model that explicitly captures multiple topological structure information to achieve accurate bilingual lexicon induction. |
| Outcome: | The proposed model captures multiple topological structure information to achieve accurate BLI on a public dataset. |
Copied to clipboard
| Challenge: | End-to-end spoken dialogue models have higher potential ceiling in expressiveness and perceptual ability than cascaded systems. |
| Approach: | They propose a modality-aware adaptive post-training recipe that constrains preference updates to the semantic channel and improves acoustic behavior via explicit anchoring. |
| Outcome: | The proposed model improves speech quality and expressiveness across spoken dialogue benchmarks and architectures. |
Copied to clipboard
| Challenge: | Word sense disambiguation (WSD) is a fundamental yet challenging task in natural language processing. |
| Approach: | a novel multi-agent Debate framework for adversarial word Sense disambiguation is proposed . the framework simulates a real-world debate environment where multiple agents engage in discussions about ambiguous words in the context of adversarials. |
| Outcome: | The proposed framework integrates with existing LLMs and improves models in Chinese language . it shows that it can be used to improve models in the Chinese language and improve performance . |
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models generate similar output embeddings which makes it difficult to discriminate for the prompt-based classifier. |
| Approach: | They propose a calibration method which rotates the embedding feature into a new metric space and adapts the ratio of each dimension to a uniform distribution. |
| Outcome: | The proposed method improves the distinguishability of learning embeddings on three datasets under various settings. |
Copied to clipboard
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
Copied to clipboard
| Challenge: | Existing methods for creating video content are limited by high costs and slow update cycles. |
| Approach: | They propose a paradigm shifting educators from manual creators to high-level directors who focus on pedagogical intents while agents handle execution. |
| Outcome: | The proposed framework reduces production costs to 0.3% of traditional course videos and provides a robust solution for scalable education. |
Copied to clipboard
| Challenge: | Existing systems rely on sentence-level labels, which fails to capture the subtle nuances of human affect. |
| Approach: | They propose to use a large-scale, context-aware speech corpus derived from multi-speaker audiobooks to generate a speech that is human-like. |
| Outcome: | The proposed model outperforms existing methods in terms of emotional expression accuracy and naturalness. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown impressive performance on a range of tasks, yet advanced instruction following (IF) remains a significant challenge. |
| Approach: | They propose a benchmark that features over 1,600 prompts and expert-curated rubrics that assess LLMs’ ability to follow complex, multi-turn, and system-level instructions. |
| Outcome: | The proposed framework improves instruction-following abilities of large language models, achieving a 6.7% gain on AdvancedIF and strong results on public benchmarks. |
Copied to clipboard
| Challenge: | Recent studies have identified a gap in the availability of tools and resources to study bias in languages other than English and social contexts outside the north of America. |
| Approach: | They use stereotypes to build a corpus of sentence pairs that cover biases in seven cultural contexts. |
| Outcome: | The proposed resource covers a wide range of languages and cultural settings . it favors sentences that express stereotypes in most bias categories . |
Copied to clipboard
| Challenge: | Named entity recognition (NER) methods can detect entity spans and classify them into pre-defined categories. |
| Approach: | They propose a type-aware decomposed framework to filter out false spans . they propose 'type-against-type' learning strategy to construct more accurate prototypes based on type names as references. |
| Outcome: | The proposed framework yields state-of-the-art on several benchmarks. |
Copied to clipboard
| Challenge: | Existing question reformulation models are based on supervised question labels without considering feedback information from answers. |
| Approach: | They propose a question reformulation model that integrates conversational history information with reinforcement learning. |
| Outcome: | The proposed model is more effective in conversational machine comprehension with reinforcement learning. |
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models (LLMs) lack explicit mechanisms for guiding diverse exploration and instead prioritize efficiency and performance over diversity. |
| Approach: | They propose a reinforcement learning-based framework that decomposes the generation process into explicitly planned intermediate steps and introduces divergence at the planning phase based on diversity variation. |
| Outcome: | The proposed method significantly outperforms existing baselines on creative writing benchmarks on a semi-structured long chain-of-thought (CoT) it introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. |
Copied to clipboard
| Challenge: | Existing work on fine-grained entity typing (FET) relies on knowledge bases as distant supervision, but lack of or incompleteness of KB can hinder training. |
| Approach: | They propose a two-step framework that trains FET models without accessing any knowledge base. |
| Outcome: | The proposed framework achieves competitive performance with respect to the models trained on the original KB-supervised datasets. |
Copied to clipboard
| Challenge: | Existing studies on the topic of common sense story understanding focus on generating guesses for a missing event or concentrating on unsupervised learning. |
| Approach: | They propose to extend attention-based neural network with external knowledge resources to understand temporal stories and predict their endings. |
| Outcome: | The proposed model outperforms state-of-the-art models and external knowledge resources. |
Copied to clipboard
| Challenge: | Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models. |
| Approach: | They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text . |
| Outcome: | The proposed framework can be used to estimate the robustness of NLP models. |
Copied to clipboard
| Challenge: | Existing methods for deciphering ancient Chinese Oracle Bone Script (OBS) treat deciphering as a closed-set image recognition problem, which fails to bridge the "interpretation gap" . |
| Approach: | They propose a vision-language model framework that integrates a VLM and an LLM to automate a reasoning chain of component identification and knowledge retrieval. |
| Outcome: | The proposed framework yields more detailed and precise decipherments compared to baseline methods. |
Copied to clipboard
| Challenge: | Existing work on meaning representations is not comprehensively evaluated due to the lack of readily-available execution engines. |
| Approach: | They propose a unified benchmark on meaning representations by integrating existing semantic parsing datasets, completing the missing logical forms, and implementing the missing execution engines. |
| Outcome: | The proposed benchmark combines existing parsing datasets, completes missing logical forms, and implements missing execution engines. |