Papers by Fu Zhang
Copied to clipboard
| Challenge: | Existing studies attribute object hallucinations to linguistic priors and data biases . MFCD method removes hallucinian distribution in the original output distribution . |
| Approach: | They propose a method that removes the hallucination distribution in the original output distribution . they propose MFCD to mitigate hallucinism in large visual-language models . |
| Outcome: | The proposed method reduces hallucination distributions without training or external tools . the proposed method can be applied to various LVLMs without modifying model architecture or training . |
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key component of natural language processing (NLP) but it is difficult to implement in specialized domains such as wind power fault diagnosis. |
| Approach: | They propose a reasoning-enhanced generative framework that integrates Chain-of-Thought prompting and recall-oriented loss optimization to address these challenges. |
| Outcome: | The proposed framework improves recall and overall F1 performance across general and industrial domains. |
Copied to clipboard
| Challenge: | a new framework for visual annotation of text-based questions is needed to improve performance . obtaining corresponding images through manual annotation often entails high costs . |
| Approach: | They propose a framework that uses visual modality to enhance the performance of text-based questions. |
| Outcome: | The proposed framework improves the alignment between text and images by using search engines or web scraping techniques. |
Copied to clipboard
| Challenge: | Syntax has been a useful source of information for statistical RST discourse parsing. |
| Approach: | They propose an implicit syntax feature extraction approach using hidden-layer vectors extracted from a neural syntax parser. |
| Outcome: | The proposed model with dynamic oracle is competitive with existing models. |
Copied to clipboard
| Challenge: | Recent advances in Relation Extraction (RE) emphasize Zero-Shot methodologies, aiming to recognize unseen relations between entities with no annotated data. |
| Approach: | They propose a plug-in retrieval adjuster that allows rapid fine-tuning without accessing LLMs’ parameters. |
| Outcome: | The proposed model demonstrates comparable performance on multiple benchmarks. |
Copied to clipboard
| Challenge: | Existing methods to predict relationships with given entity pairs are lacking in supervised methods. |
| Approach: | They propose a framework for zero-shot Relation Extraction that includes two modules: Custom Embedding and Dynamic Aggregation. |
| Outcome: | The proposed framework shows competitive performance on two ZSRE datasets. |
Copied to clipboard
| Challenge: | Existing methods to mitigate hallucinations generate erroneous or fabricated information. |
| Approach: | They propose a rank-response-based model that annotates pair-reponses and trains alignment algorithms to improve the correspondence between images and text. |
| Outcome: | The proposed model outperforms the DPO method and outperfies existing methods on two MLLMs of different sizes and four widely used benchmarks. |
Copied to clipboard
| Challenge: | Existing techniques for extending context capabilities in LLMs require additional training procedures and access to datasets with long context (e.g., sequences of 32K tokens). |
| Approach: | They propose a solution to extend context capabilities in Large Language Models by training a single process over a sequence of 4K tokens. |
| Outcome: | The proposed solution significantly reduces the cost of continual-pretraining or fine-tuning over short sequences and improves robustness to diverse relative positions. |
Copied to clipboard
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning are decoupled from structured search due to limited trajectory diversity. |
| Approach: | They propose a unified RL framework that integrates multiple agents within a shared tree-structured search environment. |
| Outcome: | Experiments show that MARS2 improves performance across diverse model combinations and training settings. |
Copied to clipboard
| Challenge: | Existing methods for camouflaged object segmentation are limited to vision-only mask prediction under fixed task assumptions. |
| Approach: | They propose a language-guided reasoning camouflaged object segmentation task that generates an intent-consistent segmentation mask from an image and an implicit query text instruction. |
| Outcome: | The proposed task can generate an intent-consistent segmentation mask from a camouflaged image and an implicit query text instruction. |
Copied to clipboard
| Challenge: | Existing methods do not examine social groups categorised by geographical information, leaving the region-related biases in pre-trained LMs unexplored. |
| Approach: | They propose a hierarchical regional bias evaluation method to quantify regional bias in pre-trained language models. |
| Outcome: | The proposed method evaluates regional bias with regard to comprehensive topics and measures potential regional bias that can be propagated to downstream tasks. |
Copied to clipboard
| Challenge: | Existing algorithms to improve the ability of LLMs to follow complex instructions are lacking. |
| Approach: | They propose a benchmark to improve the ability to follow complex instructions by using a IOPO alignment method to take input and output preference into consideration. |
| Outcome: | The proposed algorithm shows 8.15%, 2.18% improvements on in-domain data and 5.91%, 2.83% on out-of-domain datasets compared to SFT and DPO respectively. |
Copied to clipboard
| Challenge: | Existing benchmarks for Geometry problem solving lack fine-grained evaluation for long-step problems necessitating auxiliary line construction. |
| Approach: | They present a fine-grained annotated dataset with long-step reasoning and auxiliary line construction that provides a detailed evaluation of 23 leading MLLMs. |
| Outcome: | The proposed model performs significantly worse on long-step problems than short-step ones, with 18 models showing a performance drop of over 50%. |
Copied to clipboard
| Challenge: | Empathy is a desirable human trait that improves the emotional perceptivity in emotion-bonding social activities. |
| Approach: | They propose a framework that integrates emotion correlation learning, utilization, and supervising. |
| Outcome: | The proposed framework improves empathetic perception and expression on a humanized dialogue dataset. |
Copied to clipboard
| Challenge: | Existing methods for post-training quantization struggle to support weight–activation joint quantization and extreme low-bit weight quantization. |
| Approach: | They propose a framework that addresses weight–activation joint quantization and extreme weight quantization. |
| Outcome: | The proposed framework achieves superior performance under both W4A4 and highly aggressive W2 settings while incurring negligible additional computational overhead. |
Copied to clipboard
| Challenge: | Chinese named entity recognition models are vulnerable to word ambiguities due to the lack of global semantics and chain structure. |
| Approach: | They propose a lexicon-based graph neural network with global semantics to solve word ambiguities in Chinese named entity recognition (NER) Lexicons are used to construct the graph and provide word-level features. |
| Outcome: | The proposed model improves on four NER datasets on Chinese characters, potential words, and the whole-sentence semantics. |
Copied to clipboard
| Challenge: | Existing code translation benchmarks focus on individual functions, overlooking repository-level challenges like intermodule coherence and dependency management. |
| Approach: | They propose a framework for benchmarking Java-to-C# translation at the repository level . it uses a translation framework guided by skeletons and fine-grained quality evaluation . |
| Outcome: | The proposed framework improves Java-to-C# translation quality at the repository level. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. |
| Approach: | They propose two methods to improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
| Outcome: | The proposed methods improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
Copied to clipboard
| Challenge: | Current methods for multi-modal entity alignment ignore relative interactions between modalities and the accuracy of weights. |
| Approach: | They propose a relative interaction and calibration framework for multi-modal entity alignment that uses attention mechanisms to perceive the uncertainty of the weight for each modality. |
| Outcome: | The proposed framework outperforms baselines across 5 datasets and 23 settings. |
Copied to clipboard
| Challenge: | Existing methods focus on interactions between neighboring entities in the structural modality while neglecting interactions between entities in visual and attribute modalities. |
| Approach: | They propose a structure-guided multimodal entity alignment method which prioritizes structural information from knowledge graphs to enhance the visual and attribute modalities. |
| Outcome: | The proposed method achieves state-of-the-art performance across multiple datasets, validating its effectiveness and superiority in practical applications. |
Copied to clipboard
| Challenge: | Using a pointer-generator framework for reading/sampling over large documents, we propose a framework for learning over long narratives where documents easily span over thousands of tokens. |
| Approach: | They propose a curriculum learning (CL) based pointer-generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. |
| Outcome: | The proposed framework improves on the NarrativeQA reading comprehension benchmark and reaches state-of-the-art performance. |
Copied to clipboard
| Challenge: | Current approaches to temporal knowledge representation face limited generalization to unseen facts and insufficient interpretability of reasoning processes. |
| Approach: | They propose a framework that uses a denoising diffusion process to complete reasoning tasks . they propose introducing a noise source and historical conditionguiding mechanism to improve interpretability . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three benchmark datasets. |
Copied to clipboard
| Challenge: | Existing knowledge editing approaches struggle with sequential editing scenarios and harm the general capabilities of the model. |
| Approach: | They propose a framework that combines robust supervised fine-tuning and model merging for knowledge editing to combine supervised and supervised learning. |
| Outcome: | The proposed approach outperforms existing methods in sequential editing while preserving the original performance of the model. |
Copied to clipboard
| Challenge: | Existing evaluation methodologies for MWPs diverge from human judgment and face challenges in recognizing mathematically equivalent answers. |
| Approach: | They propose an evaluation metric rooted in graph edit distance that features benefits such as permutation invariance and more accurate program equivalence identification. |
| Outcome: | The proposed evaluation metric features benefits such as permutation invariance and more accurate program equivalence identification. |
Copied to clipboard
| Challenge: | Extending large language models to low-resource languages often incurs an "alignment tax" token-level fine-tuning enforces token-level surface imitation on narrow and biased data distributions. |
| Approach: | They propose a semantic-space alignment paradigm powered by group-level semantic rewards instead of likelihood maximization. |
| Outcome: | The proposed model acquires low-resource capa- bilities while mitigating alignment tax on Tibetan–Chinese machine translation and Ti- betan headline generation. |
Copied to clipboard
| Challenge: | Large language models perform well in offline machine translation when the complete source sentence is provided . however, in many real scenarios, the source tokens arrive in a streaming manner and simultaneous machine translation is required . |
| Approach: | They propose a new paradigm that includes constructing supervised fine-tuning data for simultaneous machine translation (SiMT) to achieve SiMT, source and target tokens are rearranged into interleaved sequences, separated by special tokens according to varying latency requirements. |
| Outcome: | The proposed approach achieves state-of-the-art performance across various SiMT benchmarks and evaluation metrics while maintaining efficient auto-regressive decoding. |
Copied to clipboard
| Challenge: | Generative audio modeling has been fragmented into specialized tasks such as text-to-speech (TTS), text- to-music (TTM), and text-ta (TTA) specialized models require reference audio for timbre cloning and strict phoneme alignment, whereas TTA models generate unstructured textures from open-ended captions. |
| Approach: | They propose a unified flow-matching framework capable of synthesizing speech, music, sound effects . they propose 'token injection mechanism' that projects unstructured environmental sounds into structured temporal latent space . |
| Outcome: | The proposed framework achieves state-of-the-art performance in instruction-based TTS and TTM while maintaining competitive fidelity in TTA. |
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations in LLMs rely on external knowledge for reference retrieval or require sampling multiple responses for consistency verification. |
| Approach: | They propose a reference-free, uncertainty-based method for detecting hallucinations in Large Language Models that imitates human focus in factuality checking from three aspects: focus on the most informative keywords; focus on unreliable tokens in historical context; focus of token properties such as token type and token frequency. |
| Outcome: | The proposed method achieves state-of-the-art performance across all evaluation metrics and eliminates the need for additional information. |
Copied to clipboard
| Challenge: | Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output. |
| Approach: | They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities. |
| Outcome: | The proposed model can be trained stably without any alterations to existing models or training paradigms. |
Copied to clipboard
| Challenge: | Existing models struggle to maintain stable understanding performance and low GPU memory overhead. |
| Approach: | They propose a training-free architecture for real-time and accurate understanding of video streams . HERMES reuses a compact KV cache, enabling efficient streaming understanding . |
| Outcome: | The proposed architecture achieves 10 faster TTFT compared to prior SOTA. |
Copied to clipboard
| Challenge: | unified multimodal large language models (MLLMs) are emerging but lack a systematic framework to connect them and situate current trends within a broader landscape. |
| Approach: | They present a systematic review of unified Multimodal Large Language Models . they outline the foundational concepts and prerequisites for understanding them . |
| Outcome: | The present review provides a systematic and systematic overview of unified MLLMs . it discusses persistent challenges and identify promising directions for future research . |
Copied to clipboard
| Challenge: | a new technique allows paraphrase generation to be user-controlled . a user looking for cheap hotels in New York would not find the other answer helpful . |
| Approach: | They propose a method that provides a user with explicit tags that can be placed around any arbitrary segment of text to mean "don't change me!" they propose allowing user-controllable paraphrase generation by fine-tuning model that exhibits this behavior . |
| Outcome: | The proposed technique is language agnostic and tested in English and Chinese. |
Copied to clipboard
| Challenge: | Entity Alignment (EA) is a critical task in Knowledge Graph (KG) integration. |
| Approach: | They propose a novel approach that leverages the data characteristics of synthetic benchmarks to improve performance in real-world datasets. |
| Outcome: | The proposed approach outperforms state-of-the-art models on real-world datasets and achieves a 29.94% improvement in Hits@1 on DOREMUS and 5.64% improvement on AGROLD. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are evolving towards autonomous agents . retrieval capabilities are well-benchmarked, but post-retrieval synthesis is under-evaluated due to open-ended writing. |
| Approach: | They propose a benchmark to evaluate information consolidation capabilities using survey papers as gold standards. |
| Outcome: | The proposed benchmark analyzes the post-retrieval synthesis stage of large language models . it leverages high-quality survey papers as gold standards and reverse-engineers research requests . the proposed benchmark outperforms single-turn generation and reduces hallucinations . |
Copied to clipboard
| Challenge: | Recent advances in Multimodal Large Language Models (MLLMs) have shifted visual reasoning from tool-calling to end-to-end perceptionreasoning. |
| Approach: | They synthesize the emerging paradigm of Image-Grounded Chain-of-Thought (IG-CoT) they propose a method-centric taxonomy covering prompting, supervised fine-tuning, and reinforcement learning . |
| Outcome: | The proposed model is based on a method-centric taxonomy and benchmarks. |
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) suffer from significant computational overhead due to the quadratic growth of attention computations with the number of multimodal tokens. |
| Approach: | They propose a training-free pruning framework that prunes multimodal tokens without a trained pruning method. |
| Outcome: | The proposed pruning framework outperforms existing token pruning methods and generalizes across diverse MLLMs. |
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs. |
| Approach: | They propose a novel LLMguided MMEA framework that prioritizes noise reduction before fusion. |
| Outcome: | The proposed framework prioritizes noise reduction before fusion and improves semantics on the noisy FB YG dataset. |
Copied to clipboard
| Challenge: | Existing approaches to improve LLM reasoning are limited in complex domains and lack external grounding makes verifiers unreliable on computation-intensive tasks. |
| Approach: | They propose a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process. |
| Outcome: | The proposed framework surpasses state-of-the-art ORMs by 25.2% under parallel and sequential TTS. |
Copied to clipboard
| Challenge: | Existing document-level relation extraction models focus on individual entity pairs, limiting their ability to handle complex reasoning tasks. |
| Approach: | They propose a document-level relation extraction framework based on global relations and entity pair reasoning that captures fine-grained interactions between entity pairs. |
| Outcome: | The proposed framework outperforms existing models on widely-used datasets. |
Copied to clipboard
| Challenge: | Existing methods for automatic essay scoring are based on hand-crafted surface-level features, but recent advances in representation learning have improved performance. |
| Approach: | They propose a pre-training based automated Chinese essay scoring method with weakly supervised pre- training, supervised cross- prompt fine-tuning and supervised target- prompt refine-tuneing. |
| Outcome: | The proposed method improves a state-of-the-art neural essay scorer in terms of effectiveness and domain adaptation ability, while in-depth analysis also reveals its limitations. |
Copied to clipboard
| Challenge: | Documents contain various structures that hinder the ability of machines to comprehend . user information needs are often underspecified, and the nature of heterogeneous documents poses challenges. |
| Approach: | They propose a dataset for building machines that help users seek information via conversations . their dataset contains over 100,000 turns based on Chinese documents from five domains . |
| Outcome: | The proposed tasks are challenging and worthy of further research. |
Copied to clipboard
| Challenge: | Large language models have made impressive progress in few-shot learning but still face difficulties in reasoning tasks such as GSM8K. |
| Approach: | They propose a new approach that uses a verifier to filter out incorrect answers based on a weighted voting scheme to improve reasoning ability of language models. |
| Outcome: | The proposed approach improves GSM8K reasoning rate by 17.9% to 58.1%. |
Copied to clipboard
| Challenge: | Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. |
| Approach: | They propose an explanation benchmark for analogical reasoning using a Civil Service exam . they use a free-text explanation scheme to explain whether an analogy should be drawn . |
| Outcome: | The proposed benchmark is very challenging for state-of-the-art models, it is found. |
Copied to clipboard
| Challenge: | obtaining large amounts of labeled data is expensive. |
| Approach: | They develop a semi-supervised learning framework called FLiText which improves text classification accuracy. |
| Outcome: | The proposed framework improves accuracy of lightweight models on IMDb, Yelp-5, and Yahoo! Answer . the framework improve accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDa, Yep-5 and Yahoo. Answer compared with the fully supervised method on the full dataset . |
Copied to clipboard
| Challenge: | Existing MKGC methods train with all modalities available, implicitly assuming consistent complementarity . however, this often induces modality dependence and modality competition under heterogeneous noise, which can hinder robust multi-modal fusion and limit overall performance. |
| Approach: | They propose a framework to infer missing links in multimodal knowledge graphs by leveraging structured triples together with auxiliary modalities such as text and images. |
| Outcome: | The proposed framework outperforms baselines and achieves new state-of-the-art results. |
Copied to clipboard
| Challenge: | Existing Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) systems insufficiently model the interaction between query semantics and relation types, resulting in imprecise subgraph retrieval and unstable reasoning. |
| Approach: | They propose a retrieval framework that integrates query semantics and relation embeddings directly into the attention mechanism. |
| Outcome: | Experiments on WebQSP and CWQ establish new state-of-the-art results in both Triple Recall and Answer Recall. |
Copied to clipboard
| Challenge: | Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs. |
| Approach: | They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction. |
| Outcome: | The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI. |
Copied to clipboard
| Challenge: | Existing models for explainable recommendation have neglected faithfulness of KG reasoning . |
| Approach: | They propose to draw on interpretable logical rules to guide path-reasoning process for explanation generation. |
| Outcome: | The proposed method delivers high-quality recommendations and ascertains the faithfulness of the derived explanation. |
Copied to clipboard
| Challenge: | Existing code large language models rely on large-scale instruction data distilled from proprietary LLMs for fine-tuning, which typically incurs high costs. |
| Approach: | They propose an iterative self-distillation approach to bootstrap small-scale LLMs . they use large-scale instruction data distilled from proprietary LLM for fine-tuning . |
| Outcome: | The proposed method reduces reliance on proprietary LLMs and minimizes costs. |
Copied to clipboard
| Challenge: | Existing studies show that Chain-of-thought (CoT) can enhance the performance of large language models (LLMs) however, there is limited understanding of the algorithms that Transformer+CoT can learn. |
| Approach: | They propose two metrics to evaluate Transformer+CoT's state tracking capabilities and identify the circuit responsible for tracking the world state. |
| Outcome: | The proposed model achieves 100% accuracy for each state, highlighting an implicit finite state automaton (FSA) embedded within the model. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to determine which relations hold between a given entity pair in a document. |
| Approach: | They propose a document-level relation extraction paradigm that decouples existing losses into independent positive and negative losses, which interact solely with a shared threshold. |
| Outcome: | The proposed model outperforms existing models on four datasets and achieves state-of-the-art results. |
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs. |
| Approach: | They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction. |
| Outcome: | The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models. |
Copied to clipboard
| Challenge: | a comprehensive investigation into optimization strategies for hypernetworks remains lacking. |
| Approach: | They propose restart optimization strategies to improve hypernetworks' performance for language models. |
| Outcome: | The proposed restart strategy improves hypernetworks' performance for language models, compared to conventional deep neural networks. |
Copied to clipboard
| Challenge: | Recent work on scene generation focuses on generating 3D scenes from textual descriptions . however, the task of generating industrial scenes with LLMs is complex and requires precise measurements and positioning . |
| Approach: | They propose an LLM-based agent for generating industrial scenes through C# code. |
| Outcome: | Experiments show that LLMs powered by SceneGenAgent exceed their original performance . the agent achieves 81.0% success rate in real-world industrial scene generation tasks . |
Copied to clipboard
| Challenge: | Existing systems that translate optimization formulas manually are cumbersome and time-consuming. |
| Approach: | They propose a system that converts optimization formulas from TeX document to solver language. |
| Outcome: | The proposed system helps operations research practitioners convert optimization formulations into solver modeling languages. |
Copied to clipboard
| Challenge: | Existing approaches to merge multi-modal knowledge only use one fusion strategy . however, the impact of the fusion on individual entities could be ignored . |
| Approach: | They propose an adaptive multi-modal feature fusion strategy for entity alignment that selects the optimal entity-level feature blending strategy. |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance compared to models using the same modality on a dataset with multiple inconsistent images and styles. |
Copied to clipboard
| Challenge: | Recent efforts to accelerate inference in Multimodal Large Language Models have focused on visual token compression. |
| Approach: | They propose a framework that leverages downsampling as a discriminator to denoise existing benchmarks. |
| Outcome: | The proposed evaluation framework leverages downsampling as a discriminator to denoise existing benchmarks. |
Copied to clipboard
| Challenge: | Existing studies use synthetic speech to train and evaluate SpeechRE models, hindering their development . modality gap issue limits performance of existing models, limiting future researches . |
| Approach: | They propose to use speech data to train and evaluate SpeechRE models by using real speech . they propose to train a cross-modal alignment model to bridge the modality gap . |
| Outcome: | The proposed model can train to bridge the modality gap between speech encoder and text decoder . the proposed model is based on two real SpeechRE datasets . |
Copied to clipboard
| Challenge: | Existing explainable recommendation models generate repetitive sentences for different items or empty sentences with insufficient details. |
| Approach: | They propose a visual-enhanced approach to generate rating scores and text explanations using visualization generation and text–image matching discrimination. |
| Outcome: | The proposed approach improves both the text quality and the diversity and explainability of the generated explanations. |
Copied to clipboard
| Challenge: | Existing work on AMR focuses on individual sentences, but there is a need for multi-sentence AMRs. |
| Approach: | They propose to use an end-to-end AMR coreference resolution model to generate multi-sentence AMRs. |
| Outcome: | The proposed model reduces error propagation and is more robust for both in- and out-domain situations. |
Copied to clipboard
| Challenge: | Existing embedding-based EA methods encode entities as embeddables and learn to align embeddibles. |
| Approach: | They propose to capture three types of logical inference paths with Non-Axiomatic Logic to iteratively align entities and relations by integrating the conclusions of the inference path. |
| Outcome: | The proposed method outperforms state-of-the-art methods in terms of Hits@1 on all three datasets of DBP15K with both supervised and unsupervised settings. |
Copied to clipboard
| Challenge: | Optical character recognition (OCR) is a relatively new form of tablature recognition, but its accuracy is limited due to its unbounded composition and manuscript-level variability. |
| Approach: | They propose a method that predicts component sequences under a zero-shot split and synthesize manuscript-like training images via component-wise style recomposition and manuscript-domain noise modeling. |
| Outcome: | The proposed method achieves 63.02% sequence accuracy on real-world Jianzi benchmark, surpassing Gemini-3-Pro by 35.11%. |
Copied to clipboard
| Challenge: | Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build. |
| Approach: | They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
| Outcome: | The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
Copied to clipboard
| Challenge: | Existing data selection strategies for continual pre-training of large language models often rely on scarce labeled data or computationally expensive LLMs. |
| Approach: | They propose an annotation-independent data selection framework for CPT that evaluates grammatical complexity using lexical diversity and syntactic complexity. |
| Outcome: | The proposed framework outperforms baselines on a financial dataset and surpasses full-data training by 1.7% using only 20% of the data. |
Copied to clipboard
| Challenge: | Temporal knowledge graph reasoning (TKGR) is a crucial task that involves reasoning at known timestamps to complete the future facts. |
| Approach: | They propose a temporal knowledge graph reasoning model with logicality and densification strategy that captures temporal evolving pattern and structural information in TKGs. |
| Outcome: | The proposed model outperforms the state-of-the-art models and is based on a structure-aware language model with logicality and densification strategy. |
Copied to clipboard
| Challenge: | Recent research has shown that code pre-trained models improve coding capabilities. |
| Approach: | They propose a code data pruning strategy to identify which datasets are high-quality code instruction data. |
| Outcome: | The proposed model achieves state-of-the-art performance using fewer training data. |
Copied to clipboard
| Challenge: | Traditional metrics for automatic text evaluation are tailored to specific tasks, while LLM-based evaluation metrics are costly. |
| Approach: | They propose a metric that leverages projections of LLM representations for evaluation. |
| Outcome: | The proposed metric exhibits higher correlation with human judgments than previous methods on 14 datasets. |
Copied to clipboard
| Challenge: | Existing research has focused on constraint categories, offering little guidance for improving instruction following abilities. |
| Approach: | They propose a multi-dimensional constraint framework that allows for instruction following . they construct 9,106 code-verifiable samples and evaluate 18 LLMs . |
| Outcome: | The proposed framework improves instruction following performance without compromising general performance. |
Copied to clipboard
| Challenge: | Existing research on generative AI security is driven by mutually reinforcing attack and defense methodologies grounded in empirical experience. |
| Approach: | They propose a new algorithm that uses a random sampling algorithm to control risk. |
| Outcome: | The proposed algorithm improves robustness and utility while maintaining latency comparable to existing algorithms. |
Copied to clipboard
| Challenge: | Existing methods for dialogue summarization are far from satisfactory . omission is a major factor in affecting the quality of summarizing, but few studies have explored the problem . |
| Approach: | They propose a dataset that provides high-quality omission labels for dialogue summarization . they propose to use this dataset to detect omitted dialogue utterances . |
| Outcome: | The proposed dataset improves summarization quality by providing ground-truth omission labels . the proposed dataset and codes are publicly available . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been successful in machine translation, but lack of high-quality parallel corpora and cost constrain scalability. |
| Approach: | They propose an LLM-driven dual-learning framework that enables autonomous translation . they employ a robust semantic-aware reward function that balances adequacy with reconstruction fidelity . |
| Outcome: | The proposed model outperforms larger models on benchmarks and achieves parity with state-of-the-art supervised baselines on mainstream benchmarks. |
Copied to clipboard
| Challenge: | Existing agentic applications rely on LLMs to self-assess the factuality of outputs . but current LLM systems fail to detect hallucinations . |
| Approach: | They propose a benchmark that breaks down hallucination detection into four critical steps . they show that when halluciation detection is treated as a multi-step process, all models achieve considerably better performance. |
| Outcome: | The proposed benchmark breaks down hallucination detection into four critical steps . it shows that when halluciation detection is treated as a multi-step process, all models achieve considerably better performance. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to identify relations for a given entity pair within a document. |
| Approach: | They propose to partition the label space into different sub-label spaces and learn an adaptive threshold for each sub-labeled space. |
| Outcome: | The proposed model outperforms single-loss methods on the concurrent application of multiple losses across four datasets. |
Copied to clipboard
| Challenge: | Existing prototype-based methods for ZSRE ignore abundant side information and suffer from a significant encoding gap between prototypes and sentences. |
| Approach: | They propose a framework to encode schema alignment to enhance prototype-based ZSRE methods. |
| Outcome: | The proposed method outperforms existing methods on FewRel and Wiki-ZSL datasets and exhibits substantially faster performance and reduces the need for extensive manual labor in prototype construction. |
Copied to clipboard
| Challenge: | Existing LLM agents are brittle in open-ended environments due to two limitations: 1) a closed action space; 2) myopic error recovery. |
| Approach: | They propose a novel architecture that augments the action space and revises global strategies by adding a reflective replanning mechanism to the system. |
| Outcome: | Experiments show that CAR outperforms baselines in a diagnostic benchmark with pruned toolsets to simulate tool scarcity. |
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) systems face three major challenges: reliance on handcrafted features that limit generalizability, difficulty in capturing fine-grained traits like coherence and argumentation, and inability to handle multimodal contexts. |
| Approach: | They propose a multimodal benchmark to evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
| Outcome: | The proposed system can evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
Copied to clipboard
| Challenge: | a recent study shows that large language models have limited generalization in low-resource languages like Chinese. |
| Approach: | They propose to evaluate the zero-shot generalizability of large language models to the Chinese language . they release only half of the dataset publicly, with the remainder kept private . |
| Outcome: | The Chinese Instruction-Following Benchmark evaluates the generalizability of LLMs to the Chinese language. |
Copied to clipboard
| Challenge: | Existing methods for data augmentation neglect fine-grained knowledge, such as entities and quantities, leading to insufficient diversity and high data noise. |
| Approach: | They propose a pipeline-based data augmentation method via LLMs and introduce the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model to enhance unsupervised sentence embeddings. |
| Outcome: | The proposed method achieves state-of-the-art performance in semantic textual similarity tasks using fewer data samples and smaller LLMs. |
Copied to clipboard
| Challenge: | Existing StarCraft II benchmarks rely on abstract state representations that deviate from human perception . Existing systems rely only on abstract representations, creating an artificial gap between how humans process battlefield information and limiting ecological validity of learned behaviors. |
| Approach: | They introduce AVACraft, the first multimodal benchmark environment for complex decision-making in StarCraft II. |
| Outcome: | The AVACraft benchmark supports both traditional and modern multi-agent reinforcement learning paradigms. |
Copied to clipboard
| Challenge: | Visual segmentation with instruction has been a challenging task for many years . large language models and large multimodal models have spurred a new wave of research . |
| Approach: | They review recent works in LLM-based visual segmentation and analyze their architectural innovations, training strategies, and benchmark performance. |
| Outcome: | The present study reviews the most recent works in LLM-driven visual segmentation . it identifies key challenges and promising future directions . |
Copied to clipboard
| Challenge: | Existing studies show that large language models have strong reasoning capabilities through chain-structured methods. |
| Approach: | They propose a framework for navigating and expanding thought structures to overcome blind spots in LLM reasoning. |
| Outcome: | The proposed framework overcomes blind spots in large language models by expanding thought structures . the proposed framework improves accuracy of the final answer and intermediate reasoning steps . |
Copied to clipboard
| Challenge: | Existing pretrained language models for mental health detection are inadequate . one in four people worldwide suffers from mental disorders . |
| Approach: | They train and release two pretrained masked language models to benefit machine learning for mental healthcare research . they demonstrate that language representations pretrained in the target domain improve the performance of mental health detection tasks. |
| Outcome: | The proposed models improve mental health detection tasks on several benchmarks and are available for free. |
Copied to clipboard
| Challenge: | Treebank translation is a promising method for cross-lingual transfer of syntactic dependency knowledge. |
| Approach: | They propose to map dependency arcs from source treebank to target translation according to word alignments. |
| Outcome: | Experiments on university dependency treebanks show that translated treebank translations are more effective than translated treebans. |
Copied to clipboard
| Challenge: | Entity alignment (EA) aims to identify entities in different knowledge graphs (KGs) that represent the same real-world object. |
| Approach: | They propose an end-to-end EA framework based on large language models that requires no training to implement. |
| Outcome: | The proposed framework significantly reduces the reliance on seed entity pairs while achieving state-of-the-art (SOTA) performance on diverse datasets. |
Copied to clipboard
| Challenge: | Existing approaches to extract entity and relation feature are flawed because they do not consider the intimate connection between NER and RE. |
| Approach: | They propose a partition filter network to model two-way interaction between tasks . they leverage two gates: entity and relation gate, to segment neurons into two task partitions and one shared partition. |
| Outcome: | The proposed model performs significantly better than previous approaches on six public datasets. |
Copied to clipboard
| Challenge: | Discourse representation tree structure (DRTS) parsing is a new semantic parser which ignores structural information. |
| Approach: | They propose a structural-aware model to integrate structural information into the model . they use graph attention network (GAT) to exploit structural information for effective modeling . |
| Outcome: | The proposed model can achieve the best performance on a benchmark dataset. |
Copied to clipboard
| Challenge: | Existing approaches to sequence labeling are limited due to the scarcity of domain-specific data and semantic distribution biases in domain-based contexts. |
| Approach: | They propose a framework that integrates an LLM-based knowledge enhancement workflow with a span-based Knowledge Fusion for Rich and Efficient Extraction model. |
| Outcome: | The proposed model achieves state-of-the-art performance on multiple domain-specific sequence labeling datasets and is highly efficient. |
Copied to clipboard
| Challenge: | Existing approaches focus on action selection or use pre-trained models as world models to enhance planning capabilities. |
| Approach: | They propose a new learning framework that optimizes state prediction and action selection through preference learning. |
| Outcome: | The proposed method outperforms existing methods and GPT-4o on VoTa-Bench and Qwen2-VL (7B), LLaVA-1.6 (7B) and LLama-3.2 (11B). |
Copied to clipboard
| Challenge: | Recent advances in large language models have demonstrated RL's substantial capacity to enhance multi-step reasoning beyond what supervised instruction tuning achieves. |
| Approach: | They propose a framework that converts multimodal questions into descriptive text . they propose RL-enhanced geoscience reasoning that can be fine-tuned to a text-only level . |
| Outcome: | The proposed framework improves accuracy and accuracy on multimodal questions while preserving answerability and difficulty. |
Copied to clipboard
| Challenge: | Existing knowledge editing paradigms suffer from editing decoupling failures . entity knowledge is sequestered into disentangled modality-specific pathways . |
| Approach: | They propose a method that explicitly disentangles and localizes modality-specific neuron groups for targeted knowledge. |
| Outcome: | The proposed method outperforms baselines in reliability and consistency while preserving model locality. |
Copied to clipboard
| Challenge: | Existing methods for decoding autoregressive models are temperature scaling and nucleus sampling to balance diversity and coherence. |
| Approach: | They propose a training-free decoding strategy that uses a model with a low perplexity score to select the trial with the lowest perplexities as the most probable and reliable path. |
| Outcome: | The proposed approach outperforms existing standard decoding strategies consistently by a clear margin. |
Copied to clipboard
| Challenge: | Existing work focuses on assessing in-domain knowledge, but shedding light on what pre-trained Language Models learn is important. |
| Approach: | They propose a method to assess a PLM's generalization capacity in biased scenarios by combining component combinations where it could be easy for the PLMs to learn shortcuts from the training corpus. |
| Outcome: | The proposed model can overcome distribution shifts in the training corpus and with sufficient data. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) based agent systems have made great strides in real-world applications beyond traditional NLP tasks. |
| Approach: | They propose a new LLM-based Multi-Agent System benchmark, Collab-Overcooked, built on the popular Overcooked-AI game with more applicable and challenging tasks in interactive environments. |
| Outcome: | The proposed benchmark provides a multi-agent framework supporting diverse tasks and objectives and encourages collaboration through natural language communication. |
Copied to clipboard
| Challenge: | Existing methods to learn vital information from dialogue context with limited data are limited due to limited words in utterances and huge gap between dialogue and its summary. |
| Approach: | They propose an unsupervised strategy to learn vital information from dialogue context . the proposed model uses a hypothetical foundation that a superior summary approximates a replacement of the original dialogue . |
| Outcome: | The proposed model outperforms existing models on a number of datasets. |
Copied to clipboard
| Challenge: | Existing RL methods suffer from reliability bottlenecks due to reward sparsity and intractable computations . d-TreeRPO provides fine-grained and verifiable step-wise reward signals . |
| Approach: | They propose a reliable reinforcement learning framework for diffusion large language models that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards. |
| Outcome: | The proposed framework outperforms baseline models and achieves significant improvements across reasoning benchmarks. |
Copied to clipboard
| Challenge: | Existing methods focus on sentence-level or singledocument settings, resulting in one-sided relation transfer contextual bias and incomplete reasoning chains. |
| Approach: | They propose a framework to explicitly decouple and preserve bidirectional bridge evidence and a dynamic loss optimization objective to separate head and tail contexts. |
| Outcome: | The proposed framework decouples and preserves bidirectional bridge evidence while capturing global dependencies through iterative message passing. |
Copied to clipboard
| Challenge: | Existing approaches to cluster graphs with GNNs are limited due to label scarcity. |
| Approach: | They propose to leverage large language models to enhance text-attributed graph clustering by using three LLMs as ranking-based supervision signals. |
| Outcome: | The proposed approach generates reliable guidance using collaboration of three LLM-based agents as ranking-based supervision signals. |
Copied to clipboard
| Challenge: | Existing benchmark datasets focus on low-level cognitive tasks while providing limited coverage of higher-level reasoning skills. |
| Approach: | They analyze the cognitive depth of popular LLM benchmarks using Bloom’s Taxonomy to evaluate both the cognitive and knowledge dimensions. |
| Outcome: | The results show that incorporating higher-level cognitive instructions into the current instruction fine-tuning process improves model performance. |
Copied to clipboard
| Challenge: | Prior research has focused on optimizing general-purpose large language models to downstream tasks . however, these approaches inherently introduce data dependency, which hinders generalization and reusability. |
| Approach: | They propose an algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. |
| Outcome: | The proposed algorithm achieves 2% performance improvement over baselines on three tasks while localizing the most task-sensitive attention heads. |
Copied to clipboard
| Challenge: | Existing offline alternatives to Reinforcement Learning from Human Feedback (RLHF) are available at https://github.com/AIR-hl/MWPO. |
| Approach: | They propose an offline method to optimize preference pairs based on implicit reward margins and response length margins by reweighting them using a geometric mixture. |
| Outcome: | The proposed method outperforms state-of-the-art methods on four different scales and reduces generation length by 9.4%. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive capabilities in following instructions, but manually prompting them to exhibit certain personalities may result in sub-optimal performance. |
| Approach: | They propose a plug-and-play prompting method to manipulate Large Language Models with distinct human-like personality traits by appending discrete personalized suffixes to query or dialog histories and focusing exclusively on influential tokens. |
| Outcome: | The proposed method outperforms other prompting methods and model editing methods on four models ranging from 1.1B to 13B and achieves 79.9% accuracy in customizing LLMs’ personalities. |
Copied to clipboard
| Challenge: | Unlike most of the previous work focusing on the English language, this paper focuses on the Chinese ORL task. |
| Approach: | They propose to use a standard English MPQA dataset to construct a Chinese ORL dataset and investigate the effectiveness of cross-lingual transfer methods. |
| Outcome: | The proposed method is able to detect and improve the performance of the proposed method in Chinese. |
Copied to clipboard
| Challenge: | Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance. |
| Approach: | They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning. |
| Outcome: | The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance. |
Copied to clipboard
| Challenge: | Existing approaches to stream learning NLP tasks suffer from catastrophic forgetting and are exacerbated when the previous task’s pseudo data is insufficient. |
| Approach: | They propose to use a new data format to train pseudo questions of previous tasks to stream learning NLP tasks while retaining knowledge of previous ones. |
| Outcome: | The proposed model is more robust to sufficient and insufficient pseudo-data when the task boundary is both clear and unclear. |
Copied to clipboard
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
Copied to clipboard
| Challenge: | Large language models have shown remarkable capabilities in open information extraction, but their resource requirements often restrict their deployment in resource-constrained industrial settings. |
| Approach: | They introduce an ultra-lightweight large language model trained on instruction-based samples in Chinese, English, Korean, and Russian. |
| Outcome: | The proposed model outperforms large-scale models with up to 70B parameters, reducing computational resources by 140x and delivering 11x faster response times. |
Copied to clipboard
| Challenge: | Large Language Models are a powerful tool for medical research, but the data is a bottleneck. |
| Approach: | They propose to use the largest ever medical Question Answering dataset with 26 Million QA pairs as a fine-tuning data for training large language models. |
| Outcome: | The proposed dataset demonstrates that it can be used to train large language models and improves zero-shot performance on other datasets. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract relations between entities in a document. |
| Approach: | They propose an entity pair-guided relation summarization and retrieval model for DocRE . the model uses entity pairs to guide relation summaries and retrievals . |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance on three datasets. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have transformed ML/AI development . a reevaluation of AutoML principles for Retrieval-Augmented Generation (RAG) systems is needed. |
| Approach: | They propose a framework for hyper-parameter tuning and a hierarchical MAB method for efficient exploration of large search spaces. |
| Outcome: | The proposed framework outperforms baseline methods in more challenging optimization scenarios. |
Copied to clipboard
| Challenge: | chain-of-thought (CoT) prompting has been shown to be effective on complex reasoning tasks, but the naive greedy decoding used in CoT prompting causes the repetitiveness and local optimality. |
| Approach: | They propose a generalizable ensemble-optimization method that uses a set of reasoning paths to prompt a language model one more time to determine the optimal answer. |
| Outcome: | The proposed method can be generalized to almost all scenarios where the type of input questions and answer format of reasoning paths may be unknown. |
Copied to clipboard
| Challenge: | Language Models (LMs) are increasingly challenging the dominance of domain-specific models, such as Graph Neural Networks (GNNs) and Graph Transformers (GTs). |
| Approach: | They propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the art (SOTA) GNNs on node classification tasks without requiring any architectural modifications. |
| Outcome: | The proposed approach outperforms existing GNNs on node classification tasks and is open-source upon publication. |
Copied to clipboard
| Challenge: | Recent advances in large language models have highlighted the effectiveness of chain-of-thought reasoning in symbolic domains such as mathematics and programming. |
| Approach: | They propose a tool-augmented protein reasoning agent that unifies problem decomposition, tool invocation, and grounded answer generation. |
| Outcome: | The proposed protein function understanding agent outperforms text-only reasoning models with an average performance improvement of 103%. |
Copied to clipboard
| Challenge: | a dedicated study on orthogonality constraints for transformers has been lacking . plug-and-play constraints increase the BLEU of transformers . |
| Approach: | They propose to use plug-and-play constraints to encourage matrices to be orthogonal for numerical stability. |
| Outcome: | The proposed constraint increases the BLEU on the large-scale WMT’16 EnDe benchmark by a factor of 28.4 to 29.6. |
Copied to clipboard
| Challenge: | Injecting unaligned knowledge tuple into large language models achieves comparable (and sometimes better) results than aligned knowledge. |
| Approach: | They propose a technique to inject random knowledge into large language models to improve performance. |
| Outcome: | The proposed technique overcomes the sanity problem and pushes the performance limit. |
Copied to clipboard
| Challenge: | Metaphors are a prominent linguistic device in human language and literature, as they add color, imagery, and emphasis to enhance effective communication. |
| Approach: | They propose a large-scale high quality annotated Chinese Metaphor Corpus . they use a set of guidelines to ensure the accuracy and consistency of their annotations . |
| Outcome: | The proposed corpus generates metaphors that resonate more with real-world intuition. |
Copied to clipboard
| Challenge: | Multi-hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the reasoning process. |
| Approach: | They propose a three-stage framework based on complex question decomposition that decomposes the complex question, then reads the sub-questions and then performs numerical comparison to get the final answer. |
| Outcome: | The proposed framework achieves state-of-the-art in the 2WikiMultiHopQA dataset, with a winning joint F1 score of 53.58 on the leaderboard. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | Seq2Edit approaches still face several challenges such as inflexibility in generation and difficulty in generalizing to other languages. |
| Approach: | They propose a non-autoregressive text editing method that models the edit process with latent CTC alignments and introduces the copy operation into the edit space. |
| Outcome: | The proposed method outperforms existing Seq2Edit models and achieves similar or even better results than Seq1Edit with over 4 speedup. |
Copied to clipboard
| Challenge: | Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models. |
| Approach: | They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks. |
| Outcome: | The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are hampered by inaccuracies and outdated information. |
| Approach: | They propose a framework that constructs knowledge graphs using available information to guide the direction of knowledge editing. |
| Outcome: | The proposed framework allows consistent, aligned, and stable information during large-scale editing scenarios. |
Copied to clipboard
| Challenge: | Despite significant advances in video-language modeling, hallucinations remain a persistent challenge in video large language models. |
| Approach: | They present a systematic taxonomy that categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
| Outcome: | The proposed taxonomy categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
Copied to clipboard
| Challenge: | Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences. |
| Approach: | They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner. |
| Outcome: | The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner. |
Copied to clipboard
| Challenge: | Existing methods for Chain-of-Thought evaluations do not distinguish between genuine reasoning and mere verbosity. |
| Approach: | They propose a framework for the non-intrusive, comprehensive process-centric evaluation of reasoning grounded in First-Order Logic. |
| Outcome: | The proposed framework identifies four distinct behavioral prototypes and diagnoses the failure modes. |
Copied to clipboard
| Challenge: | Existing non-simultaneous sign language translation methods suffer from inherent inference delays in real-time scenarios. |
| Approach: | They propose an adaptive policy for simultaneous sign language translation that progressively converts incrementally received sign video into its corresponding natural sentence. |
| Outcome: | The proposed policy excels in situations requiring extremely low latency. |
Copied to clipboard
| Challenge: | Existing studies have demonstrated that direct preference optimization (DPO) can be effective in generalizing large language models, but its effectiveness in video domain remains limited. |
| Approach: | They propose a framework that utilizes detailed video captions as a proxy of video content to enable language models to incorporate this information as supporting evidence for scoring video Question Answering (QA) predictions. |
| Outcome: | The proposed framework shows that it can be used to align language models with video content and improves performance on open-ended video QA tasks. |
Copied to clipboard
| Challenge: | Empirical studies on four public NER datasets demonstrate the effectiveness of our proposed method. |
| Approach: | They propose a method to perform named entity recognition using unlabeled data and named entity dictionaries. |
| Outcome: | The proposed method can estimate task loss as if there is fully labeled data. |
Copied to clipboard
| Challenge: | Personalized dialogue generation is a popular approach for conversational AI applications . however, persona profiles may not provide comprehensive descriptions of the persona . |
| Approach: | They propose a method that leverages persona profiles and dialogue context to generate personalized dialogues by leveraging personas and persona profile. |
| Outcome: | The proposed method outperforms baselines on the CONVAI2 dataset . it is expected to generate personalized dialogues based on persona profiles and dialogue context . |
Copied to clipboard
| Challenge: | Existing household robots are inability to recognize potential problems or dangers in home environments. |
| Approach: | They propose a task of creating anomaly scenarios using generative models instead of manually labeled data to build simulated environments. |
| Outcome: | The proposed framework outperforms existing models in terms of task description and scene diversity. |
Copied to clipboard
| Challenge: | Branch-Train-Stitch (BTS) is an efficient and flexible training algorithm for combining independently trained large language model (LLM) experts into a single, capable generalist model. |
| Approach: | They propose an efficient and flexible training algorithm for combining large language model (LLM) experts into a single, capable generalist model using lightweight stitch layers. |
| Outcome: | The proposed model can generalize to new domains despite being frozen . it yields the best generalist performance on a variety of downstream tasks, retaining the specialized capabilities of each of the experts. |
Copied to clipboard
| Challenge: | Positional biases in large language models hinder their ability to process long inputs. |
| Approach: | They propose a benchmark to assess positional bias in large language models involving multiple pieces of relevant information. |
| Outcome: | The proposed benchmark assesses the performance of long-context language models by examining their models with different input lengths and tasks. |
Copied to clipboard
| Challenge: | Existing methods for relational triple extraction ignore semantic information of relations or predict subjects and objects sequentially. |
| Approach: | They propose a relation-first blank filling network to capture semantic information of relations . they transform relations into relation templates with blanks which contain the fine-grained semantic representation of relations. |
| Outcome: | The proposed model outperforms current state-of-the-art methods on public benchmark datasets. |
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating long-context language models employ irrelevant noise texts to artificially extend the length of test cases, diverging from the real-world scenarios of long-constituency applications. |
| Approach: | They propose a long-context benchmark, Loong, aligning with realistic scenarios through extended multi-document question answering (QA) . |
| Outcome: | The proposed model can scale up the context window of large language models to perform in-depth analysis of multiple long documents. |
Copied to clipboard
| Challenge: | Existing multimodal question answering models rely on sequential retrieval and reasoning, but this single-path paradigm makes them vulnerable to errors due to misleading intermediate steps. |
| Approach: | They propose a multimodal multi-hop question answering framework guided by an Adaptive Planning Graph . they propose modality-specific strategies that dynamically adapt to distinct data types . |
| Outcome: | The proposed framework outperforms existing models that rely on training. |
Copied to clipboard
| Challenge: | Existing methods for ERC lack interpretability and shallow semantics capture deep semantics. |
| Approach: | They propose a Fast-Slow thinking framework for Emotion Recognition in Conversation . they use fine-grained emotion reasoning chains to capture deep semantics . |
| Outcome: | The proposed framework achieves state-of-the-art in explanation and judgment on a benchmark dataset. |
Copied to clipboard
| Challenge: | Existing models for natural language processing are heavily parameterized and memory inefficient. |
| Approach: | They propose a series of lightweight and memory efficient neural architectures for NLP tasks . they propose quaternion algebra and hypercomplex spaces for computation . |
| Outcome: | The proposed models enable expressive inter-component interactions and significantly reduce parameter size without loss of performance. |
Copied to clipboard
| Challenge: | Commosense knowledge graphs (CKGC) are powerful representations of real-world commonsense knowledge. |
| Approach: | They propose a framework that uses automatically generated prompt templates combined with pre-trained language models to improve CKGC performance. |
| Outcome: | The proposed framework mitigates the long-tail problem and improves CKGC performance on a large dataset. |
Copied to clipboard
| Challenge: | Current research focuses on purely MGT detection without adequately addressing mixed scenarios including AI-revised Human-Written Text (HWT) and human-revealed MGT. |
| Approach: | They define mixtext, a form of mixed text involving both AI and human-generated content, and then use a MixSet dataset to assess their effectiveness. |
| Outcome: | The proposed detectors struggle to identify mixtext, particularly in dealing with subtle modifications and style adaptability. |
Copied to clipboard
| Challenge: | OpenAI's GPT-4 has demonstrated remarkable multimodal capabilities, but specific mechanics of GPT4 remain unknown. |
| Approach: | They propose a data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning. |
| Outcome: | The proposed method improves on ten commonly assessed models and provides greater flexibility compared to existing methods. |
Copied to clipboard
| Challenge: | Existing approaches to improve model performance on few-shot or zero-shot datasets are not effective for Chinese few- shot NER. |
| Approach: | They propose a prompt-based Parent and Child BERT for Chinese few-shot NER to train an annotating model on high-resource datasets and then discover more implicit labels on low-resourced datasets. |
| Outcome: | The proposed model can be used on Weibo and other Chinese NER datasets and it is shown to be effective in few-shot learning. |
Copied to clipboard
| Challenge: | Recent studies in deep learning have shown significant progress in named entity recognition (NER) . however, most existing works assume clean data annotation, while real-world data typically involve a large amount of noises. |
| Approach: | They propose a confidence estimation approach for named entity recognition using noisy labels using local and global independence assumptions. |
| Outcome: | The proposed method marginalizes out labels of low confidence with a CRF model and integrates it into a self-training framework for boosting performance. |
Copied to clipboard
| Challenge: | Existing methods for multimodal sentiment analysis are often dynamically incomplete. |
| Approach: | They propose a new uncertainty-calibrated elastic alignment framework to address these issues by employing probabilistic imputation to capture cross-modal ambiguity and leverage the estimated uncertainty to drive elastic alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple benchmarks and consistently outperformed existing models. |
Copied to clipboard
| Challenge: | Existing work on opinion role labeling (ORL) is highly correlative with semantic role labeled (SRL) SRL is used to identify opinion holders and holder expressions for a given predicate. |
| Approach: | They propose a method to enhance opinion role labeling by presenting semantic-aware word representations which are learned from SRL. |
| Outcome: | The proposed method outperforms two other methods on a benchmark MPQA corpus and achieves higher F scores. |
Copied to clipboard
| Challenge: | Existing tagging systems that use sentence-level data are not well understood. |
| Approach: | They propose a larger-context approach to tagging tasks that incorporates contextual information into existing tapping systems. |
| Outcome: | The proposed aggregators improve on four tagging tasks and 13 datasets. |
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph extrapolation neglect the complex semantic relationships between relations when modeling their dynamic evolution. |
| Approach: | They propose a method for extracting semantic relationships to achieve TKG extrapolation . they use large language models to analyze the types of relations in TKGs . |
| Outcome: | The proposed method improves on five TKG datasets and shows performance gains. |
Copied to clipboard
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |
Copied to clipboard
| Challenge: | Tabular data is an essential resource for many fields, but current methods do not fully utilize the rich information available in tables. |
| Approach: | They propose a method that utilizes metadata information to generate tabular data . they propose long-token partitioning algorithms that enable AIGT to model tables of any scale . |
| Outcome: | The proposed approach achieves state-of-the-art on 14 out of 20 public datasets and two real industry datasets within the Alipay risk control system. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have paved the way for complex tasks such as role-playing. |
| Approach: | They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models. |
| Outcome: | The proposed framework improves role-playing abilities with 168,093 samples. |
Copied to clipboard
| Challenge: | Existing ensemble methods for ensembling large language models rely on fixed weighting strategies that fail to adapt to dynamic, context-dependent characteristics of LLMs. |
| Approach: | They propose a framework that reformulates LLM ensemble through a Markov Decision Process. |
| Outcome: | The proposed framework outperforms existing methods by 3.3% on a diverse set of tasks while achieving lower time latency. |
Copied to clipboard
| Challenge: | Enterprise LLM agents can dramatically improve workplace productivity, but their core capability, retrieving and using internal context to act on a user’s behalf, also creates new risks for sensitive information leakage. |
| Approach: | They propose a Contextual Integrity-grounded benchmark that simulates enterprise workflows across five information-flow directions and evaluates whether agents can convey *essential* content while withholding *sensitive* context in dense retrieval settings. |
| Outcome: | The proposed model demonstrates that privacy failures are prevalent in enterprise workflows and that higher task utility correlates with increased privacy violations. |
Copied to clipboard
| Challenge: | Continual pretraining is an important approach for Large Language Models to improve their performance in target domains, learn new topics and languages, and even boost their general capabilities. |
| Approach: | They propose a training strategy that mitigates instability by increasing the number of epochs, along with two data sampling strategies targeting data domain relevance and corpus distribution. |
| Outcome: | The proposed training strategy improves the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% using only 40% of the original training budget, while also enhancing general task performance without causing forgetting. |
Copied to clipboard
| Challenge: | Existing OCR-free approaches to document visual question answering are brittle and passive. |
| Approach: | They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation. |
| Outcome: | The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline. |
Copied to clipboard
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
Copied to clipboard
| Challenge: | Existing methods for solving math word problems ignore background common-sense knowledge . a novel knowledge-aware sequence-to-tree (KA-S2T) network incorporates external knowledge and global expression information. |
| Approach: | They propose a knowledge-aware sequence-to-tree network that incorporates external knowledge and global expression information into the problem. |
| Outcome: | The proposed model can achieve better performance than previous models on a Math23K dataset. |
Copied to clipboard
| Challenge: | Existing methods for zero-shot relationship extraction do not distinguish between unseen, semantically similar relations. |
| Approach: | They propose a framework to enable global reasoning across a set of predictions. |
| Outcome: | The proposed framework outperforms existing methods and establishes new state-of-the-art results on widely used datasets. |
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) generate extensive chain-of-thought reasoning, but we lack a principled framework for understanding how these thoughts are structured. |
| Approach: | They propose a method to analyze the reasoning traces of Large Reasoning Models using Schoenfeld’s Episode Theory. |
| Outcome: | The proposed framework provides a theoretically grounded methodology for interpreting LRM cognition and enables future work on more controllable and transparent reasoning systems. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated potential reasoning capabilities through prompt design, such as the Chain of Thought (CoT). |
| Approach: | They propose a new reasoning approach that predicts key entities which work as important “anchors” and employs a ranking algorithm to ensure the logical sequence of the predicted answers. |
| Outcome: | The proposed approach outperforms existing methods in multi-hop question reasoning and provides more accurate reasoning results in multihop question answering tasks. |
Copied to clipboard
| Challenge: | Existing methods for Chinese sequence labelling only fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. |
| Approach: | They propose a Lexicon Enhanced BERT model which integrates external lexicon knowledge into BERT layers directly by a lexiccon Adapter layer. |
| Outcome: | The proposed model integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. |
Copied to clipboard
| Challenge: | Syntax integration has been demonstrated highly effective in neural machine translation (NMT). |
| Approach: | They propose a method to integrate source-side syntax implicitly for neural machine translation . they use hidden representations of a well-trained end-to-end dependency parser to concatenate them with ordinary word embeddings to enhance basic NMT models. |
| Outcome: | The proposed method outperforms existing methods on two translation tasks . it can be easily integrated into the widely-used sequence-to-sequence (Seq2Sequen) framework . |
Copied to clipboard
| Challenge: | Reinforcement Learning with Verifiable Rewards (RLVR) has shown significant promise for enhancing the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a model-free method that refines credit assignment by leveraging the model's internal uncertainty signals. |
| Outcome: | Extensive experiments on five mathematical reasoning benchmarks show that the proposed method outperforms strong RLVR baselines on multiple model scales, including 1.5B and 7B. |
Copied to clipboard
| Challenge: | Large language models generate human-like content, but they also pose a problem with generation diversity, negatively impacting generation diversity and user experience. |
| Approach: | They propose a Logits-Addition watermark and three variants that aim to enhance diversity to overcome generation diversity challenges. |
| Outcome: | The Logits-Addition watermark outperforms the Logits+Trick-based watermark in diversity tests and outperformed other decoding-based methods by 0.1 to 0.3. |
Copied to clipboard
| Challenge: | Web applications (web apps) are a key arena for large language models to demonstrate their code generation capabilities and commercial potential. |
| Approach: | a new benchmark for large language models (LLMs) is designed to provide real-world user requirements and generalizable evaluation metrics. |
| Outcome: | a new benchmark for large language models (LLMs) provides a real-world, generalizable, and interpretable evaluation score . the benchmark measures user requirements, expression styles and human-preference-aligned weights . a web application can be used to demonstrate its commercial potential, authors say . |
Copied to clipboard
| Challenge: | Recent works of SRL mainly fall into two lines: 1) BIO-based; 2) span-based. |
| Approach: | They propose to regard flat argument spans as latent subtrees, thus reducing SRL to a tree parsing task. |
| Outcome: | The proposed model performs better than previous syntax-agnostic models on CoNLL05 and CoNll12 benchmarks. |
Copied to clipboard
| Challenge: | Aspect-term sentiment analysis (ATSA) is a fine-grained task that aims to infer the sentiment towards the given aspect-terms. |
| Approach: | They propose a novel ATSA method that is interpretable and has high accuracy . they propose SILTN, which is a neurosymbolic formalism, to improve the accuracy based on syntax knowledge distillation. |
| Outcome: | The proposed method is interpretable because it is a neurosymbolic formalism and a computational model that supports learning and reasoning about data with a differentiable first-order logic language. |
Copied to clipboard
| Challenge: | Recent advances in foundation models have impeded the ability for these fields to benefit from each other’s advancements. |
| Approach: | They propose to use a multimodal foundation model to unify various modalities and recommendation tasks under the P5 recommendation paradigm to implement personalized prompts. |
| Outcome: | The proposed model will unify visual, textual, and personalization modalities under the P5 recommendation paradigm and will improve recommendation performance and efficiency. |
Copied to clipboard
| Challenge: | Existing large language models for software engineering rely on coarse-grained pass rates obscuring specific cognitive bottlenecks. |
| Approach: | They propose a repository-level benchmark that dissects coding capabilities through atomized tasks. |
| Outcome: | The proposed framework achieves a 78.55% validity yield, surpassing the 31.7% retention rate of SWE-bench-Verified. |
Copied to clipboard
| Challenge: | Discourse parsing on multi-party dialogues is an important but difficult task in dialogue systems and conversational analysis. |
| Approach: | They propose a speaker-aware model for parsing on multi-party dialogues using interaction features between different speakers. |
| Outcome: | The proposed model achieves the best-reported performance on two standard benchmark datasets. |
Copied to clipboard
| Challenge: | Existing approaches to cross-document relation extraction (RE) focus on identifying relations between head and tail entities from single sentence or document. |
| Approach: | They propose a hierarchical relation tree-based LLM-based hierarchic classification model for cross-document relation extraction (HCRE) based on predefined relations, the model can perform hierarchically classification level by level. |
| Outcome: | The proposed model outperforms existing baselines and validates its effectiveness. |
Copied to clipboard
| Challenge: | Emotion recognition in conversation (ERC) is a task of discerning human emotions for each utterance within a conversation. |
| Approach: | They propose a framework that uses large language models to analyze speaker characteristics . they use two-stage learning to make the models reason speaker characteristics and track emotion of the speaker . |
| Outcome: | The proposed framework outperforms existing methods on three benchmark datasets. |
Copied to clipboard
| Challenge: | Existing methods for intent classification are limited due to fast-emerging intents . a recent study shows that existing methods are not effective in recognizing unseen intents. |
| Approach: | They propose to reconstruct capsule networks for zero-shot intent classification by using latent information from labeled utterances. |
| Outcome: | The proposed method outperforms existing methods on two task-oriented dialogue datasets in different languages. |
Copied to clipboard
| Challenge: | Existing static strategies for mitigating hallucinations do not explicitly model the information gain from interacting with the external environment. |
| Approach: | They propose a calibration-driven interactive learning strategy that selects clarification queries by optimizing calibration error. |
| Outcome: | The proposed method provides theoretical guarantees and empirical gains for reliability. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) provides a broad context for extracting relations for entities. |
| Approach: | They propose a method that utilizes LLMs as a refiner and task distribution and probability fusion to refine LLM-based relation extraction methods. |
| Outcome: | The proposed method outperforms existing LLM-based methods without fine-tuning by 25.2% F1. |
Copied to clipboard
| Challenge: | Existing methods for ERC lack human-like emotion reasoning and discrimination between similar emotions. |
| Approach: | They propose a multi-dimension curriculum with long CoT fine-tuning to clone human-like emotion reasoning for conversational emotion recognition. |
| Outcome: | The proposed model outperforms existing methods on three widely used datasets and shows that it is more intuitive and more accurate. |
Copied to clipboard
| Challenge: | Existing systems suffer from linguistic collapse when pursuing high intensity or fail to meet target emotional levels. |
| Approach: | They propose an inference framework that introduces a neutral prosody bias and a uniform Classifier-Free Guidance that distorts the acoustic manifold, leading to artifacts. |
| Outcome: | The proposed framework achieves superior linguistic accuracy and expressiveness without model retraining. |
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) is one of the most efficient parameter-efficient fine-tuning methods. |
| Approach: | They propose to conceptualize each LoRA module as a beam where each rank corresponds to a potential sub-solution. |
| Outcome: | The proposed method improves performance on three base models and 12 datasets. |
Copied to clipboard
| Challenge: | Existing annotation tools lack support for Large Language Models (LLMs) or use LLMs as one-off preannotation engines, compromising data reliability. |
| Approach: | They propose a text annotation platform that embeds LLM-assisted labeling into a quality-aware collaborative workflow. |
| Outcome: | Experiments show that BNLP reduces annotation time by 74.3% and improves annotation quality by 11.6% over purely manual annotation in LLM-assisted settings. |
Copied to clipboard
| Challenge: | Recent studies show that KV cache compression can increase hallucination scores in LLMs . modern LLM models support extremely long sequences, but their impact on model hallucinosity remains underexplored. |
| Approach: | They propose a decoding-phase strategy that selectively removes generated KV pairs from retrieval heads responsible for retrieving critical information from source context. |
| Outcome: | The proposed method reduces hallucination across multiple models and datasets while preserving computational efficiency. |
Copied to clipboard
| Challenge: | Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns. |
| Approach: | They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users. |
| Outcome: | The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for predicting future facts from time-evolving graphs rely on statistical co-occurrences and extensive path enumeration. |
| Approach: | They propose a Critic-Guided Rule Induction method which treats temporal rules as rule hypotheses to be examined and adopts a decoupled Generation-Discrimination pipeline to induce rules that are high-coverage and high-precision. |
| Outcome: | The proposed method outperforms strong baselines on three benchmarks and achieves state-of-the-art performance. |
Copied to clipboard
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
Copied to clipboard
| Challenge: | In-context learning paradigms that focus on large corpus are limiting compositional generalization performance. |
| Approach: | They propose a test suite to investigate in-context compositional generalization . they propose to use examples that are structurally similar to the test case . |
| Outcome: | The proposed test suite investigates in-context compositional generalization performance . it finds that the performance can be affected by the selection of in-const examples . |
Copied to clipboard
| Challenge: | Existing methods to extract triplets for unseen relations rely on costly fine-tuning and lack structured semantic guidance. |
| Approach: | They propose a framework that adopts a "frame first, then extract" paradigm to extract triplets from unstructured text. |
| Outcome: | The proposed framework achieves competitive zero-shot performance on multiple benchmarks and can be used to enhance existing extraction methods. |
Copied to clipboard
| Challenge: | Existing methods for creating video content are limited by high costs and slow update cycles. |
| Approach: | They propose a paradigm shifting educators from manual creators to high-level directors who focus on pedagogical intents while agents handle execution. |
| Outcome: | The proposed framework reduces production costs to 0.3% of traditional course videos and provides a robust solution for scalable education. |
Copied to clipboard
| Challenge: | Recent years have seen remarkable success in the use of deep neural networks on Chinese word segmentation (CWS) however, the performance of CWS systems has gradually reached a plateau with the rapid development of deep networks. |
| Approach: | They propose a fine-grained evaluation for existing Chinese word segmentation systems that allows us to diagnose the strengths and weaknesses of existing models. |
| Outcome: | The proposed model can diagnose strengths and weaknesses of existing models and alleviate negative transfer problem when doing multi-criteria learning. |
Copied to clipboard
| Challenge: | Existing research on multi-modal Named Entity Recognition (MNER) does not integrate all multi-modal representations to provide rich contextual information to improve NER. |
| Approach: | They propose an iterative reasoning framework that integrates all the diverse multi-modal representations following the strategy of "decompose, prioritize, and eliminate" . they propose to use hierarchically connected fusion layers to prioritize transitions from "easy-to-hard" and "coarse-to fine" |
| Outcome: | The proposed framework integrates all the diverse multi-modal representations following the strategy of "decompose, prioritize, and eliminate". |
Copied to clipboard
| Challenge: | Existing methods for multimodal entity alignment overlook the quality of input modality embeddings during modality interaction, amplifying noise propagation while suppressing discriminative feature representations. |
| Approach: | They propose a model for capturing latent modal association for multimodal entity alignment using a self-attention mechanism to enhance salient information while attenuating noise within individual modality embeddings. |
| Outcome: | The proposed model achieves an absolute 3.1% higher Hits@1 score than the sota method. |
Copied to clipboard
| Challenge: | Existing VLMs lack robust grounded norm understanding, a new study finds . current VLM models lack robust grounding, despite a high score for safety and privacy . |
| Approach: | They propose a pipeline to generate grounded MCQs from ego-centric videos of human interactions. |
| Outcome: | The proposed pipeline can generate grounded MCQs from egocentric video . it shows that current VLMs lack robust grounded norm understanding . |
Copied to clipboard
| Challenge: | Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. |
| Approach: | They propose a hierarchical framework for long-horizon navigation that integrates human instructions with 3D scene views. |
| Outcome: | The proposed model achieves SOTA results and can complete long-horizon navigation tasks across different robot embodiments in real-world environments. |
Copied to clipboard
| Challenge: | Existing methods for sign language translation (SLT) rely on signer identity labels, which is often impractical and costly in real-world applications. |
| Approach: | They propose a signer diversity-driven data augmentation method that can generalize to signers not encountered during training. |
| Outcome: | The proposed method achieves state-of-the-art results without relying on signer identity labels. |
Copied to clipboard
| Challenge: | a new framework for topic evolution and stance dynamics is needed to understand online discourse . topic evolution is central to understanding fragmentation of debates, spread of misinformation . |
| Approach: | They propose a stance and topic evolution reasoning framework for co-evolution of topics and stances through natural language interactions. |
| Outcome: | The proposed framework captures key empirical patterns across five real-world domains. |
Copied to clipboard
| Challenge: | Existing approaches to integrate large language models into cross-lingual entity alignment tasks pose challenges in handling large-scale data, generating suitable data samples, and adapting prompts for the EA task. |
| Approach: | They propose a framework that integrates distance feature extraction, sample **Seg**mentation, and zero-shot prompts to integrate LLMs into cross-lingual entity alignment tasks. |
| Outcome: | The proposed framework is able to extract features from large-scale data and adapt prompts to the task. |
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) exhibit impressive cross-modal understanding and reasoning abilities, but many benchmarks suffer from systematic biases. |
| Approach: | They propose a benchmark to avoid Type-I errors by creating one perception question and one knowledge anchor question through a meticulous annotation process. |
| Outcome: | The proposed benchmark avoids Type-I errors while maintaining reliability of MCQ evaluations. |
Copied to clipboard
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Copied to clipboard
| Challenge: | Open-ended Visual Question Answering (VQA) requires models to reason over visual and natural language inputs using world knowledge. |
| Approach: | They propose a new VQA pipeline that deploys a generate-then-select strategy guided by world knowledge for the first time. |
| Outcome: | The proposed pipeline expands the knowledge coverage from in-domain training data by 4.1% on OK-VQA, without additional computation cost. |
Copied to clipboard
| Challenge: | OpenCodeInterpreter-33B provides a high level of performance for code generation, executing, and iterative refinement. |
| Approach: | They propose a family of open-source code systems for generating, executing, and iteratively refining code. |
| Outcome: | The OpenCodeInterpreter-33B performs well on humanEval, MBPP, and EvalPlus benchmarks. |
Copied to clipboard
| Challenge: | Generative engines (GEs) are replacing ranked links with citation-grounded answers . current methods are unable to accumulate or transfer effective strategies across tasks and engines . |
| Approach: | They propose a multi-agent framework where planning, editing, and fidelity-aware evaluation serve as the execution layer. |
| Outcome: | The proposed framework outperforms heuristic baselines in visibility and citation fidelity on three mainstream engines. |
Copied to clipboard
| Challenge: | supervised fine-tuning (SFT) is crucial for multimodal large language models, yet a comprehensive scaling law is lacking . et al.: scaling laws focus on model size, pre-training tokens, and MLLM SFT data volumes . |
| Approach: | They propose two scaling laws to guide optimal model-data configuration . they propose one applicable when training data volumes are well defined by researchers . |
| Outcome: | The proposed scaling laws provide valuable recommendations for optimal resource allocation . they show that the proposed laws are more accurate than existing models . |
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) are based on sentence-level pre-training, which is different from the basic processing unit, i.e. element discourse unit (EDU). |
| Approach: | They propose a second-stage EDU-level pre-training approach to learn effective EDU representations continually based on well pre-trained language models. |
| Outcome: | The proposed method improves F1 score by 2.1 points on a benckmark dataset. |
Copied to clipboard
| Challenge: | Existing multi-agent debate methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is decided by majority voting in the last round. |
| Approach: | They propose a multi-agent debate framework that eliminates the need for consensus among agents and reconstructs the debate phase by introducing anti-conformity. |
| Outcome: | Experiments on eight benchmark datasets show that Free-MAD significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs. |
Copied to clipboard
| Challenge: | Existing methods to extract triplets from context often decompose into named entity recognition and relation classification, which may introduce error propagation. |
| Approach: | They propose a Relation-centric joint ZSRTE method which leverages unseen relation labels to extract triplets in one go. |
| Outcome: | The proposed method achieves state-of-the-art performance with fewer parameters and does not rely on synthetic data or manual labor. |
Copied to clipboard
| Challenge: | Traditional methods address leaks only after content is generated, which can lead to the exposure of sensitive information. |
| Approach: | They propose a proactive approach: examining LLMs’ internal states before text generation to detect potential leaks. |
| Outcome: | The proposed framework ensures adherence to copyright and licensing requirements while enhancing data privacy and ethical standards. |
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented generation (RAG) to long videos are limited by limited context windows and flatten videos into independent segments. |
| Approach: | They propose a structured and intent-aware long-video RAG framework that structures a video as a spatio-temporal graph and then performs multi-hop retrieval to aggregate evidence across distant yet contextually related events. |
| Outcome: | The proposed framework is competitive with state-of-the-art baselines without auxiliary information. |
Copied to clipboard
| Challenge: | Existing work does not fully distinguish the contribution of different mentions to entity representation and the importance of mentions in evidence sentences. |
| Approach: | They propose a document-level relation extraction task that uses entity mentions to identify relations between entities in a text. |
| Outcome: | The proposed model achieves state-of-the-art on widely-adopted datasets. |