Papers by Xiang Zhao
Copied to clipboard
| Challenge: | Existing methods for numerical reasoning are not flexible enough to handle diverse expressions. |
| Approach: | They propose a Relational Graph enhanced Hybrid table-text Numerical reasoning model with Tree decoder which captures relationship between numerical value, table schema, and text information on the encoder side. |
| Outcome: | The proposed model outperforms the baseline model and achieves state-of-the-art results on the publicly available tabletext hybrid QA benchmark. |
Copied to clipboard
| Challenge: | Evidence-enhanced detectors are able to detect malicious social text, but they are prone to evidence pollution. |
| Approach: | They propose three defense strategies to mitigate evidence pollution by large language models by machine-generated text detection and a mixture of experts. |
| Outcome: | The proposed defense strategies could mitigate evidence pollution, but they faced limitations for practical employment. |
Copied to clipboard
| Challenge: | Existing studies show that the ability of large language models to generate contextual understanding of the sentence can degrade translation quality. |
| Approach: | They propose a method that generates contextual understanding for both source and target languages separately. |
| Outcome: | The proposed method outperforms strong comparison methods in multiple domains. |
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is an essential emergent ability of Large Language Models (LLMs). |
| Approach: | They introduce CoT to exemplars of ICL to enhance the reasoning capability . however, it remains unclear whether CoT exemplar is still beneficial for recent, stronger models in such tasks. |
| Outcome: | The enhanced exemplars fail to improve the model’s reasoning performance, despite being constructed using answers from advanced models such as Qwen2.5-Max and DeepSeek-R1. |
Copied to clipboard
| Challenge: | Existing methods for detecting multimedia fake news have demonstrated excellent results . however, addressing event-level inconsistency and learning from poor-quality news remains a challenge . |
| Approach: | They propose an Event-diven fake news detection framework that integrates visual manipulation, textual emotion and multimodal inconsistency at event-level for fake news identification. |
| Outcome: | The proposed framework performs well on three large-scale fake news detection benchmarks. |
Copied to clipboard
| Challenge: | Currently, vision-Language Models are optimized for direct visual question-answering tasks. |
| Approach: | They propose a visual-language-based VLM that prioritizes reasoning within the perception process. |
| Outcome: | The proposed model outperforms existing models and domain-specific open-source models in the chemical domain. |
Copied to clipboard
| Challenge: | Existing methods to model event associations struggle with semantic ambiguity and embedding bias. |
| Approach: | They propose a Semantic and Sentiment Dual-enhanced Generative Model to address these issues . it leverages two types of script event information to enhance the generative model . |
| Outcome: | The proposed model captures both global and local sentiments of events through its sentiment awareness mechanism. |
Copied to clipboard
| Challenge: | Existing approaches to learn relations from labeled data overlook task interference in continual learning and memory requirements for different relations. |
| Approach: | They propose a framework to learn new relations from limited labeled data while preserving knowledge about previously learned relations. |
| Outcome: | The proposed framework is more practical and comprehensive for real-world scenarios. |
Copied to clipboard
| Challenge: | Existing text-to-SQL parsers lack the data to perform well with augmented synthetic data. |
| Approach: | They propose a framework that imposes strong typing constraints and incorporates key relationships from schema. |
| Outcome: | The proposed framework improves on the high-quality synthesized SQL and natural language question (NLQ) models have significant accuracy boosts and achieve new state-of-the-art performance on spider. |
Copied to clipboard
| Challenge: | LR-bench is a high-fidelity, up-to-date benchmark curated from 2024–2025 AI/NLP manuscripts with five-level self-assessed familiarity ratings collected via a large-scale email survey . |
| Approach: | They propose a reviewer-centric ranking framework that distills each reviewer’s recent publications into compact keyword-based profiles and fine-tunes an embedding model with weak preference supervision constructed from heuristic retrieval signals. |
| Outcome: | The proposed framework outperforms existing benchmarks and the CMU gold-standard dataset in the evaluation of AI/NLP manuscripts. |
Copied to clipboard
| Challenge: | Scientific research relies on accurate information retrieval from literature to support analytical decisions. |
| Approach: | They propose a task that automates fine-grained information retrieval *faithfully* grounded in the provided content in response to research-driven queries. |
| Outcome: | The proposed agent achieves 13.2% higher cross-domain accuracy than state-of-the-art RAG and research-agent baselines across seven backbone LLMs. |
Copied to clipboard
| Challenge: | Existing multi agent frameworks for large language models are brittle on code generation tasks. |
| Approach: | They propose a framework that brings pair programming to autonomous LLM collaboration. |
| Outcome: | Using PairCoder, large language models achieve better results on code generation tasks and reduce token usage by 40% to 70% on eight representative backbones. |
Copied to clipboard
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
Copied to clipboard
| Challenge: | Recent advances in large language models have improved multistep reasoning but they lose focus over the middle of long contexts. |
| Approach: | They propose a tree search framework that proactively identifies underutilized steps and minimizing redundant information between steps. |
| Outcome: | The proposed framework generates more accurate and concise rationales with reduced errors and redundancy. |
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) show impressive capabilities across visual–language tasks, but their capacity to evaluate artistic expression remains limited. |
| Approach: | They propose an attribute-specific multi-LoRA approach where each attribute corresponds to a distinct evaluation dimension in the scoring rubric. |
| Outcome: | The proposed approach increases correlation from 0.468 to 0.653 on Qwen2.5-VL-7B, with the largest gains on perceptual dimensions and narrowed gaps on higher-order attributes. |
Copied to clipboard
| Challenge: | Existing methods struggle to capture the visual layout in complex document images. |
| Approach: | They propose to integrate layout knowledge into document image translation by using a layout-aware encoder and a multi-step conductive decoder to achieve the translation step by step. |
| Outcome: | The proposed model outperforms state-of-the-art methods with better parameter efficiency. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
Copied to clipboard
| Challenge: | Existing weakly supervised text classification methods require a large number of annotated data and human annotations are expensive. |
| Approach: | They propose to query a masked language model with cloze style prompts to obtain supervision signals. |
| Outcome: | The proposed method outperforms baseline methods on three datasets by 2%, 4%, and 3%. |
Copied to clipboard
| Challenge: | Existing methods to learn informative entity embeddings are insufficient for semi-supervised entity alignment. |
| Approach: | They propose a semi-supervised method which guides the model learning with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. |
| Outcome: | The proposed method is superior to existing methods on benchmark datasets and further analyses. |
Copied to clipboard
| Challenge: | Entity linking is a fundamental task in natural language processing, says nigel kilgstrom . existing corpora for entity linking in china are lacking and deficient, he says . kilsmstrom: a new method for entity disambiguation can be developed for Chinese . |
| Approach: | They build a Chinese corpus of multi-domain long text for entity linking . they evaluate the difficulty of documents with respect to entity linking using a measure . |
| Outcome: | The proposed corpus is based on 100 documents from diverse domains and is publicly accessible. |
Copied to clipboard
| Challenge: | Existing methods to integrate knowledge graph (KG) with neural machine translation (NMT) have two problems: knowledge under-utilization and granularity mismatch. |
| Approach: | They propose a multi-task learning method on sub-entity granularity to combine machine translation and knowledge reasoning tasks. |
| Outcome: | The proposed method significantly outperforms baseline models on translation tasks and handling the entities. |
Copied to clipboard
| Challenge: | Existing models for large vision language models do not fully reflect their knowledge capacity and reliability, resulting in erroneous outputs that do not align with the image content or provide answers lacking knowledge evidence. |
| Approach: | They propose a Chinese-based benchmark for visual factuality across 8 major topics and 56 subtopics and a multi-hop question construction. |
| Outcome: | The proposed model decouples visual factuality into two parts: seeing the world and discovering knowledge. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly important for their intelligence evaluation. |
| Approach: | They propose a game theory-based evaluation platform that measures LLMs’ decision-making strategies and social behaviors in classic game-theoretic settings. |
| Outcome: | The proposed system cross-evaluates 15 leading LLMs using leaderboard rankings and scoring mechanisms. |
Copied to clipboard
| Challenge: | Currently, the volume and complexity of chat logs makes it difficult to analyze individual conversations. |
| Approach: | They propose a tool that enables fast, versatile, and large-scale conversation analysis by combining search and visualization capabilities with a list of criteria. |
| Outcome: | The proposed tool can be extended to handle millions of chat logs and other datasets. |
Copied to clipboard
| Challenge: | Existing methods to detect causal relationships in unstructured texts ignore trivial knowledge which may prejudice performance. |
| Approach: | They propose a pipeline to build a commonsense-aware pre-trained model which integrates reliable task-specific knowledge from commonsens graphs. |
| Outcome: | The proposed pipeline integrates reliable task-specific knowledge from commonsense graphs. |
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) and Relation Extraction (RE) models have limited success when extracting general schemas such as quadruples and quintuples. |
| Approach: | They propose a formal formulation that covers almost all extraction schemas and a Recursive Method with Explicit Schema Instructor for UIE. |
| Outcome: | The proposed method shows strong performance under full-shot and few-shot settings and achieves state-of-the-art results on the tasks of extracting complex schemas. |
Copied to clipboard
| Challenge: | Time series data is ubiquitous across various domains, including manufacturing, finance, and healthcare. |
| Approach: | They propose a multi-agent system to generate general and domain-specific annotations for time series data. |
| Outcome: | The proposed system outperforms existing methods on synthetic and real-world datasets. |
Copied to clipboard
| Challenge: | Low-resource questions pose a significant challenge within the field of Question-Answering (QA) tasks. |
| Approach: | They propose a method that leverages large models' internal knowledge to enhance the quality of augmented data by Prompt Answer, Question Generation, and Question Filter. |
| Outcome: | The proposed method outperforms existing augmentation strategies on high-resource QA tasks like SQUAD1.1 and TriviaQA. |
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications. |
| Approach: | They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions. |
| Outcome: | The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions. |
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) focus on reducing discrepancy between tokens and tokens, but transfer of valuable label information is often not considered or ignored. |
| Approach: | They propose a framework that borrows entity information from the source domain to enhance NER in the target domain. |
| Outcome: | The proposed model improves over the state-of-the-art model on several datasets. |
Copied to clipboard
| Challenge: | Existing 3D AIGC methods don’t fully unleash human creativity. |
| Approach: | They propose a framework that generates 3D content from multimodal inputs . they propose 198 multimodal text inputs for 3D generation tasks . |
| Outcome: | The proposed framework generates 3D content from multimodal inputs without human intervention. |
Copied to clipboard
| Challenge: | Existing methods to address toxicity issues with large language models are inadequate . lack of domain-specific knowledge leads to false negatives and excessive sensitivity to toxic speech limits freedom of speech. |
| Approach: | They propose a method that leverages graph search on a meta-toxic knowledge graph to enhance hatred and toxicity detection. |
| Outcome: | The proposed method lowers false positive rate and improves toxicity detection performance in out-of-domain scenarios. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown promise in multi-step reasoning tasks, yet relying on extensive manual labeling to provide procedural feedback remains a significant impediment. |
| Approach: | They propose a self-supervised framework that decomposes complex problems into manageable subquestions with a controllable granularity switch and sequentially applies reinforcement learning to iteratively improve the subquest solver. |
| Outcome: | The proposed framework improves performance on mathematical and commonsense reasoning tasks over SOTA. |
Copied to clipboard
| Challenge: | Existing black-box large language models (LLMs) have excellent performance in task-oriented dialogue (TOD) tasks, but obtaining suitable prompts for specific tasks is challenging. |
| Approach: | They propose a black-box large language model that generates domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation. |
| Outcome: | The proposed framework outperforms existing prompting methods on the MultiWOZ 2.0 dataset. |
Copied to clipboard
| Challenge: | Existing methods for detoxification of text often rely on manually annotated data . xiangli: "detoxification of texts is a powerful way to remove toxic content" |
| Approach: | They propose a reinforcement learning framework that optimizes detoxification and semantic preservation without annotating large amounts of data. |
| Outcome: | The proposed method overcomes major limitations and surpasses humanannotated references across multiple benchmarks. |
Copied to clipboard
| Challenge: | Existing efforts to improve CoT prompting have limitations that require extensive human effort or performance needs to be improved. |
| Approach: | They propose a prompt approach for automatic reasoning called LBS3 inspired by curriculum learning which better reflects human learning habits. |
| Outcome: | The proposed approach achieves strongly competitive performance compared to baselines in reasoning-intensive tasks with varying open- and closed-source LLMs. |
Copied to clipboard
| Challenge: | Multi-agent systems (MAS) powered by large language models struggle to adapt to evolving task dependencies and to handle uncertainties. |
| Approach: | They propose a Dynamic Environment-Aware Manager-Player Agents Coordination framework that enhances multi-agent coordination through long-term strategic planning. |
| Outcome: | The proposed framework outperforms traditional reinforcement learning and human-agent collaboration in the Overcooked simulation. |
Copied to clipboard
| Challenge: | Empathy relies on the cognitive capacity to relate to similar past experiences. Existing methods prioritize semantic similarity over emotion characteristics, leading to unempathetic responses. |
| Approach: | They propose a framework that integrates four Emotion Attributes into the retrieval process to ensure explicit emotional alignment. |
| Outcome: | Empirical results show that REG significantly outperforms baselines, offering a robust solution for empathetic generation. |
Copied to clipboard
| Challenge: | Social media platforms provide an ideal environment to spread misinformation, where social bots can accelerate the spread. |
| Approach: | They construct a large-scale dataset that includes annotations for misinformation and social bots on the Sina Weibo platform. |
| Outcome: | The proposed dataset contains 65,749 social bots and 345,886 genuine accounts, annotated using a weakly supervised annotator. |
Copied to clipboard
| Challenge: | Existing supervised relation extraction methods can still misclassify unknown relations into known relations due to the lack of supervision signals. |
| Approach: | They propose a method that regularizes the model by dynamically synthesizing negative instances that can provide the missing supervision signals. |
| Outcome: | The proposed method achieves SOTA unknown relation detection without compromising the classification of known relations. |
Copied to clipboard
| Challenge: | Syntactic language models (SLMs) incorporate syntactical biases into Transformers . authors identify key aspects of design choices in existing models and novel variants based on experimental results . |
| Approach: | They propose a framework that incorporates existing and new SLMs to enhance Transformers by incorporating syntactic biases. |
| Outcome: | The proposed framework improves on existing models and novel variants across language modeling, syntactic generalization, summarization, and inference efficiency. |
Copied to clipboard
| Challenge: | Traditional Video Quality Assessment (VQA) focuses on aesthetic fidelity and technical distortions. |
| Approach: | They propose a new task that evaluates whether a UGC item has positive community resonance based on multimodal attributes rather than visual quality alone. |
| Outcome: | The proposed task outperforms state-of-the-art baselines on CASTER-Bench . it provides interpretable and empathetic reasoning paths that align with real community feedback. |
Copied to clipboard
| Challenge: | Existing methods for document image translation rely on the vanilla encoder-decoder paradigm . a novel dynamic aggregation mechanism is designed to enhance the text semantics in query features toward translation. |
| Approach: | They propose a Query-Response DIT framework that reformulates the DIT task into a parallel response/translation process of multiple queries. |
| Outcome: | The proposed framework improves translation quality on four translation directions on three benchmarks. |
Copied to clipboard
| Challenge: | Existing MMEA datasets consider multi-modal data as attributes of textual entities, neglecting correlations between the multi-modal data. |
| Approach: | They propose a multi-modal entity alignment dataset that models multi-dimensional data as textual entities in the MMKG. |
| Outcome: | The proposed dataset can learn the structural information of entities by considering both intra-modal and cross-modal relations and infer the similarity of different types of entity pairs. |
Copied to clipboard
| Challenge: | Large language models can teach small language models to solve complex reasoning tasks by Chain-of-thought Distillation (CoTD) e.g., mathematical question answering. |
| Approach: | They propose a method that distills two student models to solve a multi-hop question . they use chain-of-thought distillation to generate step-by-step reasoning paths . |
| Outcome: | The proposed method surpasses existing methods on knowledge-intensive multi-hop questions. |
Copied to clipboard
| Challenge: | Recent knowledge graph (KG) augmented models have achieved notable success on commonsense reasoning tasks. |
| Approach: | They propose a KG-augmented model that contextualizes extracted and generated knowledge by reasoning over both within a single graph structure. |
| Outcome: | The proposed model outperforms existing models on four commonsense reasoning benchmarks and a user study on edge validness and helpfulness. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | Traditional approaches only target labeled attributes, ignoring biases from unlabeled ones. |
| Approach: | They propose a method that ensures protected group independence from all attributes and mitigates inpainting biases through data filtering. |
| Outcome: | The proposed approach achieves an average reduction of 46.1% in leakage-based bias metrics for multi-label classification and 74.8% for image captioning. |
Copied to clipboard
| Challenge: | Document Image Machine Translation (DIMT) faces generalization challenges due to limited training data and the complex interplay between visual and textual information. |
| Approach: | They propose a single-to-mix Modality alignment framework leveraging Multimodal Large Language Models (MLLMs) this framework aligns an imageonly encoder with multimodal representations of an MLLM pre-trained on large-scale document image datasets. |
| Outcome: | The proposed framework improves translation quality in cross-domain generalization and challenging document image scenarios. |
Copied to clipboard
| Challenge: | Existing self-reflection methods lack effective feedback information, limiting the translation performance of large language models (LLMs). |
| Approach: | They propose a framework that leverages the dual learning of translation tasks to provide effective feedback, thereby enhancing the models’ self-reflective abilities and improving translation performance. |
| Outcome: | The proposed framework improves the models’ self-reflective abilities and improves translation accuracy and eliminating ambiguities across translation tasks. |
Copied to clipboard
| Challenge: | Existing TIMT tasks focus on text-line-level images. |
| Approach: | They propose to extend the existing TIMT task and introduce a new framework to translate a source document image to markdown-formatted target translation. |
| Outcome: | The proposed task aims to translate a source document image with long context and complex layout structure to markdown-formatted target translation. |
Copied to clipboard
| Challenge: | LSLMs have impressive conversational generation abilities, but consistently fall short of traditional pipeline systems on semantic understanding benchmarks. |
| Approach: | They propose to analyze the performance gap between speech and text inputs through a systematic experiment . they find that representation similarity is strongly correlated with the modality gap . |
| Outcome: | The proposed models improve the accuracy of speech inputs and their semantic understanding benchmarks. |
Copied to clipboard
| Challenge: | Existing rerankers are mainly trained on well-edited texts, but stylistic features can be misled by reranked models. |
| Approach: | They propose a style-augmented multi-task framework that prioritizes effective knowledge over stylistic perturbations by using an LLM to derive passage-level supervision on whether a passage helps or harms answer correctness. |
| Outcome: | Extensive experiments show that SARK improves generation performance across multiple LLMs under mixed-style conditions. |
Copied to clipboard
| Challenge: | Existing studies treat named entity recognition as a sequential labeling problem. |
| Approach: | They propose a span selection framework for nested named entity recognition . they propose nesting entities with different input categories would be separately extracted . |
| Outcome: | The proposed framework outperforms competing models on four benchmark datasets. |
Copied to clipboard
| Challenge: | Existing work on temporal knowledge graphs ignores fact that real-life applications of TKGQA are complex in temporal granularity. |
| Approach: | They propose a large scale dataset for multi-granularity temporal question answering over knowledge graphs . they propose comparing MultiQA over MultiTQ to better reflect real-world challenges . |
| Outcome: | The proposed dataset is among the first of its kind and features multiple temporal granularities. |
Copied to clipboard
| Challenge: | a new method to enhance temporal knowledge reasoning in large language models addresses this challenge . Abstract Reasoning Induction (ARI) framework provides factual knowledge support to LLMs . |
| Approach: | They propose an abstract reasoning induction framework which divides temporal reasoning into two phases: Knowledge agnostic and Knowledge-based. |
| Outcome: | The proposed method achieves significant gains on two temporal QA datasets. |
Copied to clipboard
| Challenge: | Existing methods generate DocIDs based on textual content, which may result in weak semantic connections for similar documents due to variations in expression. |
| Approach: | They propose a new retrieval paradigm that generates unique document identifiers . they propose to use queries as a bridge to connect documents with varying relevance levels . |
| Outcome: | The proposed approach outperforms existing methods on multilingual e-commerce search datasets. |
Copied to clipboard
| Challenge: | Existing methods for detecting euphemisms are ineffective in impromptu euphorism detection . Existing approaches for e-mail detection are limited to word-level ephemismals . |
| Approach: | They propose a framework for impromptu euphemism detection that integrates context augmentation and multi-round iterative training to better predict the actual meaning of a masked token. |
| Outcome: | The proposed framework improves 76-fold over the previous state-of-the-art euphemism detector. |
Copied to clipboard
| Challenge: | Current Chain-of-thought Distillation methods hinder CoT reasoning performance . student models are separately distilled from specific reasoning tasks . parameter update of student models severely harms CoT ability on unseen reasoning tasks. |
| Approach: | They propose a method which distills Chain-of-thought reasoning ability of large language models to much smaller student models. |
| Outcome: | The proposed method improves the reasoning ability of large language models on 14 datasets. |
Copied to clipboard
| Challenge: | Existing work evaluating commonsense reasoning focuses on making inferences about common, everyday situations. |
| Approach: | They propose to use an English language corpus to investigate commonsense reasoning . they characterize performance differences between human explainers and best-performing large language models . |
| Outcome: | The proposed method reduces the loss rate of human-written explanations on commonsense reasoning compared with the vanilla supervised fine-tuning approach . |
Copied to clipboard
| Challenge: | Existing question answering systems use a retriever-reader framework to answer multi-hop questions . existing models lack retrieval, selector, and reasoner capabilities . |
| Approach: | They propose a three-stage text tableQA framework which comprises of retriever, selector, and reasoner. |
| Outcome: | The proposed framework outperforms baseline methods in the few-shot setting and ranks first on the HybridQA leaderboard. |
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) has evolved from flat to overlapped and discontinuous . NER is a text recognition task that recognizes mentions that represent entities in text . |
| Approach: | They propose a two-stage span-based framework to solve a unified NER task using two stages . they extract entity spans, classify over all entity span pairs and combine them to train two stages. |
| Outcome: | The proposed framework beats all the current competitive baselines on eight benchmark datasets, obtaining the best performance of unified NER. |
Copied to clipboard
| Challenge: | Multi-tenant Model-as-a-Service (MaaS) workloads exhibit non-stationarity across multiple time scales . existing request schedulers often rely on a fixed policy that remains unchanged at runtime . |
| Approach: | They propose a hierarchical multi-agent scheduler that operates in a layered closed loop . they propose to maintain 1.2–3.0 higher Goodput than SGLang and vLLM . |
| Outcome: | Experiments show that H-MAS achieves 1.2–3.0 higher Goodput than SGLang and vLLM . it maintains more stable QoS under diverse request lengths and heterogeneous SLO targets . |
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have shown strong performance in document image tasks, especially Optical Character Recognition (OCR). However, they struggle with Document Image Machine Translation (DIMT), which requires handling both cross-modal and cross-lingual challenges. |
| Approach: | They propose a novel fine-tuning paradigm that allows the model to generate OCR text before producing translation text, which allows it to leverage its strong monolingual OCR ability while learning to translate text across languages. |
| Outcome: | The proposed model can leverage its strong monolingual OCR ability while learning to translate text across languages. |
Copied to clipboard
| Challenge: | Existing work on event extraction (EE) is pipelined or uses a joint structure but does not utilize information interactions among event triggers, event arguments, and argument roles. |
| Approach: | They propose to exploit role information of arguments in an event and devise a Hierarchical Policy Network to perform joint EE. |
| Outcome: | The proposed system outperforms existing methods and is more powerful for sentences with multiple events. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have high computational costs and privacy concerns due to their high computational expenses and data privacy. |
| Approach: | They propose a method that empowers SLMs to internalize symbolic knowledge and few-shot examples gradually through a progressive fine-tuning process. |
| Outcome: | The proposed approach outperforms state-of-the-art baselines by over 5% while reducing inference costs by up to 4 across a wide range of SLMs in both in-domain (ID) and out-of domain (OOD) tasks. |
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |
Copied to clipboard
| Challenge: | Expressive zero-shot voice conversion (VC) aims to modify source timbre to match unseen speaker . existing zero- shot VC systems struggle to reproduce paralinguistic information in highly expressive speech . |
| Approach: | They propose a framework for expressive zero-shot voice conversion that uses hybrid content encoding and memory-augmented context-aware timbre modeling. |
| Outcome: | The proposed framework surpasses state-of-the-art VC systems in speech naturalness, speaker similarity, and speaker similarness. |
Copied to clipboard
| Challenge: | Recent advances in vision-language learning have significantly advanced Human-Computer Interactions (HCI). |
| Approach: | They propose a method to align the semantic spaces between speech and text by incorporating two modules to align semantic spaces. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on AVOS benchmarks. |
Copied to clipboard
| Challenge: | Back-translation methods rely on large-scale parallel corpora to enhance performance, but ignore the semantic quality of monolingual data. |
| Approach: | They propose a method which prioritizes sentences with higher semantic uncertainty as training samples by computationally evaluating the complexity of unannotated monolingual data. |
| Outcome: | The proposed method improves translation accuracy and fluency by +1.7 on all three translation tasks. |
Copied to clipboard
| Challenge: | Document Image Translation (DIT) aims to translate documents in images from one language to another. |
| Approach: | They propose a novel end-to-end network called Zoom-out DIT to improve document translation by combining word positioning, sentence recognition and document organization. |
| Outcome: | The proposed network improves word positioning, sentence recognition and document organization, and improves translation quality. |
Copied to clipboard
| Challenge: | Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech. |
| Approach: | They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness. |
| Outcome: | The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines . |
Copied to clipboard
| Challenge: | Semantic parsing (SP) maps a natural language utterance into a formal language . standard Seq2Seq models ignore underlying grammars and may give ill-formed results. |
| Approach: | They propose an end-to-end model for semantic parsing that transduces a natural language sentence to the formal semantic representation. |
| Outcome: | The proposed model outperforms the state-of-the-art models and does not need expertise like predefined grammar or sketches in the meantime. |
Copied to clipboard
| Challenge: | Logic-Induced-Knowledge-Search (LINK) is a framework for generating factually-correct yet long-tail inferential knowledge. |
| Approach: | They introduce a framework to obtain factually-correct yet long-tail inferential statements using variable-wise prompting grounded on symbolic rules. |
| Outcome: | The proposed framework is able to obtain factually-correct yet long-tail inferential statements while ensuring factual correctness. |
Copied to clipboard
| Challenge: | Existing repository-level code completion benchmarks focus on a limited number of languages . existing benchmarks report overall average scores of different languages ignoring fine-grained abilities . |
| Approach: | They propose to use repository-level code completion benchmarks to evaluate general code intelligence abilities across languages for existing code Large Language Models. |
| Outcome: | The proposed benchmarks improve the code completion abilities of existing LLMs by using two types of annotations on the parsed syntax tree. |
Copied to clipboard
| Challenge: | Currently, the performance of transformer-based model editing methods is limited to statements about encyclopedic knowledge with a single correct answer. |
| Approach: | They propose to improve MEMIT's model editing algorithm by varying edit tokens and improving the layer selection strategy to improve commonsense knowledge. |
| Outcome: | The MEMIT editing algorithm outperforms baseline models on PEP3k and 20Q datasets while fine-tuning baselines shows significant trade-offs. |
Copied to clipboard
| Challenge: | Existing large language models fall short of translating culturally significant content . existing models fall behind in achieving such translations, authors say . |
| Approach: | They propose a suitable benchmark for translating classical Chinese poetry into English . they propose RAT, a retrieval-augmented machine translation method that enhances the translation process . |
| Outcome: | The proposed method improves translation quality in terms of adequate, fluent, and elegant translations. |