Papers by Zheng Shen
Copied to clipboard
| Challenge: | Existing models cannot fully recognize the specific expressions given by users due to the informality and diversity of natural language expressions. |
| Approach: | They propose a Heterogeneous User History graph convolution network which utilizes the user’s historical answers grouped by DA labels as additional clues to recognize the DA label of utterances. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on two benchmark datasets and shows that it integrates user’s historical answers. |
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness often depends on costly commercial APIs or cloud services. |
| Approach: | They propose a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism. |
| Outcome: | The proposed approach reduces redundant outputs and response times while reducing computational costs by over 50% and cascade latency by over 80%. |
Copied to clipboard
| Challenge: | Existing studies have focused on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, but the interpretabilty of these metrics has been neglected. |
| Approach: | They propose a reference-based metric that describes four aspects of GEC systems: hit-correction, wrong-corrections, under-correcties, and over-corrects. |
| Outcome: | The proposed metric reveals critical qualities and locates drawbacks of GEC systems. |
Copied to clipboard
| Challenge: | Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models. |
| Approach: | They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains . |
| Outcome: | The proposed scheme yields 5.51% OOD gain over positive-only training. |
Copied to clipboard
| Challenge: | Existing studies focus on misspelled characters, ignoring faked characters which are more common and difficult to correct. |
| Approach: | They propose to use Chinese character checking to identify and correct wrong characters in texts by human annotation. |
| Outcome: | The proposed dataset is the first real-world visual and the largest human-crafted dataset for the Chinese character checking scenario. |
Copied to clipboard
| Challenge: | Existing evaluation metrics for travel planning rely on unrealistic simulated data . fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
| Approach: | They propose a benchmark for personalized travel planning in real-world scenarios . they identify several critical challenges in travel planning including feasibility and rationality . |
| Outcome: | The proposed benchmarks show that fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
Copied to clipboard
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
Copied to clipboard
| Challenge: | GUI agents have demonstrated remarkable progress in automating complex user interface interactions . training such agents for long-horizon tasks remains challenging due to limited rewards and prohibitive costs. |
| Approach: | They propose a method that leverages expert trajectories as environment experiences for on-policy multi-turn training. |
| Outcome: | The proposed method achieves significant gains over the base model with 1K public trajectories as RL experiences . it achieves competitive performance against strong baselines such as UI-TARS-7B and GPT-4o . |
Copied to clipboard
| Challenge: | Xu et al., 2015) proposed a noise reduction mechanism to disentangle semantics of words . hard and soft attention mechanisms are used to reduce noise in NLP tasks . |
| Approach: | They propose a prism module to disentangle semantic aspects of words and reduce noise . they propose combining prism modules with downstream models to improve model performance . |
| Outcome: | The proposed method significantly improves the performance of baselines on named entity recognition (NER) tasks. |
Copied to clipboard
| Challenge: | Existing approaches to align large language models with human preferences lack flexibility . static alignment preferences lack the ability to correct misaligned behaviors as they emerge . |
| Approach: | They propose a framework that enables dynamic and continuous alignment of large language models with human preferences. |
| Outcome: | The proposed framework improves safety and accuracy of a 7B model with human annotations. |
Copied to clipboard
| Challenge: | Evaluating the performance of Grammatical Error Correction systems is a challenging task due to its subjectivity. |
| Approach: | They propose a method to evaluate GEC systems in multi-reference evaluation setting . they use consistent edit boundaries to eliminate bias caused by inconsistent edit boundaries . |
| Outcome: | The proposed evaluation metric eliminates bias caused by inconsistent edit boundaries on six English reference sets. |
Copied to clipboard
| Challenge: | Existing approaches to machine unlearning treat all tokens indiscriminately and enforce uncertainty over the entire vocabulary. |
| Approach: | They propose a framework that targets the prefix in a response and minimizes uncertainty in the critical subspace. |
| Outcome: | The proposed framework achieves superior forgetting efficacy and utility preservation compared to baselines. |
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models are inefficient and redundant . a light-PEFT framework can be used to prune redundant parameters during training . |
| Approach: | They propose a parameter-efficient fine-tuning framework that freezes most parameters of the foundation model and finetuns only a small number of parameters. |
| Outcome: | The proposed framework achieves training and inference speedup, reduces memory usage, and maintains comparable performance and plug-and-play feature of PEFT. |
Copied to clipboard
| Challenge: | Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots. |
| Approach: | They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios. |
| Outcome: | The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning . |
Copied to clipboard
| Challenge: | specialized quantization framework for Mixture of Experts architectures is inadequate for model compression. |
| Approach: | They propose a specialized quantization framework for Mixture of Experts architectures . they find that expert networks exhibit distinctive channel-wise outlier distributions ." |
| Outcome: | The proposed framework improves on the Mixtral-8x7b-v0.1 architecture while maintaining minimal computational overhead. |
Copied to clipboard
| Challenge: | Existing work on generating empathetic responses by utilizing the speaker's emotion has not been successful. |
| Approach: | They propose an approach which incorporates an adaptive module for commonsense knowledge selection to ensure consistency between the generated empathetic responses and the speaker’s situation. |
| Outcome: | The proposed approach outperforms baseline models in both automatic and human evaluations, exhibiting the generation of more coherent and empathetic responses. |
Copied to clipboard
| Challenge: | Existing methods to improve the learning of data-scarce target domains have negative transfer due to the data distributions between source and target domain. |
| Approach: | They propose a method that uses a reinforced selector to select helpful data for transfer learning and a Wasserstein-based discriminator to maximize the distance between the selected data and target data. |
| Outcome: | The proposed method performs better on three real-world text mining tasks. |
Copied to clipboard
| Challenge: | Structured pruning is a feasible solution for end-side LLM deployment . however, achieving a high compression ratio for scaled-up LLMs remains a challenge . |
| Approach: | They propose a task-agnostic structured pruning approach coupled with a compact Transformer architecture to prune LLMs into an intra-module low-rank architecture. |
| Outcome: | The proposed approach reduces transitional activations inside multi-head attention (MHA) and multi-layer perceptron (MLP) modules while preserving inter-module activations sensitive to perturbations. |
Copied to clipboard
| Challenge: | Existing models that induce grammar structures from data focus on constituency or dependency structures alone. |
| Approach: | They propose a model that can induce dependency and constituency structure at the same time. |
| Outcome: | The proposed model can induce both constituency and dependency structures at the same time. |
Copied to clipboard
| Challenge: | Existing methods for long-form outline generation have low knowledge density and lack detail . retrieval-augmented approaches struggle to maintain logical coherence across retrieved information . |
| Approach: | They propose a system that mimics human writers' refinement process by mimicking outlines through imitation and critical self-refinement. |
| Outcome: | The proposed system improves on the FreshWiki and WikiOutline datasets and establishes a coherent planning framework and structured knowledge base. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on online one-on-one chatting or human-AI interactions, neglecting real-world scenarios. |
| Approach: | They propose a framework to curate a lifelog benchmark that combines two subsets of audio data to address temporal leakage in offline settings. |
| Outcome: | The proposed framework outperforms existing benchmarks on live chats and AI interactions. |
Copied to clipboard
| Challenge: | Existing tools for ambiguous and incomplete queries are limited by manual construction and lack of error correction mechanisms during multi-turn clarification. |
| Approach: | They propose a framework that exploits the mapping between queries and their tool invocation solutions by removing key parameters from queries while retaining them as ground truth. |
| Outcome: | The proposed framework outperforms existing methods while maintaining high accuracy in tool invocation. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used for text understanding and generation . existing methods that assume single-turn interactions break down in multi-turn settings . |
| Approach: | They propose a differentially private prompt perturbation framework for multi-turn LLM inference . DP3 constructs a perturbation mapping table to reuse perturbations for recurring tokens . |
| Outcome: | The proposed framework reduces privacy costs and degrades cross-turn semantic coherence . it also provides a context-aware utility function to maintain semantic consistency across turns . |
Copied to clipboard
| Challenge: | Chinese Spelling Correction (CSC) is a model that detects and corrects spelling errors in given sentences. |
| Approach: | They propose a model-agnostic model with an evolving teacher model and dynamic distillation weights for knowledge transfer in each domain rather than focusing solely on new domain knowledge. |
| Outcome: | The proposed model-agnostic framework is based on an evolving teacher model and dynamic distillation weights for knowledge transfer in each domain, rather than focusing solely on new domain knowledge. |
Copied to clipboard
| Challenge: | Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning. |
| Approach: | They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
| Outcome: | The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
Copied to clipboard
| Challenge: | Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands. |
| Approach: | They propose an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content. |
| Outcome: | The proposed framework reduces the KV cache size to 4.74% of the original and achieves a 6% speedup. |
Copied to clipboard
| Challenge: | Existing work mitigates memory overhead by offloading or compressing the Key-Value cache. |
| Approach: | They propose a method that integrates quantization and offloading into a generative large language model by using a hybrid compression method. |
| Outcome: | The proposed method outperforms the state-of-the-art in long-context evaluations. |
Copied to clipboard
| Challenge: | Existing benchmark datasets focus on short to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. |
| Approach: | X-LeBench is a benchmark dataset designed to evaluate long egocentric video recordings . it uses a life-logging pipeline to produce realistic, coherent daily plans . |
| Outcome: | X-LeBench is a new benchmark dataset designed to evaluate long-form egocentric video understanding . the approach produces realistic, coherent daily plans aligned with real-world video data . |
Copied to clipboard
| Challenge: | X-ray and CT are the gold standard for COVID-19 diagnosis and treatment . however, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. |
| Approach: | They propose to use X-ray and CT to generate medical reports automatically . they evaluate DeltaNet on a COVID-19 dataset, where it outperforms state-of-the-art approaches . |
| Outcome: | The proposed system outperforms state-of-the-art methods on a COVID-19 dataset. |
Copied to clipboard
| Challenge: | Existing methods for deep question generation focus on enhancing document representations, but little attention is paid to the answer information. |
| Approach: | They propose a deep question generation model that makes better use of the target answer as a guidance to facilitate question generation. |
| Outcome: | The proposed model outperforms state-of-the-art models in automatic and human evaluations on the hotpotQA dataset. |
Copied to clipboard
| Challenge: | Existing methods for token reduction for SSMs lead to performance drops . a recent study shows that Mamba-2 improves the accuracy of the model by 5.7% to 13.1% . |
| Approach: | They propose a token reduction method that integrates token importance and similarity into SSMs and takes advantage of pruning and merging. |
| Outcome: | The proposed method improves accuracy by 5.7% to 13.1% on six benchmarks with Mamba-2 compared to existing methods while reducing computational demands and memory requirements. |
Copied to clipboard
| Challenge: | Currently, open-domain chatbots are far from satisfactory. |
| Approach: | They propose a unified, readily scalable neural approach which reconciles all subtasks like intent prediction and knowledge retrieval. |
| Outcome: | The proposed approach outperforms commercial systems replying on complex rules on static and interactive tests and shows that the results are remarkably good. |
Copied to clipboard
| Challenge: | a new study examines the association capabilities of large language models . as models scale up, their ability to associate entities/information intensifies . however, there is a performance gap when associating commonsense knowledge versus PII, with the latter showing lower accuracy. |
| Approach: | They examine the association capabilities of large language models and identify factors that influence their proficiency in associating information. |
| Outcome: | The proposed models show a performance gap when associating commonsense knowledge versus PII, with the latter showing lower accuracy. |
Copied to clipboard
| Challenge: | In multivariate long-term time series forecasting, it is widely believed that the effectiveness of self-attention arises from its attention matrix. |
| Approach: | They propose a multi-branch MLP that isolates the ‘multi-brain mapping with element-wise operation’ structure from the Transformer and shows that it achieves competitive performance. |
| Outcome: | The proposed model outperforms three classic and three latest Transformer models and shows that it achieves competitive performance. |
Copied to clipboard
| Challenge: | Existing studies have examined how large language models’ social reasoning capabilities evolve during model size scaling or reasoning tokens scaling. |
| Approach: | They propose to optimize evaluation of Large Language Models from both data and model perspectives and to analyze their reasoning trajectories to identify notable cognitive "Aha Moments" |
| Outcome: | The proposed model outperforms the o1-preview model by 19.0 points in the evaluation of large language models. |
Copied to clipboard
| Challenge: | Existing statute retrieval benchmarks emphasize formal and professional queries from sources like bar exams and legal case documents . existing retrieval approaches that lack domain-specific knowledge may struggle to capture the meanings of specialized terms accurately. |
| Approach: | They propose a dataset that captures the complexity and diversity of real queries from the general public. |
| Outcome: | The proposed dataset captures the complexity and diversity of real queries from the general public. |
Copied to clipboard
| Challenge: | Existing methods for multimodal sarcasm detection neglect high-order relationships and underestimate high-frequency messages. |
| Approach: | They propose a Dual Graph-based Learning Framework to capture inter-modal inconsistencies . they propose combining a hypergraph and a vanilla graph to achieve enhanced propagation . |
| Outcome: | The proposed model outperforms existing state-of-the-art methods on two benchmark datasets. |
Copied to clipboard
| Challenge: | Existing backdoor defense paradigms focus on detecting and removing poisoned samples at pre-training or inference time. |
| Approach: | They propose a new approach where the backdoor attack is directly reversed by incorporating maximum entropy loss into training to neutralize the minimal cross-entropiness loss fine-tuning on poisoned data. |
| Outcome: | The proposed model significantly lowers the attack success rate on classification tasks and reduces the risk of backdoor attacks on clean data. |
Copied to clipboard
| Challenge: | Existing methods to generate empathetic responses are monotonous and generic, resulting in shallow empathy and few connections to the context. |
| Approach: | They propose to use explicit control to guide the empathy expression and a framework DiffusEmp to unify the utilization of dialogue context and attribute-oriented control signals. |
| Outcome: | The proposed framework outperforms baselines on EmpatheticDialogue in terms of controllability, informativeness, diversity, and diversity without the loss of context-relatedness. |
Copied to clipboard
| Challenge: | Existing LLM leaderboards often reference scores reported in other papers without consistent settings and prompts, which may encourage cherry-picking favored settings and for better results. |
| Approach: | They propose an open-source and reproducible LLM evaluation suite built on top of OpenAI Evals that systematically evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
| Outcome: | The evaluation suite is built on top of OpenAI Evals and evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
Copied to clipboard
| Challenge: | Existing studies focus on modeling emotion influences with utterance-level features, with little attention paid on phrase-level semantic connection between utterrances. |
| Approach: | They propose a two-stage Summarization and Aggregation Graph Inference Network which integrates inference for topic-related emotional phrases and local dependency reasoning over neighbouring utterances in a global-to-local fashion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on three CER benchmark datasets. |
Copied to clipboard
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
Copied to clipboard
| Challenge: | Existing approaches for optimizing human annotation efforts are limited . et al., 2015) suggest that densely annotated image captions improve vision-language alignment . |
| Approach: | They propose an AI-in-the-loop methodology to maximize the number of annotated samples and improve their comprehensiveness under fixed budget constraints. |
| Outcome: | The proposed method improves annotation speed and retrieval performance over the parallel method. |
Copied to clipboard
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values. |
| Approach: | They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values . |
| Outcome: | The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets. |
Copied to clipboard
| Challenge: | Large vision-language models have been widely used but stereotypical biases are unexplored. |
| Approach: | They propose a framework to SCAN stereotypical bias within large vision-language models . they examine stereotype biases with respect to gender and race in three scenarios . |
| Outcome: | The proposed framework can reduce stereotypical biases in large vision-language models . the currently popular models show significant stereotype biase . |
Copied to clipboard
| Challenge: | Existing approaches ignore relationships between medical items and statuses in the multi-turn doctor-patient dialogue. |
| Approach: | They propose a task to extract structured medical information from free text dialogues . they propose 'Dialogue Medical Information Extraction' to model relationships between items . |
| Outcome: | The proposed model outperforms previous models and achieves state-of-the-art performance on the public benchmark data set. |
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |
Copied to clipboard
| Challenge: | Tool-integrated reasoning (TIR) enables LLM agents to solve tasks through planning, tool use, and iterative revision, but outcome-only reinforcement learning suffers from sparse, delayed rewards and weak step-level credit assignment. |
| Approach: | They propose a tool-integrated reasoning approach that localizes the first irrecoverable step and leverages it for fine-grained credit assignment. |
| Outcome: | The proposed algorithm outperforms strong Agentic RL benchmarks in math, science QA, and code execution with additional gains in Pass@K and Major@K scaling, rollout ranking quality, and tool-call efficiency. |
Copied to clipboard
| Challenge: | Existing methods for Chinese relation extraction suffer from segmentation errors and ambiguity of polysemy. |
| Approach: | They propose a multi-grained lattice framework for Chinese relation extraction . they incorporate word-level information into character sequence inputs to avoid segmentation errors . |
| Outcome: | The proposed model outperforms existing models on three real-world datasets in distinct domains. |
Copied to clipboard
| Challenge: | Existing work on 3D radiograph report generation focuses on 2D images, but 3D medical images provide more comprehensive diagnostic information. |
| Approach: | They propose a comprehensive training recipe for building high-performing VLMs for 3DRRG using a publicly available 3D CT-report dataset. |
| Outcome: | The proposed model achieves superior performance across different model sizes and input 3D medical image resolutions. |
Copied to clipboard
| Challenge: | State-of-the-art contrastive learning models like CLIP and ALIGN are less interpretable and suffer from inferior accuracy than dense representations. |
| Approach: | They extend CLIP and ALIGN models to build a sparse semantic representation that is interpretable and easy to integrate with existing retrieval systems. |
| Outcome: | The proposed model outperforms CLIP and ALIGN models on image and text retrieval tasks with a 4.9% and +4.3% improvement on COCO-5k textimage and imagetext retrieval respectively. |
Copied to clipboard
| Challenge: | Existing approaches to theorem proving in large language models rely on value functions and/or Monte Carlo Tree Search (MCTS), but the potential of simpler methods like Best-First Tree Search remains underexplored. |
| Approach: | They propose a scalable expert iteration framework that implements strategic data filtering at each expert iteration round, excluding problems solvable via beam search node expansion to focus on harder cases. |
| Outcome: | The proposed framework achieves a state-of-the-art score of 72.95 on the MiniF2F test set and challenges the perceived necessity of complex tree search methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors pose significant risks. |
| Approach: | They propose a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. |
| Outcome: | The proposed framework offers a principled and interpretable framework for safe and controllable LLM behavior serving as a foundation for future research. |
Copied to clipboard
| Challenge: | Syntactic movement is a core concept in generative linguistics to account for word-order variation and long-distance dependencies. |
| Approach: | They annotated every sentence in the audiobook The Little Prince using X-bar style tree annotations. |
| Outcome: | The proposed model shows that deep structure significantly predicts neural responses in English but not in Chinese. |
Copied to clipboard
| Challenge: | Chinese Grammatical Error Correction (CGEC) is a challenging NLP task and a common application in human daily life. |
| Approach: | They propose a linguistic rules-based approach to construct large-scale CGEC training corpora with automatically generated grammatical errors. |
| Outcome: | The proposed method improves performance of existing CGEC models and the benchmark is excellent resource for further development. |
Copied to clipboard
| Challenge: | a recent study focuses on generating impartial and interpretable judicial judgments based on established criminal fact. |
| Approach: | They propose a law reasoning schema enriched with hierarchical factum probandum, evidence, and implicit experience that enables public scrutiny and preventing bias. |
| Outcome: | The proposed schema enables public scrutiny and prevents bias in the "Intelligent Court" it employs a suite of legal analysis tools to address the challenge task. |
Copied to clipboard
| Challenge: | Existing methods for reconstruction of large language models overlook diversity among experts, leading to potential redundancy. |
| Approach: | They propose a pruning-based expert reconstruction method that prunes a specific LLM and retrains it on routers, experts and normalization modules. |
| Outcome: | The proposed method outperforms pruning and MoE reconstruction methods on Llama-style models with open-source training corpora. |
Copied to clipboard
| Challenge: | Existing statically compressed pre-trained language models lack spatial and temporal efficiency due to their large size and wide width. |
| Approach: | They propose a spatially and temporally efficient model which retains the major capacity of PLMs. |
| Outcome: | The proposed model retains the major capacity of pre-trained language models at high compression and acceleration rate with 1/8 parameters and 1/19 FLOPs of BERT. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful at question-answering but prone to hallucinations due to limited domain-specific or up-to-date knowledge. |
| Approach: | They propose a framework for IDentifying RAG properties in LLM services that integrates LLMs with retrieval systems and adds an external retriever and knowledge database to mitigate hallucinations. |
| Outcome: | The proposed framework detects RAG-enhanced LLMs with 99.97% accuracy with partial or no optional knowledge and nearly 100% when the LLM and database are known. |
Copied to clipboard
| Challenge: | Existing fact-checking methods focus on verification of individual facts, overlooking logical dependencies . a recent study shows that text containing logical errors may still be misjudged as factual . |
| Approach: | They propose a content–logic coupled factuality evaluation paradigm that conceptualizes factual dimension along two complementary dimensions: content factualism and logic factuity. |
| Outcome: | The proposed paradigm bridges the gap between factual verification and content factuality . it incorporates the logical dimension and a logic-aware metric to expose and penalize logical fallacies. |
Copied to clipboard
| Challenge: | This survey provides **the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models** . integrating large language model with mathematical reasoning tasks is becoming significant as AI advances . |
| Approach: | They review over 200 studies published since 2021 and examine the state-of-the-art developments in Math-LLMs . they identify five major challenges hindering the realization of AGI in this domain . |
| Outcome: | The authors examine the state-of-the-art developments in Math-LLMs with a focus on multimodal settings. |
Copied to clipboard
| Challenge: | Named Entity Recognition and Entity Linking are challenging for voice assistants . utterances are relatively short, so there is not much context to help disambiguate . |
| Approach: | They propose a Named Entity Understanding system that combines NER and EL in a joint reranking module. |
| Outcome: | The proposed framework improves NER accuracy by up to 3.13% and EL accuracy by 3.6% in F1 score . it also leads to better accuracies in other natural language understanding tasks . |
Copied to clipboard
| Challenge: | Recent efforts to integrate large language models into English education lack adaptability to language learning. |
| Approach: | They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks . |
| Outcome: | The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education. |
Copied to clipboard
| Challenge: | Manually "engineering" prompts for large language models can be laborious and time-intensive. |
| Approach: | They propose a new metric to quantify the expected utility of a language prompt. |
| Outcome: | The proposed metric outperforms previous prompt selection metrics with 10% increase in Pearson correlation across 6 classification benchmarks and the prompt selected by the proposed meter gains 5% higher accuracy than previous metrics. |
Copied to clipboard
| Challenge: | Existing benchmarks fail to evaluate egocentric clinical intent understanding of medical multimodal large language models. |
| Approach: | They propose a benchmark leveraging clinician gaze as a Cognitive Cursor to assess intent understanding across surgery, emergency simulation and diagnostic interpretation. |
| Outcome: | The proposed benchmark addresses challenges of visual homogeneity of anatomical structures, strict temporal-causal dependencies in clinical workflows, and implicit adherence to safety protocols. |
Copied to clipboard
| Challenge: | Existing methods for generative semantic hashing assume a factorized posterior distribution, enforcing independence among the bits of hash codes. |
| Approach: | They propose to use a Boltzmann machine distribution as the variational posterior to introduce correlations among the bits of hash codes. |
| Outcome: | The proposed method can achieve significant performance gains by combining two hash codes. |
Copied to clipboard
| Challenge: | Recent VSE models combine simple pooling methods with hard triplet loss to improve performance. |
| Approach: | They propose an adaptive pooling strategy that allows the model to learn how to aggregate features through a combination of simple pooling methods. |
| Outcome: | The proposed strategy outperforms current state-of-the-art systems on image-to-text and text-toimage retrieval. |
Copied to clipboard
| Challenge: | Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. |
| Approach: | They propose a model that enables automatic and scalable data generation without manual annotations to overcome the data scarcity. |
| Outcome: | The proposed model overcomes the data scarcity and lacks manual annotations. |
Copied to clipboard
| Challenge: | Experimental results prove that language models can learn from human feedback better, irrespective of sequence length . emergence of length bias often induces the model to favor longer outputs . |
| Approach: | They propose to separate reward modeling from the influence of sequence length by using the Product-of-Experts technique. |
| Outcome: | The proposed approach shows that language models perform better regardless of sequence length . the main expert is focused on understanding human intents, while the biased expert targets the identification and capture of length bias. |
Copied to clipboard
| Challenge: | Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations. |
| Approach: | They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference. |
| Outcome: | The proposed method significantly mitigates object hallucinations across various model architectures. |
Copied to clipboard
| Challenge: | Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored. |
| Approach: | They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes. |
| Outcome: | The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks. |
Copied to clipboard
| Challenge: | a key intent behind many emails is to get a reply from the recipient. |
| Approach: | They propose to model the intents, expectations, and responsiveness in email exchanges by using a dataset containing 1800 emails annotated with nuanced types of intents and expectations. |
| Outcome: | The proposed model is based on 1800 emails annotated with nuanced types of intents and expectations . it shows that social status, argumentation, and strength of social connection influence email response rates . |
Copied to clipboard
| Challenge: | Recent work has shown excellent performance on text generation tasks by combining reinforcement learning (RL) and generative models. |
| Approach: | They propose a model-based imitation-learning approach to improve text generation performance by focusing on a long horizon. |
| Outcome: | The proposed model improves on a number of text-generation tasks and provides intermediate rewards for generator optimization. |
Copied to clipboard
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
Copied to clipboard
| Challenge: | OpenCodeInterpreter-33B provides a high level of performance for code generation, executing, and iterative refinement. |
| Approach: | They propose a family of open-source code systems for generating, executing, and iteratively refining code. |
| Outcome: | The OpenCodeInterpreter-33B performs well on humanEval, MBPP, and EvalPlus benchmarks. |