Papers by Rui Hu

57 papers
F2TEval: Human-Aligned Multi-Dimensional Evaluation for Figure-to-Text Task (2025.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation methods for Figure-to-Text tasks are limited due to the inherent ambiguity and semantic compression of figures, the generated texts suffer from factual inaccuracies, incomplete coverage, and weak logical reasoning.
Approach: They propose a five-dimensional reference-free evaluation method aligned with expert criteria to support fine-grained evaluation.
Outcome: The proposed method outperforms Gemini-2.0 and Claude-3.5 with only 0.9B parameters.
XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for embedding binary messages into LLM-generated text suffer from key limitations, such as a poor trade-off between text quality and decoding accuracy.
Approach: They propose a method for embedding binary messages into Large Language Model (LLM)-generated text that uses a limited number of tokens to decode and recover the encoded message.
Outcome: The proposed method significantly outperforms existing methods in multiple downstream tasks and will be made publicly available upon acceptance.
TellWhisper: Tell Whisper Who Speaks When (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches decouple temporal modeling and speaker modeling when addressing 'when' and 'who' . a new framework that couples temporal structure with speaker dynamics is proposed to address these limitations .
Approach: They propose a framework that couples temporal and speaker identity within the speech encoder . they propose TS-RoPE, a time-speaker rotary positional encoding that partitions Query/Key channels into temporal, speaker subspaces and applies region-specific rotations to align "when" and "who" cues in selfattention.
Outcome: The proposed framework couples temporal structure with speaker dynamics in speech encoder . it uses frame-level speaker activity to estimate speaker-activity estimates .
Neural Topic Modeling with Cycle-Consistent Adversarial Training (2020.emnlp-main)

Copied to clipboard

Challenge: Recent advances on deep generative models have attracted significant interest in neural topic modeling.
Approach: They propose an adversarial-neural topic model which uses Dirichlet prior to capture the semantic patterns in latent topics.
Outcome: The proposed models outperform competing models on unsupervised/supervised topic modeling and text classification.
Uncovering Scaling Laws for Large Language Models via Inverse Problems (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have achieved remarkable success across diverse domains.
Approach: inverse problems can efficiently uncover scaling laws that guide the building of LLMs, authors argue . authors propose brute-force approaches to improve LLM training costs due to high costs .
Outcome: This paper advocates that inverse problems can efficiently uncover scaling laws that guide the building of LLMs to achieve the desirable performance with significantly better cost-effectiveness.
One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues (P19-1)

Copied to clipboard

Challenge: Currently, retrieval-based dialogues are performed in shallow ways . a recent study investigated the problem of context-response matching in open-domain .
Approach: They propose a model that lets utterance-response interaction go deep by stacking interaction blocks.
Outcome: The proposed model outperforms state-of-the-art methods on three benchmark data sets.
Who Is Speaking to Whom? Learning to Identify Utterance Addressee in Multi-Party Conversations (D19-1)

Copied to clipboard

Challenge: In multi-party conversations, addressee information is not always explicit . researchers have spent great efforts to understand conversations between two participants, which is known as multi-part conversation.
Approach: They propose a who-to-whom model which models users and utterances in a conversation session jointly in an interactive way.
Outcome: The proposed model outperforms baseline models on the Ubuntu Multi-Party Conversation Corpus and shows consistent improvements.
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors (2026.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) can call tools effectively, but they remain brittle in multi-turn execution.
Approach: They propose a framework that converts execution errors into on-policy corrective supervision within the RL training loop.
Outcome: The proposed framework improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% on BFCL v4 Multi-Turn.
Towards Robust Few-Shot Relation Classification: Incorporating Relation Description with Agreement (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to recognize relational relationships with a few support samples are limited for unlimited queries.
Approach: They propose a simple but effective framework that uses relation descriptions as external knowledge to enhance the model’s comprehension of the relation semantics.
Outcome: The proposed framework outperforms strong baselines while being robust against various NOTA rates.
QiMeng-PRepair: Precise Code Repair via Edit-Aware Reward Optimization (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to program repair are based on correctness alone.
Approach: They propose a framework that mitigates over-editing and improves repair accuracy by generating buggy programs and re-edits.
Outcome: The proposed framework improves repair precision by 31.4% under fix1@1, a metric that considers repair correctness and extent, and significantly increases decoding throughput when combined with speculative editing.
Data Pollination: An Emergent Ecological Process Driving AI Population Evolution (2026.acl-long)

Copied to clipboard

Challenge: evidence from deployed systems suggests that language models interact through a shared data ecosystem.
Approach: They propose to use data pollination to investigate stability dynamics under synthetic data training to investigate model collapse.
Outcome: The proposed model can mitigate model collapse observed in recursive training, and improve performance across benchmarks.
Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach: They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome: The proposed method achieves the strongest alignment-forging Pareto front among competing methods.
CLEAN–EVAL: Clean Evaluation on Contaminated Large Language Models (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods to evaluate large language models are prone to data contamination.
Approach: They propose a method which parses contaminated data and back-translates it into a candidate set.
Outcome: The proposed method reduces data contamination and evaluates the LLMs more cleanly.
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)

Copied to clipboard

Challenge: Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed .
Approach: They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities.
Outcome: The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech .
Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogues (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing knowledge-grounded dialogue systems focus on a single knowledge source or ignore the dependency between multiple knowledge sources.
Approach: They propose a framework that integrates multiple knowledge sources and dependencies between them.
Outcome: The proposed framework can produce persona-consistent and knowledge-enhanced responses on a knowledge-grounded dialogue dataset.
MELA: Multilingual Evaluation of Linguistic Acceptability (2024.acl-long)

Copied to clipboard

Challenge: Existing benchmarks on linguistic acceptability have been used to evaluate language models' ability to distinguish between acceptable and unacceptable sentences.
Approach: They present the largest benchmark to date on linguistic acceptability: MELA . they establish LLM baselines on this benchmark and investigate cross-lingual transfer in acceptability judgements with XLM-R.
Outcome: The proposed model outperforms open-source models on cross-lingual transfer in acceptability judgements.
Simple but Effective Compound Geometric Operations for Temporal Knowledge Graph Completion (2024.acl-long)

Copied to clipboard

Challenge: Current methods embed factual knowledge into continuous vector space and apply geometric operations to learn potential patterns in temporal knowledge graphs.
Approach: They propose a temporal knowledge graph completion method that uses two geometric operations to learn missing facts in temporal graphs.
Outcome: The proposed method significantly outperforms existing temporal knowledge graph embedding models.
Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning (2024.findings-emnlp)

Copied to clipboard

Challenge: Achieving consistent high-quality machine translation across diverse domains remains a challenge due to limited and imbalanced parallel training data available in various domains.
Approach: They propose a domain Chain of Thought technique that uses the multi-domain intelligence of LLMs to improve translation performance.
Outcome: The proposed method achieves significant improvements in translation accuracy and domain robustness over traditional fine-tuning on a small dataset of four domains.
TRUST: Towards Robust Social Bot Detection via Uncertainty-Guided Pseudo-Labeling and Graph Structure Purification (2026.findings-acl)

Copied to clipboard

Challenge: Existing graph-based detection models are vulnerable to deceptive message propagation, where bots deliberately interact with legitimate users.
Approach: They propose a framework to mitigate deceptive message propagation by node-level uncertainty estimation and graph structure purification.
Outcome: The proposed framework improves on three benchmark datasets and six GNN backbones on real-world social bots.
SarcNet: A Multilingual Multimodal Sarcasm Detection Dataset (2024.lrec-main)

Copied to clipboard

Challenge: Sarcasm is an implicit form of sarcasm, involving an intended meaning that contradicts the literal expression . human use conflict between factual information and a statement as cues to detect sarcasm . sarkasmatic analysis is challenging due to its implicit nature .
Approach: They propose a multimodal sarcasm detection dataset that uses multiple modalities to detect sarcasm.
Outcome: The proposed model improves on previous models based on a single label . human sarcasm cannot be detected using a unified label across multiple modalities .
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese (2026.tacl-1)

Copied to clipboard

Challenge: Using sub-linear length normalized log-probabilities (SLLN-LP), we find unequal lengths of sentences in minimal pairs difficult for LMs even up to 32B parameters.
Approach: They propose to use ZhoBLiMP as a linguistic minimal pair benchmark for Chinese language models to mitigate biases.
Outcome: The proposed metric mitigates biases in Chinese language models with over 100 paradigms . Anaphor, Quantifiers, and Ellipsis are difficult for LMs even up to 32B parameters .
MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to visual chain-of-thought are limited by external tools or fail to generate high-fidelity diagrams.
Approach: They propose a framework to enable large multimodal models with VCoT capabilities . they pre-train a model on a 15.2M-pair corpus and teach it how to leverage visual aids .
Outcome: The proposed framework unlocks complex, human-like visual reasoning in large language models . it pre-trains the model on a 15.2M-pair corpus and fine-tunes it on MathCanvas-Instruct .
Neural Topic Modeling by Incorporating Document Relationship Graph (2020.emnlp-main)

Copied to clipboard

Challenge: Graph Topic Models (GNNs) capture relationships between graph nodes via message passing . recent research has focused on topic modeling using latent Dirichlet Allocation .
Approach: They propose a Graph Topic Model (GTM) that captures relationships between graph nodes via message passing.
Outcome: The proposed model captures the relationships between nodes via message passing . the results demonstrate that the proposed model is effective in generating documents .
Distilling the Essence, Discarding the Dross: Improving Fairness in Multimodal Large Language Models via Historical Reflection-Guided Prompt Optimization (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to debiase MLLMs rely on handcrafted prompts that are brittle and difficult to generalize across tasks and bias types.
Approach: They propose an adaptive self-debiasing framework that optimizes task-specific debiasers to suppress stereotypical outputs.
Outcome: The proposed framework suppresses stereotypical outputs while maintaining performance.
DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization (2025.findings-naacl)

Copied to clipboard

Challenge: Existing text-to-image (T2I) synthesis diffusion models raise misuse concerns, particularly in creating prohibited or not-safe-for-work (NSFW) images.
Approach: They propose a method which uses zeroth order optimization to procure gradient approximations and harnesses both C-PRV and D-PRv to enhance attack prompts within a discrete prompt space.
Outcome: The proposed method achieves an 8.5% higher average attack success rate than previous works on multiple state-of-the-art safety mechanisms.
SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024.emnlp-demo)

Copied to clipboard

Challenge: Large language models (LLMs) have shown remarkable achievements across various language tasks.
Approach: They propose a scientific literature LLM and a knowledge service system based on it . they collect scientific literature and then pre-train it using autoregressive training .
Outcome: The proposed system provides literature investigation, paper reading, and academic writing functions.
TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Industry Systems (2024.emnlp-industry)

Copied to clipboard

Challenge: Large language models have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools.
Approach: They propose a framework to enhance the task planning and tool usage abilities of LLMs in industrial systems.
Outcome: The proposed framework enhances the task planning and tool usage abilities of LLM-based agents in industrial systems.
CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing Vision-language models are prone to hallucinating nonexistent entities or events and missing subtle but critical visual cues.
Approach: They propose a Traffic VideoQA Benchmark that enforces a single structured decision pattern over each video question quadruple and provides actionable diagnostics that decompose failures into positive omission, positive swap, negative hallucination, mutual-exclusivity violation.
Outcome: The proposed model detects true hazards when an accident occurs, and rejects plausible-but-false hypotheses under near-identical counterfactual scenes.
Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees (2026.acl-long)

Copied to clipboard

Challenge: Existing training data detectors fail to detect clean samples from contaminated test sets . existing methods fail to identify clean samples due to black-box nature of LLMs .
Approach: They propose a framework that detects and filters contaminated evaluation data . they propose 'failure detection' to reduce the proportion of contaminated samples mistakenly retained .
Outcome: The proposed framework reduces false discovery rate (FDR) under valid FDR control while maintaining evaluation consistency.
Translation vs. Dialogue: A Comparative Analysis of Sequence-to-Sequence Modeling (2020.coling-main)

Copied to clipboard

Challenge: Existing models for machine translation and dialogue response generation require a large number of handcrafted features.
Approach: They propose to interpret a general neural model comparatively by using the seq2seq model in two mainstream NLP tasks.
Outcome: The proposed model is used in two mainstream NLP tasks and is compared with a standard model.
LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in Large Language Models have opened new avenues for solving complex optimization problems, including Neural Architecture Search (NAS).
Approach: They propose a framework that leverages LLMs for cross-domain neural architecture optimization without extensive domain-specific tuning.
Outcome: The proposed framework achieves competitive performance in both in-domain and out-of-domain tasks.
MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free (2026.findings-acl)

Copied to clipboard

Challenge: Existing research on Large Language Models (LLMs) limited to textual input modality . acoustic information is intrinsically heterogeneous, entangling attributes such as speech, music, and environmental context.
Approach: They propose a sparse Mixture-of-Experts architecture to decouple acoustic information by routing audio tokens to specialized experts.
Outcome: The proposed architecture outperforms existing models on audio semantic and paralinguistic tasks while retaining shared experts for global context.
Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders (D19-1)

Copied to clipboard

Challenge: Existing work on variableal autoencoders and waterstein autoencoding models has shown significant progress in open-domain response generation.
Approach: They propose to embed user-level and utterance-level information into two multimodal distributions and combine them into a mixed distribution.
Outcome: The proposed model outperforms state-of-the-art models on a large-scale real-world dataset.
Towards Effective and Efficient Continual Pre-training of Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks.
Approach: They propose a Continual pre-training method that can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Outcome: The proposed method can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis (2025.findings-acl)

Copied to clipboard

Challenge: Current generative CSS models face interpretability limitations due to insufficient emotional perception and redundant discrete speech coding.
Approach: They propose a framework that aligns synthesized speech with the emotional context of user-agent interactions to achieve empathy.
Outcome: The proposed framework produces more expressive speech than existing methods on three datasets.
Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations (P19-1)

Copied to clipboard

Challenge: Experimental results show that our model outperforms competitive translation-based baselines on cross-lingual relevance ranking tasks.
Approach: They propose to match queries and documents in both source and target languages with deep bilingual query-document representations.
Outcome: The proposed model outperforms translation-based baselines on English-Swahili, English-Tagalog, and English-Somali cross-lingual retrieval tasks.
Beyond Modality Collapse: Taming Guided Modality Entropy for Omni-modal Emotion Reasoning (2026.findings-acl)

Copied to clipboard

Challenge: EmoOmni is a data paradigm for omni-modal large language models that can be used for emotion reasoning.
Approach: They propose a data paradigm that interleaves guided tokens into reasoning traces to enforce structured evidence extraction.
Outcome: The proposed paradigm over-relys on a dominant modality while neglecting complementary cues.
TreeRL: LLM Reinforcement Learning with On-Policy Tree Search (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for On-Policy LLM RL typically train a separate process reward model, which suffers from distribution mismatch and reward hacking.
Approach: They propose a reinforcement learning framework that directly incorporates on-policy tree search for RL training.
Outcome: Experiments on math and code reasoning benchmarks show that tree search achieves superior performance compared to traditional ChainRL.
LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystem (2025.findings-emnlp)

Copied to clipboard

Challenge: distributing LLMs without a proven track record like ‘meta-llama‘ or ‘qwen‘ rarely gains community traction.
Approach: They propose a simple, efficient, yet specific recipe for a backdoor LoRA to be injected into task-enhancing LoRAs and examine the mechanisms of such infections.
Outcome: The proposed model allows attackers to scale the distribution of compromised LoRAs with minimal effort by leveraging the rich pool of shared LoRA assets.
VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Current Automatic Speech Recognition models, such as Whisper, have demonstrated impressive performance in general domains, but their accuracy often deteriorates significantly in specialized scenarios.
Approach: They propose a visually-anchored policy optimization approach to decouple visual perception from auditory processing to optimize the model's inference process.
Outcome: The proposed model eliminates visual interference and achieves state-of-the-art performance on SlideASR-Bench and public datasets.
Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese (2025.emnlp-main)

Copied to clipboard

Challenge: Translationese is a linguistic property that is often introduced in the translation process that is different from those of original texts.
Approach: They propose to use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments.
Outcome: The proposed measure can generalize to unseen genres, authors, and language pairs.
OTExtSum: Extractive Text Summarisation with Optimal Transport (2022.findings-naacl)

Copied to clipboard

Challenge: Extractive text summarisation aims to select salient sentences from a document to form a short yet informative summary.
Approach: They propose to formulate extractive text summarisation as an Optimal Transport (OT) problem and use it to obtain an optimal summary that minimises the transportation cost to a given document.
Outcome: The proposed method outperforms state-of-the-art methods and learning-based methods on multiNews, PubMed, BillSum, and CNN/DM datasets.
MEraser: An Effective Fingerprint Erasure Approach for Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have raised critical concerns about model ownership and intellectual property protection.
Approach: They propose a method for effectively removing backdoor-based fingerprints from LLMs . they propose deleting backdoor fingerprints using a transferable erasure mechanism .
Outcome: The proposed method removes backdoor-based fingerprints while maintaining model performance.
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to chart-to-code generation are constrained by data-centric limitations . authors present a new framework that redesigns both training and alignment data .
Approach: They propose a data-centric framework that redesigns both training and alignment data for chart-to-code generation.
Outcome: The proposed framework outperforms open-source baselines and is competitive with GPT-5.
Structure Trumps Size: Rethinking Data Quality for LLM Reasoning (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for fine-tuning Large Language Models rely on heuristic strategies and lack systematic, quantitative frameworks for evaluating data quality.
Approach: They propose a multi-dimensional quantitative framework for reasoning data management . they rigorously evaluate and optimize datasets along six orthogonal dimensions .
Outcome: The proposed framework rigorously evaluates and optimizes datasets along six orthogonal dimensions.
From Style to Story: A Curriculum Learning Approach for Imitative Novel Generation (2026.findings-acl)

Copied to clipboard

Challenge: Novels create rich, immersive worlds with intricate plots and distinct styles, captivating readers through complex storytelling.
Approach: They propose a novel generation system that imitates novel elements by predicting plot developments and writing concrete details using vivid, expressive language.
Outcome: The novel imitative novel generation system is trained through a curriculum learning paradigm, progressing from low-level stylistic mastery to high-level narrative coherence.
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) often struggle to accurately express factual knowledge, especially in cases where the knowledge boundaries are ambiguous.
Approach: They propose a framework that leverages Uncertainty estimations to represent knowledge boundaries and incorporates these representations into prompts for LLMs to Align with factual knowledge.
Outcome: The proposed framework significantly improves the LLMs’ capacities to confidently answer known questions and refuse unknown questions on both in-domain and out-of-domain tasks.
Sparse Frame Grouping Network with Action Centered for Untrimmed Video Paragraph Captioning (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for paragraph captioning videos without event ground truths generate one sentence for each event, but without event labels, it is difficult to locate the transitions between events and minimize repetition.
Approach: They propose a module that dynamically groups event information with the help of action information for the entire video and excludes redundant frames within pre-defined clips.
Outcome: The proposed module outperforms the state-of-the-art methods on all metrics.
Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Recent years have witnessed significant advancements in large language models (LLMs) but still struggle with integrating vision and audio.
Approach: They propose a self-knowledge distillation method to improve vision-audio capabilities of OLLMs by learning from the vision-text components.
Outcome: The proposed method improves vision-audio capabilities of OLLMs by learning from vision-text components, which improves interaction between audio and images and results in improved performance on multimodal tasks.
FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data (2025.naacl-long)

Copied to clipboard

Challenge: Prior research on training grounded factuality classification models to detect hallucinations in large language models (LLMs) has relied on public natural language inference (NLI) data and synthetic data.
Approach: They propose a method that leverages multi-hop reasoning on context graphs extracted from documents to generate complex multi-level claims without relying on LLMs to decide data labels.
Outcome: The proposed model outperforms GPT-4-o on the LLM-Aggrefact benchmark with much smaller model size.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Challenge: Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach: They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome: The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.
DNASpeech: A Contextualized and Situated Text-to-Speech Dataset with Dialogues, Narratives and Actions (2025.acl-long)

Copied to clipboard

Challenge: Existing TTS datasets lack situated descriptive prompts aligned with speech data.
Approach: They propose a contextualized and situated text-to-speech task to promote more accurate and customized speech generation using DNA prompts.
Outcome: The proposed task promotes more accurate and customized speech generation using DNA prompts.
ReasonerRank: Redefining Language Model Evaluation with Ground-Truth-Free Ranking Frameworks (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly adopted across real-world applications . traditional evaluations rely on expensive, domain-specific ground-truth labels . obtaining labeled data is expensive, time-consuming, and often requires domain expertise .
Approach: They propose a ground-truth-free evaluation framework focused on reasoning consistency and instruction following.
Outcome: The proposed framework outperforms existing label-free methods, including majority voting, triplet ranking, and peer-review approaches.
Position Paper: Data-Centric AI in the Age of Large Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: a paper proposes a data-centric perspective of AI research, focusing on large language models.
Approach: They propose a data-centric viewpoint of AI research, focusing on large language models . they propose four scenarios centered around data, including data curation, attribution, knowledge transfer .
Outcome: The proposed research focuses on large language models with data centric benchmarks . the proposed benchmarks can be used to develop new data curation methods .
MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated remarkable capabilities on general text, but their proficiency in specialized scientific domains remains uncharacterized.
Approach: They evaluate the capabilities of large language models in metabolomics research using MetaBench . they found that models perform well on text generation tasks, but cross-database identifier grounding remains challenging .
Outcome: The evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks.
ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold (2024.findings-acl)

Copied to clipboard

Challenge: Existing knowledge graphs focus on the representation and reasoning of general factual knowledge, while there are significant deficiencies in the understanding and reasoning for emotional knowledge.
Approach: They propose a commonsense knowledge graph that can be used to represent emotional knowledge by combining theories from psychology, cognitive science, and linguistics.
Outcome: The proposed model surpasses GPT-4-Turbo in the emotion-related tasks.
Neural Topic Modeling with Bidirectional Adversarial Training (2020.acl-main)

Copied to clipboard

Challenge: Recent studies have shown that neural topic models for automatic topic extraction avoid complicated mathematical derivations for model inference.
Approach: They propose a bidirectional adversarial topic model which uses a generator and an encoder to infer topic distribution.
Outcome: The proposed model outperforms baselines and competitive models in three benchmark corpora.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations