Papers with Llama3
Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-step reasoning suffer from inefficiency and redundancy . existing methods neglect the diversity of task difficulties leading to extensive searches even for easy tasks . |
| Approach: | They propose a method that explores reasoning paths with a gating mechanism that decides whether to conduct a tree search based on the confidence level of answers from a previous simple reasoning method. |
| Outcome: | The proposed method significantly improves accuracy by 4.3% on average while requiring only 31% of computational costs. |
Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing abstention fine-tuning methods cause models to suffer from label noise near the decision boundaries. |
| Approach: | They propose a latent space representation perspective for abstention fine-tuning . they propose 'geometric denoising' framework that constructs a truth hyperplane . |
| Outcome: | The proposed framework significantly enhances model truthfulness and demonstrates strong generalization in out-of-distribution scenarios. |
It’s Not What You Say, It’s How You Say It: Evaluating LLM Responses to Expressions of Belief (2026.acl-long)
Copied to clipboard
| Challenge: | a typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone. |
| Approach: | They propose a typology to evaluate how different EoBs affect whether models follow context versus prior knowledge. |
| Outcome: | The proposed model systematically evaluates 16 LLMs that differ in architecture, scale, and training stages . human listeners subconsciously interpret the belief based on how it is expressed, i.e., its explicitness, tone, or contextual cues. |
Rethinking Data Selection at Scale: Random Selection is Almost All You Need (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing data selection techniques are designed for small data pools, a study finds . filtering data by token length is an efficient method for improving results . |
| Approach: | They use self-scoring methods that do not rely on external help to perform fine-tuning . they also find that filtering data by token length offers a stable and efficient method . |
| Outcome: | The proposed methods outperform random selection on large datasets on large data pools. |
Multilingual Blending: Large Language Model Safety Alignment Evaluation with Language Mixture (2025.findings-naacl)
Copied to clipboard
| Challenge: | a range of representative Large Language Models have exhibited remarkable generalization capabilities across numerous downstream tasks. |
| Approach: | They propose a query-response scheme to evaluate the safety alignment of LLMs . they found that multilingual query-responding significantly amplifies the detriment of malicious queries . |
| Outcome: | The proposed scheme improves the safety alignment of state-of-the-art LLMs under multilingual conditions. |
Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate long one-sentence responses that are less effective because they overlook two crucial factors: intra-cluster similarity and inter-c cluster similarity. |
| Approach: | They propose a method that generalizes semantic entropy and uses token probabilities to quantify uncertainty in large language models. |
| Outcome: | The proposed method can be extended to white-box settings by incorporating token probabilities. |
Empirical Study of Zero-shot Keyphrase Extraction with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | a prompting-based approach can effectively supersede traditional KE methods, a study shows . our code is available at https://github.com/kangnlp/zero-shot-keyphrase-extraction-with-LLMs. |
| Approach: | They propose four prompting strategies for zero-shot keyphrase extraction using Large Language Models. |
| Outcome: | The proposed prompting strategies outperform state-of-the-art prompting methods on KE benchmark datasets. |
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities (2025.naacl-long)
Copied to clipboard
Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
| Challenge: | Recent research shows that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks. |
| Approach: | They propose a framework that crafts adversarial LLMs with enhanced jailbreak ability. |
| Outcome: | ADV-LLM significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. |
Locate-then-Merge: Neuron-Level Parameter Fusion for Mitigating Catastrophic Forgetting in Multimodal LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal instruction tuning often causes catastrophic forgetting of the base LLM’s language ability, even in strong models like Llama3. |
| Approach: | They propose a training-free parameter fusion framework that locates important parameters and selectively merges them. |
| Outcome: | The proposed framework preserves the influence of neurons with large parameter shifts while attenuating the influence . of neurons likely responsible for newly acquired visual capabilities while mitigating language degradation. |
White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Social biases manifest in language agency, but there is no comprehensive benchmark for evaluating such biase in language models. |
| Approach: | They propose a benchmark to evaluate language agency biases in large language models . they propose 'Mitigation via Selective Rewrite' to selectively revise parts of generated texts . |
| Outcome: | The proposed language agency bias evaluation benchmark identifies gender, racial, and intersectional biases in 3 recent LLMs. |
Grammar Control in Dialogue Response Generation for Language Learning Chatbots (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing language learning chatbots and research on second language acquisition benefit from these affordances. |
| Approach: | They ground a dialogue response generation model in a pedagogical repository of grammar skills and evaluate prompting, fine-tuning, and decoding strategies for grammar-controlled dialogue response generators. |
| Outcome: | The proposed model outperforms GPT-3.5 when tolerating minor response quality losses and predicts grammar-controlled responses to support grammar acquisition adapted to learner proficiency. |
Exploiting Instruction-Following Retrievers for Malicious Information Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-following retrievers are increasingly used in real-world applications, but little research has investigated the safety risks associated with their increasing search capabilities. |
| Approach: | They investigate the ability of retrievers to satisfy malicious queries . they find that for >50% of queries, retrievers can select harmful passages . |
| Outcome: | The findings highlight the safety risks associated with instruction-following retrievers . they show that even safety-aligned LLMs can satisfy malicious requests . |
Atomic Self-Consistency for Better Long Form Generations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the accuracy of the information in responses. |
| Approach: | They propose a technique that improves the recall of relevant information in an LLM. |
| Outcome: | The proposed technique improves the recall of relevant information in an LLM. |
Preference Tuning For Toxicity Mitigation Generalizes Across Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Detoxifying multilingual Large Language Models (LLMs) has become crucial due to their increasing global use. |
| Approach: | They propose to use English preference tuning to study cross-lingual detoxification of LLMs. |
| Outcome: | The proposed method reduces toxicity in multilingual LLMs by reducing the probability of mGPT-1.3B generating toxic continuations across 17 languages. |
Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to optimise pretraining performance have not addressed the complexities of domain-adaptive continual pretraining. |
| Approach: | They propose a framework that dynamically assesses learning velocity and adjusts data proportions accordingly, favouring slower learning domains while de-emphasising faster learning ones. |
| Outcome: | The proposed framework achieves performance gains in math and code reasoning tasks and command-line generation benchmarks. |
TUBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning (2025.findings-acl)
Copied to clipboard
Xuanli He, Jun Wang, Qiongkai Xu, Pasquale Minervini, Pontus Stenetorp, Benjamin I. P. Rubinstein, Trevor Cohn
| Challenge: | Despite the increasing support for multilingual capabilities, the impact of backdoor attacks on LLMs remains under-explored. |
| Approach: | They propose to use poisoned instructiontuning data to attack multilingual LLMs . their results show that more powerful models show increased susceptibility to transferable cross-lingual backdoor attacks . |
| Outcome: | The proposed attack is effective in models like BLOOM and GPT-4o with high success rates in more than 7 out of 12 languages. |
Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models (2025.findings-emnlp)
Copied to clipboard
Xiaojun Wu, Junxi Liu, Huan-Yi Su, Zhouchi Lin, Yiyan Qi, Chengjin Xu, Jiajun Su, Jiajie Zhong, Fuwei Wang, Saizhuo Wang, Fengrui Hua, Jia Li, Jian Guo
| Challenge: | Existing financial benchmarks suffer from limited language and task coverage, low-quality datasets, and inadequate adaptability for LLM evaluation. |
| Approach: | They propose a bilingual benchmark for financial LLMs that assesses models’ language understanding and generation capabilities. |
| Outcome: | The proposed bilingual benchmark assesses models’ language understanding and generation capabilities. |
“You are Beautiful, Body Image Stereotypes are Ugly!” BIStereo: A Benchmark to Measure Body Image Stereotypes in Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | BIStereo is a suite of language models that uncover body image stereotypes in language models. |
| Approach: | They propose a metric, TriSentBias, that captures the biased preferences of LMs towards a certain body type over others. |
| Outcome: | The proposed metric captures biased preferences of LMs towards a certain body type over others. |