Papers with FM
From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the operational characteristics of different integration strategies for robotics . end-to-end vision-language-action models implicitly unify perception and planning . |
| Approach: | They propose end-to-end vision-language-action models that implicitly unify perception and planning . they also propose modular pipelines using either vision-linguistic models or MLLMs . |
| Outcome: | The proposed frameworks implicitly unify perception and planning, and modular pipelines using either vision-language models or multimodal large language models. |
Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) are a promising tool for disentangling FM representations, but they struggle to capture rare, yet crucial concepts in the data. |
| Approach: | They propose a technique to train Sparse Autoencoders to illuminate elusive dark matter features by focusing on specific subdomains. |
| Outcome: | The proposed method achieves 12.5% better classification accuracy than general-purpose SAEs when applied to remove spurious gender information. |
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)
Copied to clipboard
Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product. |
| Approach: | They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap . |
| Outcome: | The proposed method categorizes hallucination into six types based on their degree, orientation, and category . |
Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle to meet user’s needs when required to generate responses of a specific length due to their inherent difficulty in accurately perceiving numerical constraints. |
| Approach: | They propose a Target Length Generation Task and propose RULER, a model-agnostic approach that controls generated length for large language models. |
| Outcome: | The proposed model-agnostic approach improves instruction-following ability of large language models under length-constrained instructions and can generate appropriate MLT when length constraints are not explicitly provided. |
Low-Cost Generation and Evaluation of Dictionary Example Sentences (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior studies have shown that language models can be trained to generate example sentences, but they relied on costly customized models and word sense datasets for generation and evaluation. |
| Approach: | They propose a new automatic evaluation metric called OxfordEval that measures the win-rate of generated sentences against existing Oxford Dictionary sentences. |
| Outcome: | The proposed model achieves over 85.1% win rate against baseline sentences compared to 39.8% win rate for prior model-generated sentences. |
Duplicate-Aware Controlled Code Generation: Enhancing Copyright Protection with Targeted Reordering Beam Search in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental results demonstrate that TRBS effectively reduces verbatim repetition while maintaining functional adequacy. |
| Approach: | They propose a plug-and-play decoding method that dynamically reorders beam candidates to reduce direct copying. |
| Outcome: | The proposed method reduces verbatim repetition while maintaining functional adequacy on a multi-language code generation benchmark. |
MP-RNA: Unleashing Multi-species RNA Foundation Model via Calibrated Secondary Structure Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental evaluations demonstrate that our RNA FM consistently outperforms existing RNA . |
| Approach: | They propose to use filtered high-fidelity structure annotations to enhance the modeling ability of FMs in single nucleotide resolution tasks. |
| Outcome: | The proposed model outperforms existing RNA FMs on four genomic benchmarks and achieves top-tier results on DNA genomic benchmark. |
Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Foundation Models (FMs) have demonstrated success in a wide range of applications, but their optimization often requires access to sensitive data. |
| Approach: | They propose a framework that combines FMs and Federated Learning to enable privacy-preserving and collaborative learning across multiple end-users. |
| Outcome: | The proposed framework combines benefits of FMs and Federated Learning (FL) it enables privacy-preserving and collaborative learning across multiple end-users. |
Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index (2025.emnlp-main)
Copied to clipboard
| Challenge: | Modern language models are trained on text data downsampled from massive text corpora like Common Crawl. |
| Approach: | They propose an efficient and scalable system that can make petabyte-level text corpora searchable by using the FM-index data structure. |
| Outcome: | The proposed system indexes 83TB of Internet text in 99 days with a single 128-core CPU node (or 19 hours if using 137 such nodes). |