Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

Challenge:	Apertus is a fully open suite of large language models (LLMs) designed to address responsibility shortcomings in today’s open model ecosystem, namely data responsibility and global representation.
Approach:	They propose to release a fully open suite of large language models (LLMs) that address data responsibility and global representation shortcomings in today’s open model ecosystem.
Outcome:	The proposed model is pretrained on openly available data and suppresses verbatim recall of data while retaining task performance.

Similar Papers

Overview

Challenge

Approach

Outcome

H2O Open Ecosystem for State-of-the-art Large Language Models (2023.emnlp-demo)

Copied to clipboard

Arno Candel, Jon McKinney, Philipp Singer, Pascal Pfeiffer, Maximilian Jeblick, Chun Ming Lee, Marcos Conde

Language Models

Challenge:	Large Language Models (LLMs) are a revolution in AI, but they pose many significant risks, such as the presence of biased, private, copyrighted or harmful text.
Approach:	They propose to develop and test Large Language Models using open-source tools and frameworks.
Outcome:	The proposed framework and models are licensed under Apache 2.0 licenses.

LLMs for Low Resource Languages in Multilingual, Multimodal and Dialectal Settings (2024.eacl-tutorials)

Copied to clipboard

Firoj Alam, Shammur Absar Chowdhury, Sabri Boughorbel, Maram Hasanain

Multimodality

Challenge:	Recent advances in AI can be attributed to the remarkable performance of Large Language Models (LLMs) success of LLMs depends on specific training techniques, such as instruction tuning and prompting .
Approach:	They explore the capabilities of Large Language Models (LLMs) in various tasks and languages . they also examine their performance, fine-tuning, instructions tuning, and close vs. open models .
Outcome:	The proposed model can be used for speech and multimodal tasks across modalities, languages, and dialects.

The Impossibility of Fair LLMs (2025.acl-long)

Copied to clipboard

Jacy Reese Anthis, Kristian Lum, Michael Ekstrand, Avi Feller, Chenhao Tan

Responsible & Trustworthy NLP

Challenge:	Existing frameworks for evaluating large language models do not extend to general-purpose AI contexts or are infeasible in practice.
Approach:	They analyze a variety of technical fairness frameworks to find inherent challenges . they find that each framework does not logically extend to the general-purpose AI context .
Outcome:	The proposed frameworks do not logically extend to the general-purpose AI context or are infeasible in practice due to large amounts of unstructured training data and potential combinations of human populations, use cases, and sensitive attributes.

Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains (2026.findings-acl)

Copied to clipboard

Md. Faiyaz Abdullah Sayeedi, Subhey Sadi Rahman, Md. Mahbub Alam, Md. Adnanul Islam, Jannatul Ferdous Deepti, Tasnim Mohiuddin, Md Mofijul Islam, Swakkhar Shatabda

Multilinguality

Challenge:	Large Language Models (LLMs) have redefined Machine Translation, enabling context-aware and fluent translations across hundreds of languages and textual domains.
Approach:	They propose a framework and dataset to evaluate the translation quality and fairness of open-source LLMs.
Outcome:	The proposed framework and dataset evaluates translation quality and fairness of open-source LLMs.

Toward Global AI Inclusivity: A Large-Scale Multilingual Terminology Dataset (GIST) (2025.findings-acl)

Copied to clipboard

Jiarui Liu, Iman Ouzzani, Wenkai Li, Lechen Zhang, Tianyue Ou, Houda Bouamor, Zhijing Jin, Mona T. Diab

Multilinguality

Challenge:	Despite advances in machine translation, domain-specific terminology translation remains challenging.
Approach:	They propose a large-scale multilingual AI terminology dataset that combines LLMs for extraction with human expertise for translation.
Outcome:	The proposed framework combines human translation expertise with LLMs to improve translation accuracy and improve BLEU and COMET scores.

Enhancing LLM Capabilities Beyond Scaling Up (2024.emnlp-tutorials)

Copied to clipboard

Wenpeng Yin, Muhao Chen, Rui Zhang, Ben Zhou, Fei Wang, Dan Roth

Language Models

Challenge:	general-purpose large language models (LLMs) are expanding in scale and access to unpublic training data.
Approach:	This tutorial aims to examine the capabilities of general-purpose large language models . authors discuss adaptation of LLMs to address conflicts, defense against attacks .
Outcome:	This tutorial aims to examine the evolution of general-purpose large language models (LLMs) the authors argue that the evolution is dependent on the availability of training data and the scale of the models.

The Data Frontier for Large Language Models: Selection, Synthesis, and Tools (2026.acl-tutorials)

Copied to clipboard

Lijun Wu, Wentao Zhang, Conghui He

Language Models

Challenge:	acquiring and curating high-quality training data remains a significant bottleneck . acquiring such high-quality data is a key challenge for researchers and practitioners .
Approach:	This tutorial provides a comprehensive and practical guide to the state-of-the-art in data research directions for LLMs.
Outcome:	The tutorial covers methods for curating the most valuable information from vast, noisy datasets and the synthetic data revolution.

Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)

Copied to clipboard

Anoop Kunchukuttan, Raj Dabre, Rudra Murthy, Mohammed Safi Ur Rahman Khan, Thanmay Jayakumar

Language Models

Challenge:	Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research .
Approach:	This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc.
Outcome:	This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages .

Machine Unlearning of Pre-trained Large Language Models (2024.acl-long)

Copied to clipboard

Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, Xiang Yue

Responsible & Trustworthy NLP

Challenge:	Using curated datasets, we establish a robust benchmark for unlearning performance, demonstrating that these methods are over 105 times more computationally efficient than retraining.
Approach:	They propose a framework for machine unlearning in pre-trained LLMs and integrate gradient ascent with gradient descent on in-distribution data to achieve robustness.
Outcome:	The proposed framework is over 105 times more efficient than retraining on in-distribution data and provides detailed guidelines for efficient hyperparameter tuning in the unlearning process.

VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)

Copied to clipboard

Jiu Sha, Mengxiao Zhu, Chong Feng, Yuming Shang

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks .
Approach:	They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support.
Outcome:	The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks.