Papers by James Zhang

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) have highlighted the need for effective unlearning mechanisms to comply with data regulations and ethical AI practices.
Approach:	They propose a second-order optimization-based LLM unlearning framework which extends the static, one-shot model update using influence unlearning to a dynamic, iterative unlearning process.
Outcome:	The proposed framework outperforms first-order methods across unlearning tasks, models, and metrics.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Reasoning

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Platforms for Non-speakers Annotating Names in Any Language (P18-4)

Copied to clipboard

Ying Lin, Cash Costello, Boliang Zhang, Di Lu, Heng Ji, James Mayfield, Paul McNamee

Challenge:	Traditionally, native speakers of a language have been asked to annotate a corpus in that language.
Approach:	They propose two annotation platforms that allow an English speaker to annotate names for any language without knowing the language.
Outcome:	The proposed annotations achieved state-of-the-art performance on two surprise languages and ten languages at TAC-KBP EDL2017.

A Survey on LLMs for Story Generation (2025.findings-emnlp)

Copied to clipboard

Maria Teleki, Vedangi Bengali, Xiangjue Dong, Sai Tejas Janjur, Haoran Liu, Tian Liu, Cong Wang, Ting Liu, Yin Zhang, Frank Shipman, James Caverlee

Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers (2024.emnlp-main)

Challenge:	Methods for story generation with Large Language Models (LLMs) have come into the spotlight recently.
Approach:	They propose a novel taxonomy of LLMs for story generation consisting of two major paradigms: independent story generation by an LLM, and author-assistance for story creation .
Outcome:	The proposed taxonomy compares existing work on the topic with those of novel author-assistance models.

Copied to clipboard

Tianhua Zhang, Kun Li, Hongyin Luo, Xixin Wu, James Glass, Helen Meng

Natural Language Interfaces

Challenge:	Existing methods to incorporate retriever’s preference during the training of query rewriting models rely on extensive annotations such as in-domain rewrites and/or relevant passage labels, limiting their generalization and adaptation capabilities.
Approach:	They propose a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label.
Outcome:	The proposed approach decontexualizes conversational queries into self-contained questions suitable for off-the-shelf retrievers.

Reasoning or Knowledge: Stratified Evaluation of Biomedical LLMs (2026.eacl-long)

Copied to clipboard

Rahul Thapa, Qingyang Wu, Kevin Wu, Harrison G Zhang, Angela Zhang, Eric Wu, Haotian Ye, James Zou

Robustness in NLP

Challenge:	Medical reasoning in large language models is a complex cognitive process through which clinicians interpret patient data and make diagnostic and therapeutic decisions.
Approach:	They propose an evaluation framework that disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks.
Outcome:	The proposed evaluation framework disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks.

Werewolf Among Us: Multimodal Resources for Modeling Persuasion Behaviors in Social Deduction Games (2023.findings-acl)

Copied to clipboard

Bolin Lai, Hongxin Zhang, Miao Liu, Aryan Pariani, Fiona Ryan, Wenqi Jia, Shirley Anugrah Hayati, James Rehg, Diyi Yang

Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification (2023.emnlp-main)

Challenge:	Existing studies on persuasive behavior modeling focus on textual dialogues . a multimodal dataset is available for persuasion modeling .
Approach:	They propose a multimodal dataset for modeling persuasive behaviors using visual signals.
Outcome:	The proposed dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting and 26,647 utterance level annotations of persuasion strategy and game level annotation of deduction game outcomes.

Copied to clipboard

Mujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas, Raphael Shu, Salvatore Romeo, Yi Zhang, Vittorio Castelli

Search Augmented Instruction Learning (2023.findings-emnlp)

Challenge:	Existing methods for IC training do not provide sufficient examples for each intent . a novel pre-training method is proposed to provide a better understanding of intents .
Approach:	They propose a method that uses contrastive learning with intent psuedo-labels to produce embeddings that are well-suited for IC tasks.
Outcome:	The proposed method achieves 5.4% and 4.0% higher accuracy than the current state-of-the-art method on four IC datasets.

Copied to clipboard

Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass

Responsible & Trustworthy NLP

Challenge:	Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information.
Approach:	They propose a search-augmented instruction learning model which grounds the language generation and instruction following abilities on complex search results generated by in-house and external search engines.
Outcome:	The proposed model outperforms plain LLMs on zero-shot language tasks and can generate both natural and programming languages following natural language guidance and requests.

A Large Scale Speech Sentiment Corpus (2020.lrec-1)

Copied to clipboard

Eric Chen, Zhiyun Lu, Hao Xu, Liangliang Cao, Yu Zhang, James Fan

CONFETTI: Conversational Function-Calling Evaluation Through Turn-Level Interactions (2025.acl-long)

Challenge:	Existing corpus for sentiment analysis uses text inputs, but voice inputs are becoming more important as smart assistants and mobile voice control become more prevalent.
Approach:	They propose to extend the Switchboard-1 Telephone Speech Corpus by adding sentiment labels from 3 different human annotators for every transcript segment.
Outcome:	The proposed corpus contains 49500 labeled speech segments covering 140 hours of audio.

Copied to clipboard

Tamer Alkhouli, Katerina Margatina, James Gung, Raphael Shu, Claudia Zaghi, Monica Sunkara, Yi Zhang

Dialogue Systems & Conversational Agents

Challenge:	Current benchmarks lack comprehensive assessment of large language models in complex conversational scenarios.
Approach:	They introduce a conversational benchmark to evaluate function-calling capabilities of large language models.
Outcome:	The proposed benchmark evaluates the function-calling capabilities and response quality of large language models.

Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages (D19-61)

Copied to clipboard

James Route, Steven Hillis, Isak Czeresnia Etinger, Han Zhang, Alan W Black

Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning (2025.emnlp-main)

Challenge:	Grapheme-to-phoneme conversion (g2p) is a task of predicting the pronunciation of words from their orthographic representation.
Approach:	They propose to leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal grapheme representation.
Outcome:	The proposed model reduces phoneme error rate to 2.46% on in-domain test set compared to unimodal spelling- pronunciation model.

Copied to clipboard

Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, Qun Liu, Bo Han, James Kwok, Yu Zhang

Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition (2020.emnlp-main)

Challenge:	Visual Instruction Tuning (VIT) aims to enhance Multimodal Large Language Models (MLLMs), but its effectiveness is often compromised by corrupted datasets with issues such as hallucinated content and poor OCR quality.
Approach:	They propose a corruption-robust training paradigm that surpasses existing strategies for mitigating the effects of corrupted data.
Outcome:	The proposed training paradigm surpasses existing strategies for mitigating the effects of corrupted data.

Copied to clipboard

Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, James Caverlee

Value Compass Benchmarks: A Comprehensive, Generative and Self-Evolving Platform for LLMs’ Value Evaluation (2025.acl-demo)

Challenge:	Existing methods to augment pre-trained language models with disease knowledge are lacking.
Approach:	They propose a method to augment BERT-like pre-trained language models with disease knowledge.
Outcome:	The proposed method improves on a suite of BERT models over three tasks.

Copied to clipboard

Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Yang Ou, Scarlett Li, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, James Evans, Xing Xie

Challenge:	Current evaluation methods for large language models face two key challenges: 1. evaluation validity and 2. Result interpretation reduce the pluralistic and incommensurable values to one-dimensional scores.
Approach:	They propose a platform for comprehensive value diagnosis of large language models (LLMs) that provides a generative evaluation paradigm that automatically creates real-world test items co-evolving with ever-advancing LLMs.
Outcome:	The proposed platform provides a framework for comprehensive value diagnosis of large language models (LLMs) with fine-grained scores and case studies across 27 value dimensions for 33 leading LLMs, customized comparisons, and visualized analysis of LLM’s alignment with cultural values.

COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)

Copied to clipboard

Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych

Structured Data in NLP

Challenge:	a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications .
Approach:	a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19.
Outcome:	a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing .

Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning (2024.findings-naacl)

Copied to clipboard

Tianhua Zhang, Jiaxin Ge, Hongyin Luo, Yung-Sung Chuang, Mingye Gao, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass

PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge (2020.emnlp-main)

Challenge:	Existing methods for surfacing symbolic reasoning capabilities are limited to narrow tasks . arithmetic computations are unnatural to perform in pure language space, and hence present difficulties for LLMs.
Approach:	They propose a natural language embedded program framework for solving symbolic reasoning tasks.
Outcome:	The proposed framework improves on strong baselines across math and symbolic reasoning, text classification, question answering, and instruction following tasks.

Copied to clipboard

Yun He, Zhuoer Wang, Yin Zhang, Ruihong Huang, James Caverlee

Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models (2023.findings-acl)

Challenge:	Paraphrase identification requires specialized domain knowledge to perform . state-of-the-art neural models and non-expert human annotators have poor performance on PARADE .
Approach:	They propose a benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge.
Outcome:	The proposed dataset shows state-of-the-art models and non-expert human annotators have poor performance on PARADE.

Copied to clipboard

Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, Serena Yeung

DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings (2022.naacl-main)

Challenge:	Recent studies show that some tasks exhibit inverse scaling, or U-shaped scaling, where the performance degrades as models are scaled up.
Approach:	They propose a task that asks questions with negation to show positive scaling . they hypothesize that solving NeQA depends on question answering and negation understanding .
Outcome:	The proposed task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the scaling trends shift as the task is more powerful.

Copied to clipboard

Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, James Glass

Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (D18-1)

Challenge:	Recent work shows that finetuning pretrained models with contrastive learning makes it possible to learn good sentence embeddings without labeled data.
Approach:	They propose an unsupervised contrastive learning framework for learning sentence embeddings . they use a masked language model to mask out the edited sentence .
Outcome:	The proposed framework outperforms SimCSE on semantic textual similarity tasks by 2.3 absolute points.

Copied to clipboard

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev

Programming Languages in NLP

Challenge:	Existing datasets for semantic parsing are too small in terms of number of programs for training modern data-intensive models.
Approach:	They propose a large-scale complex and cross-domain semantic parsing task for a database . they use a dataset with 10,181 questions and 5,693 unique complex SQL queries .
Outcome:	The proposed task is different from previous tasks because it uses the same database and program . the best model achieves only 9.7% exact matching accuracy on a database split setting.

Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompts (2023.findings-emnlp)

Copied to clipboard

Gangwei Jiang, Caigao Jiang, Siqiao Xue, James Zhang, Jun Zhou, Defu Lian, Ying Wei

KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion (2023.findings-emnlp)

Challenge:	Continual pre-training has been used for a multitude of domains and tasks . a continually pre-trained model can show a non-decreasing performance on unseen domains .
Approach:	They propose a method that generates domain-specific prompts by agreement and disagreement losses.
Outcome:	The proposed method achieves improvements of 3.57% and 3.4% on two real-world datasets.

Copied to clipboard

Yanbin Wei, Qiushi Huang, Yu Zhang, James Kwok

Tanbih: Get To Know What You Are Reading (D19-3)

Challenge:	Existing knowledge graph completion methods struggle with long-tail entities due to limited structural information and imbalanced distributions of entities.
Approach:	They propose a framework that integrates a large language model and a triple-based KGC retriever to alleviate the long-tail problem without incurring additional training overhead.
Outcome:	The proposed model reduces training overhead and finetuning costs on benchmark datasets.

Copied to clipboard

Yifan Zhang, Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James Glass, Preslav Nakov

Challenge:	Nowadays, more and more readers consume news online.
Approach:	They propose a news platform that displays news grouped into events and generates media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting and stance with respect to various claims and topics of a media outlet.
Outcome:	The proposed news platform displays news grouped into events and generates media profiles that show the factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting and stance with respect to various claims and topics of a news outlet.

More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)

Copied to clipboard

Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang

Low-Resource NLP

Challenge:	Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance?
Approach:	They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions.
Outcome:	The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset.

Protein Large Language Models: A Comprehensive Survey (2025.findings-emnlp)

Copied to clipboard

Yijia Xiao, Wanjia Zhao, Junkai Zhang, Yiqiao Jin, Han Zhang, Zhicheng Ren, Renliang Sun, Haixin Wang, Guancheng Wan, Pan Lu, Xiao Luo, Yu Zhang, James Zou, Yizhou Sun, Wei Wang

Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing (2023.findings-emnlp)

Challenge:	Existing studies focus on specific aspects or applications, but this study provides a comprehensive overview of Protein-specific large language models.
Approach:	This paper proposes a structured taxonomy of state-of-the-art ProteinLLMs . they analyze how they leverage large-scale protein sequence data for improved accuracy .
Outcome:	The proposed model covers their architectures, training datasets, evaluation metrics, and diverse applications.

Copied to clipboard

Shufan Wang, Sébastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, Yi Zhang

On the Blind Spots of Model-Based Evaluation Metrics for Text Generation (2023.acl-long)

Challenge:	In task-oriented semantic parsing, the system aims to translate users’ utterances in natural language to machine-interpretable programs (API calls) However, Large Language Models (LLMs) are known to hallucinate and therefore pose a formidable challenge in constraining generated content.
Approach:	They propose to use large language models to translate user's utterances to machine-interpretable programs (API calls) they identify constraints violations in task-oriented utterrances and define fine-grained metrics that complement traditional ones.
Outcome:	The proposed methods reduce constraints violations and improve quality of the generated API calls, but require careful consideration given their implementation complexity and latency.

Copied to clipboard

Tianxing He, Jingyu Zhang, Tianle Wang, Sachin Kumar, Kyunghyun Cho, James Glass, Yulia Tsvetkov

PCFG-Based Natural Language Interface Improves Generalization for Controlled Text Generation (2023.starsem-1)

Challenge:	Existing methods for text generation evaluation metrics are lacking in robustness analysis.
Approach:	They propose to use stress tests to test for errors in text generation evaluation metrics . they find that BERTScore is confused by truncation errors in summarization .
Outcome:	The proposed stress tests show that they are insensitive to errors in open-ended generation, translation, and summarization.

Copied to clipboard

Jingyu Zhang, James Glass, Tianxing He

Zero-Shot Open-Schema Entity Structure Discovery (2026.eacl-long)

Challenge:	Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes.
Approach:	They propose a natural language interface to embed control attributes into natural language commands and propose variants of existing CTG models that take commands as input.
Outcome:	The proposed model can generalize to unseen attributes and unsealed attribute combinations.

Copied to clipboard

Xueqiang Xu, Jinfeng Xiao, James Barry, Mohab Elkaref, Jiaru Zou, Pengcheng Jiang, Yunyi Zhang, Maxwell J Giammona, Geeth De Mel, Jiawei Han

Information Extraction & Text Mining

Challenge:	Existing methods based on large language models (LLMs) rely heavily on predefined entity attribute schemas or annotated datasets, often leading to incomplete extraction results.
Approach:	They propose a novel approach to entity structure extraction that does not require any schema or annotated datasets.
Outcome:	Experiments show that ZOES improves LLMs’ ability to extract more complete entity structures across three different domains, showcasing both the effectiveness and generalizability of the method.

Intermediate Layer Distillation with the Reused Teacher Classifier: A Study on the Importance of the Classifier of Attention-based Models (2024.findings-emnlp)

Copied to clipboard

Hang Zhang, Seyyed Mozafari, James Clark, Brett Meyer, Warren Gross

PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR (2026.eacl-long)

Challenge:	Existing methods underestimate the importance of utilizing the teacher's discriminative classifier and face challenges in establishing proper layer mappings.
Approach:	They propose to reuse pre-trained teacher classifiers to improve student performance . they use projectors to match hidden size of the teacher model to student .
Outcome:	The proposed method outperforms existing methods on 97.7% of the teacher BERT base without additional trainable parameters.

Copied to clipboard

James Burgess, Jan N. Hansen, Duo Peng, Yuhui Zhang, Alejandro Lozano, Min Woo Sun, Emma Lundberg, Serena Yeung-Levy

Information Retrieval

Challenge:	Recent methods supervise only the final answer accuracy using reinforcement learning with verifiable rewards (RLVR).
Approach:	They propose to train search agents to search and reason over scientific papers and a factoid QA dataset with 60k biomedical paper abstracts.
Outcome:	The proposed model outperforms non-RL retrieval baselines and is scalable and extendable to other scientific domains.

Discovering Language Model Behaviors with Model-Written Evaluations (2023.findings-acl)

Copied to clipboard

Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, Jared Kaplan

Mixture of insighTful Experts (MoTE): The Synergy of Reasoning Chains and Expert Mixtures in Self-Alignment (2025.acl-long)

Challenge:	Prior work creates evaluations with crowdwork or existing data sources, which are not always available.
Approach:	They generate evaluations automatically with language models (LMs) using crowdwork or existing data sources to find out how they behave .
Outcome:	The results show that large LMs repeat back a dialog user’s preferred answer and express greater desire to pursue concerning goals like resource acquisition and goal preservation.

Copied to clipboard

Zhili Liu, Yunhao Gou, Kai Chen, Lanqing Hong, Jiahui Gao, Fei Mi, Yu Zhang, Zhenguo Li, Xin Jiang, Qun Liu, James Kwok

Reasoning

Challenge:	Recent studies show that reasoning abilities contribute significantly to model safety, while integrating Mixture-of-Experts (MoE) architectures can further enhance alignment.
Approach:	They propose a framework that synergistically combines reasoning chains and expert mixtures to improve self-alignment.
Outcome:	The proposed framework improves model safety, jailbreak resistance, and over-refusal capabilities, achieving performance comparable to OpenAI’s state-of-the-art o1 model.

SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (2024.emnlp-main)

Copied to clipboard

Holy Lovenia, Rahmad Mahendra, Salsabil Akbar, Lester James Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno Kampman, Joel Moniz, Muhammad Habibi, Frederikus Hudi, Jann Montalan, Ryan Hadiwijaya, Joanito Lopo, William Nixon, Börje Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Irawan, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Ryanda, Sonny Hermawan, Dan Velasco, Muhammad Kautsar, Willy Hendria, Yasmin Moslem, Noah Flynn, Muhammad Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Tai Chia, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Aji, Sedrick Keh, Genta Winata, Ruochen Zhang, Fajri Koto, Zheng Xin Yong, Samuel Cahyawijaya

Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2021.emnlp-main)

Challenge:	Southeast Asia (SEA) is home to over 1,300 indigenous languages and 671 million people . prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA .
Approach:	They propose to provide a resource center that provides standardized corpora in nearly 1,000 SEA languages across three modalities.
Outcome:	a new benchmark assesses the quality of AI models on 36 SEA languages across 13 tasks . the results highlight the importance of SEA as a culturally diverse region .

Copied to clipboard

Tianxing He, Jingzhao Zhang, Zhiming Zhou, James Glass