GPTS

Papers with GPTs

Open-ended Long Text Generation via Masked Language Modeling (2023.acl-long)

Copied to clipboard

Xiaobo Liang, Zecheng Tang, Juntao Li, Min Zhang

Challenge:	Pre-trained autoregressive language models have dominated OPen-ended Long Text Generation (Open-LTG) however, the low inference efficiency of AR impedes their usability.
Approach:	They propose a representative iterative non-autoregressive (NAR) decoding strategy to improve inference efficiency for Open-LTG.
Outcome:	The proposed model can generate short text and collapse for long text modeling.

Do GPTs Produce Less Literal Translations? (2023.acl-short)

Copied to clipboard

Vikas Raunak, Arul Menezes, Matt Post, Hany Hassan

Multilinguality

Challenge:	Large Language Models (LLMs) are general-purpose language models capable of many natural language generation or understanding tasks.
Approach:	They investigate how LLMs differ qualitatively from standard Neural Machine Translation models by measuring literalness and monotonicity.
Outcome:	The proposed models achieve close to state-of-the-art translation performance under few-shot prompting . the results are backed up by human evaluations and a newer MT quality metrics .

VariErr NLI: Separating Annotation Error from Human Label Variation (2024.acl-long)

Copied to clipboard

Leon Weber-Genzel, Siyao Peng, Marie-Catherine De Marneffe, Barbara Plank

Responsible & Trustworthy NLP

Challenge:	Existing work on label variation and annotation errors has focused on them in isolation.
Approach:	They propose a 2-round annotation procedure to separate human label variation from annotation errors by pairing valid explanations with annotators' validations.
Outcome:	The proposed procedure is based on the NLI task in English and contains 7,732 valid judgements on 1,933 explanations for 500 re-annotated items.

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training (2024.acl-long)

Copied to clipboard

Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, Minlie Huang

Language Models

Challenge:	Large language models are often not well aligned with human intents, which requires additional training.
Approach:	They propose to use Black-Box Prompt Optimization (BPO) to perform alignments on large language models that are not well aligned with human intents.
Outcome:	The proposed model outperforms existing models and is model-agnostic.

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022.emnlp-main)

Copied to clipboard

Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, Zhiting Hu

Responsible & Trustworthy NLP

Challenge:	Existing methods for finding the optimal prompt for a task are difficult to optimize.
Approach:	They propose an efficient discrete prompt optimization approach with reinforcement learning that generates the optimal discrete stimulus after training with reward.
Outcome:	The proposed approach is based on a parameter-efficient policy network that generates the optimal discrete prompt after training with reward.

Can We Edit Factual Knowledge by In-Context Learning? (2023.emnlp-main)

Copied to clipboard

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

Semantic Text Processing

Challenge:	In-context knowledge editing (IKE) is a new paradigm for NLP research that can be applied to large language models with tens or hundreds of parameters.
Approach:	They propose to use in-context knowledge editing (IKE) without gradient updating to edit factual knowledge without a gradient update.
Outcome:	The proposed method achieves a competitive success rate compared to gradient-based methods on GPT-J but with fewer side effects.

LegalViz: Legal Text Visualization by Text To Diagram Generation (2025.naacl-long)

Copied to clipboard

Eri Onami, Taiki Miyanishi, Koki Maeda, Shuhei Kurita

Multimodality

Challenge:	Graphviz provides diagrams for legal documents that are easy to understand and understand . a novel dataset of 23 languages and 7,010 cases of legal document and visualization pairs is proposed .
Approach:	They propose a dataset of legal diagrams using DOT graph description language of Graphviz.
Outcome:	The proposed dataset outperforms existing models including GPTs in 23 languages and 7,010 cases of legal document and visualization pairs.

Question-Instructed Visual Descriptions for Zero-Shot Video Answering (2024.findings-acl)

Copied to clipboard

David Mogrovejo, Thamar Solorio

Multimodality

Challenge:	Existing models for video QA rely on complex architectures, expensive pipelines or closed models like GPTs.
Approach:	They propose a single instruction-aware open vision-language model to tackle videoQA using frame descriptions.
Outcome:	The proposed framework achieves higher performance than current state-of-the-art models on videoQA benchmarks.

Revisiting the Knowledge Injection Frameworks (2023.emnlp-main)

Copied to clipboard

Peng Fu, Yiming Zhang, Haobo Wang, Weikang Qiu, Junbo Zhao

Semantic Text Processing

Challenge:	Injecting unaligned knowledge tuple into large language models achieves comparable (and sometimes better) results than aligned knowledge.
Approach:	They propose a technique to inject random knowledge into large language models to improve performance.
Outcome:	The proposed technique overcomes the sanity problem and pushes the performance limit.

KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions (2025.coling-main)

Copied to clipboard

Yanxu Zhu, Jinlin Xiao, Yuhang Wang, Jitao Sang

Structured Data in NLP

Challenge:	Existing benchmarks that assess this vulnerability rely on manual construction, resulting in limited size and lack of expandability.
Approach:	They propose a method to generate false premise questions based on knowledge graphs . they modify true triplets extracted from KGs to create false premises .
Outcome:	The proposed method generates semantically rich FPQs using state-of-the-art GPTs.

When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs (2025.acl-long)

Copied to clipboard

Xinyue Shen, Yun Shen, Michael Backes, Yang Zhang

Responsible & Trustworthy NLP

Challenge:	Existing studies show that adversarial prompts can induce GPTs to leak knowledge file content.
Approach:	They propose a workflow inspired by Data Security Posture Management to identify five leakage vectors for knowledge file leakage using 651,022 GPT metadata and 11,820 flows.
Outcome:	The proposed workflow analyzes 651,022 GPT metadata, 11,820 flows, and 1,466 responses to identify five leakage vectors: metadata, GPT initialization, retrieval, sandboxed execution environments, and prompts.

FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping (2024.emnlp-main)

Copied to clipboard

Ajay Jaiswal, Bodun Hu, Lu Yin, Yeonju Ro, Tianlong Chen, Shiwei Liu, Aditya Akella

Challenge:	Autoregressive Large Language Models (LLMs) are omnipresent but typically come with a substantial model size.
Approach:	They propose a novel fine-grained skip strategy for autoregressive large language models . they observe the saturation of computationally expensive feed-forward blocks of LLMs .
Outcome:	The proposed method can skip 25-30% of FFN blocks with marginal change in performance on knowledge-intensive generation tasks.

Segment-Based Attention Masking for GPTs (2025.acl-long)

Copied to clipboard

Shahar Katz, Liran Ringel, Yaniv Romano, Lior Wolf

Language Models

Challenge:	Causal masking is a fundamental component in Generative Pre-Trained Transformers (GPT) models, playing a crucial role during training.
Approach:	They propose to apply causal masking to all input tokens step-by-step, mimicking the generation process.
Outcome:	The proposed model can process the entire user prompt at once, but it is applied to all input tokens step-by-step, mimicking the generation process.