QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models (2026.eacl-demo)
Copied to clipboard
| Challenge: | Questionnaire-like prompts have become an important format to probe, assess, and utilize large language models (LLMs) |
| Approach: | They propose an open-source Python framework for generating responses from questionnaire-style prompts to support in-silico surveys and annotation tasks with large language models (LLMs). |
| Outcome: | The proposed framework can be used to generate responses from questionnaire-style prompts and to perform annotations on large language models. |
Similar Papers
Qsnail: A Questionnaire Dataset for Sequential Question Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Questionnaires are a professional research methodology used for qualitative and quantitative analysis of human opinions, preferences, and behaviors. |
| Approach: | They propose a questionnaire-based dataset that consists of 13,168 human-written questionnaires. |
| Outcome: | The proposed dataset contains 13,168 human-written questionnaires gathered from online platforms. |
SurveyGen: Quality-Aware Scientific Survey Generation with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Automated survey generation is a key task in scientific document processing due to lack of standardized evaluation datasets. |
| Approach: | They propose a survey-based framework that integrates quality indicators into literature retrieval to assess higher-quality sources. |
| Outcome: | The proposed framework enhances the standard Retrieval-Augmented Generation pipeline and enables human-guided writing. |
Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior work has focused on using large language models to simulate human behaviors . but, LLMs are known to generate erroneous, stereotypical, or overconfident answers . |
| Approach: | They propose to specialize large language models for simulating survey response distributions by first-token probabilities. |
| Outcome: | The proposed model outperforms other methods and zero-shot classifiers on unseen questions, countries, and a completely unseened survey. |
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages (2025.coling-main)
Copied to clipboard
| Challenge: | Question Answering datasets are scarce for languages other than English due to the cost and difficulties of collection and manual annotation. |
| Approach: | They propose a method for generating and validating QA datasets for low-resource languages . they use English data as context to generate synthetic multiple-choice (MC) question-answer pairs . |
| Outcome: | The proposed method maintains quality, reduces likelihood of factual errors, and circumvents costly annotation. |
Learning to Generate Structured Output with Schema Reinforcement Learning (2025.acl-long)
Copied to clipboard
Yaxi Lu, Haolun Li, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun
| Challenge: | Recent advances in large language models have facilitated the development of intelligent applications like automatic web search (Qin et al., 2023) Several methods exist for generating JSON strings from LLMs, including Prompting but often miss certain schemas. |
| Approach: | They propose to use 40K different JSON schemas to assess models' ability to generate valid JSON outputs. |
| Outcome: | The proposed model improves both in generating JSON outputs and downstream tasks. |
Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on generating closed-ended survey responses with large language models, whereas LLMs are typically trained to generate open-ended text. |
| Approach: | They evaluate the impact of various Survey Response Generation Methods on simulated responses by generating closed-ended responses from large language models. |
| Outcome: | The proposed methods perform best in individual-level and subpopulation-level alignment. |
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation (2023.findings-acl)
Copied to clipboard
Xingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine, Rania Abdelghani, Hélène Sauzéon, Pierre-Yves Oudeyer
| Challenge: | Large Language Models (LLMs) have demonstrated impressive prowess in natural language generation. |
| Approach: | They propose a method to select high-quality questions from LLM-generated candidates using round-trip and prompt-based scoring. |
| Outcome: | The proposed approach can select high-quality questions from a set of LLM-generated candidates without modification of the underlying model nor rely on human annotations. |
Large Language Models for Generative Recommendation: A Survey and Visionary Discussions (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized the field of natural language processing but are not fully able to leverage the generative power of LLM. |
| Approach: | They examine the progress, methods, and future directions of large language models . they examine what generative recommendation is, why RS should advance to generative recommendations . |
| Outcome: | The proposed approach can be simplified to generate recommendations from the entire pool of items. |
PythonSaga: Redefining the Benchmark to Evaluate Code Generating LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | *HumanEval* and *MBPP* are two popular benchmarks for Python code generation. |
| Approach: | They propose a large-scale human evaluation of two popular Python benchmarks . they propose 185 hand-crafted prompts in a balanced representation of 38 programming concepts across diverse difficulty levels. |
| Outcome: | The proposed benchmarks show a critical bias towards a limited set of programming concepts, neglecting most of the other concepts entirely. |
Textagon: Boosting Language Models with Theory-guided Parallel Representations (2025.acl-demo)
Copied to clipboard
| Challenge: | Pretrained language models do not account for the wide variety of available expert-generated language resources and lexicons that explicitly encode linguistic/domain knowledge. |
| Approach: | They propose a Python package for generating parallel representations for text based on predefined lexicons and selecting representations that provide the most information. |
| Outcome: | The proposed model can generate parallel representations of text based on predefined lexicons and select representations that provide the most information. |