Papers with CommonGen
Mention Flags (MF): Constraining Transformer-based Text Generators (2021.acl-long)
Copied to clipboard
| Challenge: | Constrained decoding algorithms produce hypotheses satisfying all constraints, but they are computationally expensive and can lower the generated text quality. |
| Approach: | They propose a Mention Flag mechanism which traces whether lexical constraints are satisfied in outputs of an S2S decoder. |
| Outcome: | The proposed models maintain higher constraint satisfaction and text quality than baseline models and other constrained decoding algorithms. |
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability (2025.acl-long)
Copied to clipboard
| Challenge: | generative large language models (LLMs) compose sentences that include all given concepts but must generate sentences that adhere to the specified order. |
| Approach: | They propose a benchmark to evaluate compositional generalization and instruction-following abilities of generative large language models (LLMs) based on ordered coverage, which allows simultaneous evaluation of both abilities. |
| Outcome: | The proposed benchmark evaluates compositional generalization and instruction-following abilities of LLMs. |