PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. |
| Approach: | They propose two methods to improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
| Outcome: | The proposed methods improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality. |
Similar Papers
Zero-Shot Strategies for Length-Controllable Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models struggle with precise length control, particularly in zero-shot settings. |
| Approach: | They propose to use length approximation, target adjustment, sample filtering and automated revisions to improve LLMs' length control capabilities. |
| Outcome: | The proposed methods improve length control in large language models while maintaining or enhancing summary quality without the need for model fine-tuning or architectural changes. |
DcLM: Output Length Control of Large Language Models via Dynamic Length Markers (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have limited awareness of output length, making it difficult to satisfy precise length requirements. |
| Approach: | They propose a model-agnostic approach that introduces dynamic length markers to guide length-controllable outputs. |
| Outcome: | The proposed method significantly reduces length deviation across multiple datasets. |
Length Representations in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable capabilities across various tasks, that are learned from massive amounts of text-based data. |
| Approach: | They propose to scale hidden units within the model to control output sequence length without losing the informativeness of the generated text. |
| Outcome: | The output sequence length is controlled by multiple head attention mechanisms, which can be adjusted in a disentangled manner. |
Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence. |
| Approach: | They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself. |
| Outcome: | The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios. |
PIC: Unlocking Long-Form Text Generation Capabilities of Large Language Models via Position ID Compression (2025.acl-long)
Copied to clipboard
| Challenge: | Long-context understanding is crucial for large language models (LLMs) however, the ability to “output-long” is underexplored. |
| Approach: | They propose a position ID compression approach to unlock the long-form text generation potential of large language models (LLMs). |
| Outcome: | The proposed approach can extend LLMs' generation length by 1.5 times without compromising generation quality. |
Can We Instruct LLMs to Compensate for Position Bias? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies reveal that position bias in large language models (LLMs) leads to difficulty in accessing information retrieved from the retriever. |
| Approach: | They propose to direct LLMs to allocate more attention towards a selected segment of the context through prompting. |
| Outcome: | The proposed approach improves the performance of large language models by promoting instruction with an exact document index. |
Instruction Position Matters in Sequence Generation with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can perform conditional sequence generation tasks, such as translation or summarization, through instruction fine-tuning. |
| Approach: | They propose to shift the position of task instructions after the input sentences to enhance the model's instruction-following capability. |
| Outcome: | The proposed method outperforms traditional settings across various model scales (1B / 7B & 13B) and different sequence generation tasks (translation and summarization) without any additional data or annotation costs. |
Computation Mechanism Behind LLM Position Generalization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have explored how LLMs handle positional relevance, but how they handle it remains unexplored. |
| Approach: | They propose to enforce certain computational mechanisms to allow for the tolerance in position perturbations in large language models (LLMs) they also find a pattern in intermediate features that allows this effect to be observed . |
| Outcome: | The proposed models can understand text with position perturbations and generalize to longer sequences than those seen during training with the latest techniques. |
Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle to meet user’s needs when required to generate responses of a specific length due to their inherent difficulty in accurately perceiving numerical constraints. |
| Approach: | They propose a Target Length Generation Task and propose RULER, a model-agnostic approach that controls generated length for large language models. |
| Outcome: | The proposed model-agnostic approach improves instruction-following ability of large language models under length-constrained instructions and can generate appropriate MLT when length constraints are not explicitly provided. |
Working Memory Identifies Reasoning Limits in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using large language models, we examine the limitations of their cognitive capabilities and their working memory. |
| Approach: | They examine the limitations of large language models from a scaling perspective . they also assess various prompting strategies, revealing their diverse impacts on LLM performance. |
| Outcome: | The proposed models perform poorly on n-back tasks and on prompting strategies. |