Papers with chunking
DadmaTools: Natural Language Processing Toolkit for Persian Language (2022.naacl-demo)
Copied to clipboard
| Challenge: | Existing tools for Persian language processing are based on conventional non-neural models and do not take full advantage of the latest developments. |
| Approach: | They propose to use a Python neural pipeline for Persian text processing tasks . they use 'parsBERT' to fine-tune the Python pipeline using the PerDT dataset . |
| Outcome: | The proposed toolkit can achieve state-of-the-art performance on multiple NLP tasks. |
Retrieval Enhancements for RAG: Insights from a Deployed Customer Support Chatbot (2026.eacl-industry)
Copied to clipboard
Daniel González Juclà, Mohit Tuteja, Marcos Esteve Casademunt, Keshav Unnikrishnan, Yasir Usmani, Arvind Roshaan
| Challenge: | a persistent gap remains between Recall@10 and Recall @50 across datasets . |
| Approach: | They evaluate embedding model comparison, Reciprocal Rank Fusion and embedded concatenation techniques to improve retrieval quality. |
| Outcome: | The proposed methods outperform traditional cross-encoders in identifying high-relevance passages. |
Sketching a Linguistically-Driven Reasoning Dialog Model for Social Talk (2022.acl-srw)
Copied to clipboard
| Challenge: | a new study shows that dialog systems that can hold social talk and make sense of conversational content are not efficient for context-sensitive natural language understanding and reasoning. |
| Approach: | They propose a linguistically-informed architecture to handle social talk in English . they propose linguistic models that fit the context-sensitive components into a Bayesian game-theoretic model . |
| Outcome: | The proposed architecture is based on corpus-based methods but does not track what is happening in a conversation. |
TreeRAG: Unleashing the Power of Hierarchical Storage for Enhanced Knowledge Retrieval in Long Documents (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional RAG frameworks struggle to retrieve all relevant knowledge points . a new approach to retrieve long documents is proposed to improve performance in NLP . |
| Approach: | They propose a tree-based approach to document knowledge retrieval that preserves hierarchical structure . treeRAG is a key technique for enhancing the text generation capabilities of Large Language Models . |
| Outcome: | The proposed approach improves recall quality and precision compared to existing methods and better performance to question-answering tasks. |
Enhance Robustness of Sequence Labelling with Masked Adversarial Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Adversarial training (AT) has shown strong regularization effects on deep learning algorithms by introducing small input perturbations to improve model robustness. |
| Approach: | They propose to use adversarial training to improve robustness from contextual information in sequence labelling tasks by masking or replacing some words in the sentence. |
| Outcome: | The proposed method shows significant improvements on accuracy and robustness of sequence labelling on CoNLL 2000 and 2003 benchmarks. |
Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data? (C18-1)
Copied to clipboard
| Challenge: | Existing neural models take long distance dependencies into account when predicting the tag of the current token. |
| Approach: | They propose a method to capture long distance tag dependencies and use them for dependency analysis. |
| Outcome: | The proposed model can predict multiple tags for the current token without taking dependencies between tags into account. |
SAKI-RAG: Mitigating Context Fragmentation in Long-Document RAG via Sentence-level Attention Knowledge Integration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Traditional Retrieval-Augmented Generation (RAG) frameworks segment documents into larger chunks to preserve contextual coherence . however, such chunking methods lead to fragmented contexts, isolated chunk semantics, and broken inter-chunk relationships . |
| Approach: | They propose a framework that maintains granular chunks while recovering their intrinsic semantic connections. |
| Outcome: | The proposed framework achieves better recall and precision compared to other RAG frameworks in long-document retrieval scenarios. |
Late Code Chunking: A Code Chunking Strategy for Repository-Level Code Completion (2026.acl-short)
Copied to clipboard
| Challenge: | Despite significant advancements in Large Language Models (LLMs), repository-level code completion remains a challenging area. |
| Approach: | They propose a chunking strategy to improve the semantic understanding of code segments for Large Language Models. |
| Outcome: | The proposed strategy improves the semantic understanding of code segments for Large Language Models. |
AutoChunker: Structured Text Chunking and its Evaluation (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for text chunking struggle with document structure and noise . Existing approaches struggle with maintaining semantic coherence while handling complex documents. |
| Approach: | They propose a bottom-up approach to chunking that combines document structure awareness with noise elimination. |
| Outcome: | The proposed method outperforms existing methods in noise reduction, completeness, context coherence, task relevance, and retrieval performance. |
PharmaQA.IT: an Italian dataset for Q&A in the pharmaceutical domain (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing medical QA datasets are mostly English and centred on scientific articles or clinical notes. |
| Approach: | They propose an extractive QA dataset built from Riassunti delle Caratteristiche del Prodotto . the final dataset contains 861 high-quality question–answer pairs . |
| Outcome: | The proposed dataset contains 861 high-quality question–answer pairs on indications, contraindications, dosage, warnings, interactions, and pharmacological properties. |
Grounding Language Model with Chunking-Free In-Context Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | CFIC retrieval approach eliminates the need for document chunking and provides a more efficient and efficient method for RAG systems. |
| Approach: | They propose a Chunking-Free In-Context retrieval approach specifically tailored for RAG systems . they employ auto-aggressive decoding to accurately identify specific evidence text . |
| Outcome: | The proposed method is better than traditional methods on open question answering datasets. |
Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking (2024.eacl-long)
Copied to clipboard
| Challenge: | Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates. |
| Approach: | They propose a framework to integrate Chinese geographic semantics into re-ranking pipelines. |
| Outcome: | The proposed framework improves on two Chinese benchmark datasets. |
TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG (2025.emnlp-industry)
Copied to clipboard
Savini Kashmira, Jayanaka L. Dantanarayana, Joshua Brodsky, Ashish Mahendra, Yiping Kang, Krisztian Flautner, Lingjia Tang, Jason Mars
| Challenge: | Retrieval-Augmented Generation (RAG) relies on query-chunk text-to-text similarity in the embedding space for retrieval, can fail to capture deeper semantic relationships across chunks, is highly sensitive to chunking strategies, and is prone to hallucinations. |
| Approach: | They propose a graph-based retrieval framework that first constructs the knowledge graph from unstructured data dynamically and automatically. |
| Outcome: | The proposed framework outperforms multiple RAG implementations in both precision and recall, significantly enhancing user experience through improved retrieval accuracy. |
Is Semantic Chunking Worth the Computational Cost? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Retrieval-Augmented Generation (RAG) systems have popularized semantic chunking. |
| Approach: | They evaluate the effectiveness of semantic chunking using three common retrieval tasks . they find that the computational costs associated with semantic chunks are not justified by consistent performance gains. |
| Outcome: | The proposed semantic chunking approach is not able to deliver consistent performance gains in three retrieval-related tasks. |
Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement (2026.findings-eacl)
Copied to clipboard
| Challenge: | Current approaches focus on specific MWE types, such as transformer-based models that incorporate linguistic features like dependency parsing for verbal discontinuous patterns. |
| Approach: | They propose a binary token-level classification approach that integrates linguistic feature integration and data augmentation to improve multiword expression (MWE) identification. |
| Outcome: | The proposed model outperforms the Qwen-72B model on the CoAM dataset by 12 points while using 165 times fewer parameters. |
Fine-Grained Error Analysis and Fair Evaluation of Labeled Spans (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotations with incorrect label or boundaries count as two errors instead of one, despite being closer to the target annotation than false positives or false negatives. |
| Approach: | They propose an algorithm for error identification in flat and multi-level annotations and propose a procedure for calculating meaningful precision, recall, and F1-scores based on the more fine-grained error types. |
| Outcome: | The proposed procedure prevents double penalties and allows for a more detailed error analysis, providing more insight into the actual weaknesses of a system. |
MC-indexing: Effective Long Document Retrieval via Multi-view Content-aware Indexing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document question answering do not consider content structures, resulting chunks exclude vital information or include irrelevant content. |
| Approach: | They propose a method that segments document into content chunks and represents each content chunk in raw-text, keywords, and summary views. |
| Outcome: | The proposed method significantly improves recall of long document question answering datasets compared to state-of-the-art chunking schemes. |
Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts (2025.emnlp-industry)
Copied to clipboard
Anwesan Pal, Karen Hovsepian, Tinghao Guo, Mengnan Zhao, Somendra Tripathi, Nikos Kanakaris, George Mihaila, Sumit Nigam
| Challenge: | Recent studies into effective context lengths of flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. |
| Approach: | They propose a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios without degrading and altering the integrity and composition of retrieved documents. |
| Outcome: | The proposed strategy boosts performance in long-context scenarios without degrading and altering the integrity and composition of retrieved documents. |
SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing RAG performance rely on heuristic-based augmentation . Existing approaches rely heavily on a heuriistic-driven approach, resulting in poor generalization and skews in the evidence length. |
| Approach: | They propose a model-based evidence extraction learning framework that optimizes a vanilla model as an evidence extractor with desired properties through self-aligned learning. |
| Outcome: | The proposed method reduces the evidence length by 9.25 times and improves reliability and reliability. |
Does Chinese BERT Encode Word Structure? (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work has focused on analyzing the features captured by representative models such as BERT . however, little work has investigated word features for character languages such as Chinese . |
| Approach: | They investigate Chinese BERT using attention weight distribution statistics and probing tasks to understand word features. |
| Outcome: | The proposed model improves syntactic, semantic and word sense knowledge on a wide range of NLP tasks. |
MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text chunking are limited by text chunks and lack of domain-specific knowledge. |
| Approach: | They propose a dual-metric evaluation method to quantify text chunking quality . they aim to generate a structured list of chunking regular expressions . |
| Outcome: | The proposed method enables direct quantification of chunking quality . it substantiates the need to integrate LLMs into chunking process . |
Abstractive Summarization of Bengali Academic Videos Based on Audio Subtitles (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for summarizing educational videos in Bengali are limited due to the rapid growth of educational video content. |
| Approach: | They propose an end-to-end pipeline for the abstractive summarization of Bengali videos . they fine-tuned the BanglaT5 model on a new benchmark dataset . |
| Outcome: | The proposed system preprocesses audio and converts speech to text using Google's Speech Recognition API. |
LumberChunker: Long-Form Narrative Document Segmentation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Modern NLP tasks rely on dense retrieval methods to access up-to-date and relevant contextual information. |
| Approach: | They propose a method that leverages an LLM to dynamically segment documents by iterating on a set of sequential passages to identify the point where the content begins to shift. |
| Outcome: | The proposed method outperforms the most competitive baseline by 7.37% in retrieval performance and integrates into a RAG pipeline. |
Large Language Models Are No Longer Shallow Parsers (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have reshaped the field of natural language processing (NLP) however, fundamental NLP tasks that involve linguistic analysis still play essential roles in the field. |
| Approach: | They propose to use constituency parsing to improve performance of LLMs on deep syntactic parse trees to prompt LLM chunking, filter out low-quality chunks and add remaining chunks to prompts to instruct LLM for parser. |
| Outcome: | The proposed approach improves LLMs' performance on constituency parsing on English and Chinese benchmark datasets. |
Dissecting Span Identification Tasks with Performance Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Span identification tasks are a staple of applied NLP, but there is little insight on how their properties influence their difficulty. |
| Approach: | They propose to build a model to predict span ID performance for unseen span ID tasks that can support architecture choices. |
| Outcome: | The proposed model predicts span ID tasks for unseen span ID task in English, and the meta model predictable span ID performance. |
Document Segmentation Matters for Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
Zhitong Wang, Cheng Gao, Chaojun Xiao, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun
| Challenge: | Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence. |
| Approach: | They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary. |
| Outcome: | Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training. |
cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing line-based chunking heuristics often break semantic structures, splitting functions or merging unrelated code. |
| Approach: | They propose a structure-aware method that breaks large AST nodes into smaller chunks . this method generates self-contained, semantically coherent units across programming languages . |
| Outcome: | The proposed method boosts Recall@5 by 4.3 points on RepoEval retrieval and Pass@1 by 2.67 points on SWE-bench generation. |
A Web Service for Pre-segmenting Very Long Transcribed Speech Recordings (L18-1)
Copied to clipboard
| Challenge: | a new algorithm that pre-segments long speech recordings into manageable chunks is proposed . the run time of classical text-to-speech alignment algorithms is quadratically growing with the length of the input . |
| Approach: | They propose two algorithms that pre-segment long speech recordings into manageable chunks . first algorithm is fast but cannot guarantee short chunks on noisy recordings or erroneous transcriptions a second algorithm delivers short chunk but is less effective in terms of run time and chunk boundary accuracy . |
| Outcome: | The proposed algorithms reduce the run time of the speech segmentation system to under real-time even on recordings that could not previously be processed. |
Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Inductive biases play a critical role in NLP, especially in learning from limited data and generalizing systematically outside of the training distribution. |
| Approach: | They propose to strengthen the structural inductive bias of a Transformer by intermediate pre-training to perform syntactic transformations of dependency trees given a description of the transformation. |
| Outcome: | The proposed model can perform syntactic transformations and generalize semantic parsing with attention heads that keep track of which syntaktic transformation needs to be applied to which token. |
ChuLo: Chunk-Level Key Information Representation for Long Document Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional approaches to truncate inputs, sparse self-attention, and chunking often lead to information loss and hinder the model’s ability to capture long-range dependencies. |
| Approach: | They propose a novel chunk representation method that uses unsupervised keyphrase extraction to group input tokens to retain core document content while reducing input length. |
| Outcome: | The proposed method minimizes information loss and improves the efficiency of Transformer-based models. |
HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to document-based Opendomain Question Answering (ODQA) use flat text chunks or page-level images to locate the correct document. |
| Approach: | They propose a hierarchical tree-based multimodal retrieval framework that elevates document hierarchy to a first-class retrieval signal. |
| Outcome: | The proposed framework outperforms page- and chunk-based baselines on ODQA benchmarks and improves retrieval recall by 12.9% and end-to-end QA performance by 6.8%. |
Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new open-source layout-aware IE test suite is available for download at https://github.com/gayecolakoglu/layIE-LLM. |
| Approach: | They propose an open-source layout-aware IE test suite that provides a layout-based IE pipeline. |
| Outcome: | The proposed method achieves 13.3–37.5 F1 points more than a baseline configuration using the same LLM. |
HiChunk: Evaluating and Enhancing Retrieval Augmented Generation with Hierarchical Chunking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for document chunking are inadequate due to evidence sparsity . evaluators are unable to evaluate different chunking methods due to the evidence sparing . |
| Approach: | They propose a QA benchmark for document chunking and a hierarchical document structuring framework for it. |
| Outcome: | The proposed framework improves document chunking quality within reasonable time consumption. |