chunk-lite: token-aware text chunking for RAG pipelines that doesn't cut mid-sen

<p>Most RAG retrieval problems I've debugged came down to the same thing: someone swapped the embedding model three times, added a reranker, then gave up — and never once changed

arXiv:2604.04936v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems critically depend on effective document chunking strategies to balance retrieval

<p>Chunk clean article content for embeddings, summarization, and full-text search—skip nav, clap bars, and scripts.</p> <h1> Extract Plain Text from Medium Posts for RAG and

<p><strong>Sliding window chunking</strong><br /> To understand this method, we need to know about two parameters, <strong>window size</strong> and <strong>step size</strong>.