Getting Started: Text Splitters
Language Models are often limited by the amount of text that you can pass to them. Therefore, it is neccessary to split them up into smaller chunks. LangChain provides several utilities for doing so.
Using a Text Splitter can also help improve the results from vector store searches, as eg. smaller chunks may sometimes be more likely to match a query. Testing different chunk sizes (and chunk overlap) is a worthwhile exercise to tailor the results to your use case.
All Text Splitters
📄️ Examples
Splitters are components or tools used to divide texts into smaller, more manageable parts or specific segments. This division can be necessary for various reasons, such as improving the processing, analysis, or understanding of large or complex texts. Splitters can be simple, like dividing a text into sentences or paragraphs, or more complex, such as splitting based on themes, topics, or specific grammatical structures.