feat: add SemanticTextSplitter using EmbeddingModel cosine similarity

[anuragg-saxenaa]

Summary

Add SemanticTextSplitter — a new TextSplitter that uses semantic similarity rather than fixed token counts to determine chunk boundaries.

Closes #5464

Motivation

TokenTextSplitter splits at fixed token counts, often breaking sentences mid-thought and degrading RAG retrieval quality. Users frequently reach for external tools (e.g., Docling) or write custom solutions. A native semantic splitter addresses this gap without introducing new external dependencies.

Implementation

SemanticTextSplitter extends the existing TextSplitter base class:

1. Sentence tokenisation — split on ., !, ? followed by whitespace.
2. Embedding — embed each sentence via the injected EmbeddingModel.
3. Cosine similarity — compute similarity between consecutive sentence embeddings.
4. Chunking — start a new chunk when similarity drops below similarityThreshold or the buffer would exceed maxChunkSize characters.

Defaults: similarityThreshold = 0.5, maxChunkSize = 1000.
No new dependencies — reuses Spring AI's own EmbeddingModel.

Example

SemanticTextSplitter splitter = SemanticTextSplitter.builder()
    .embeddingModel(embeddingModel)
    .similarityThreshold(0.6)
    .maxChunkSize(800)
    .build();

List<Document> chunks = splitter.split(document);

Tests (SemanticTextSplitterTests)

Test	What it verifies
emptyTextReturnsEmptyList	blank/empty input
singleSentenceReturnsSingleChunk	single-sentence edge case
highSimilarityKeepsSentencesTogether	identical embeddings → one chunk
lowSimilaritySplitsSentences	orthogonal embeddings → separate chunks
maxChunkSizeForcesNewChunk	size cap overrides similarity
thresholdExactlyAtSimilarityKeepsTogether	boundary: sim >= threshold
thresholdJustAboveSimilaritySplits	boundary: sim < threshold
splitDocumentPreservesMetadata	metadata propagation
cosineSimilarity* (×3)	math helper correctness
Constructor validation (×4)	null model, invalid threshold/size