rag-cli

rag-cli is a Python command-line experiment in semantic retrieval. It loads movie titles and descriptions, embeds them locally with all-MiniLM-L6-v2 and ranks them against a query using cosine similarity. The output is a list of matching titles, scores and description previews.

The repository name describes the direction of the project. The current implementation stops at retrieval: it does not pass results to a language model or generate an answer.

What the search command runs

cli.py uses argparse to expose commands for loading the model, inspecting embeddings, searching, displaying chunks and building chunk embeddings. search takes a query and an optional --limit, which defaults to five.

The command constructs a SemanticSearch instance and loads the movie dataset. For each record, the document text is assembled as title: description, then encoded with Sentence Transformers. Embedding both fields gives the model the movie's name as well as its synopsis.

The query uses the same model. Each document vector is compared with the query vector, and the scores are sorted in descending order. The CLI prints the requested number of results, truncating each description to its first 100 characters. It does not return a matching passage or explain why a title ranked above another.

The loader expects data/movies.json to contain a movies array, with an ID, title and description for each record. That dataset is not included in the reviewed checkout. Before running the search, it needs to be supplied locally, along with a writable cache directory; the code does not create that directory.

Scoring without a vector database

The cosine similarity helper divides the vectors' dot product by the product of their norms. If either norm is zero, it returns a score of zero.

Search evaluates that function once per document in a Python loop, then sorts all scores. There is no approximate nearest-neighbor index or vector database. For a small dataset, this keeps the scoring path easy to inspect. As the dataset grows, every query still scans every document, and sorting costs grow with the number of records.

Caching avoids encoding the corpus again, but each CLI invocation still loads the embedding model and encodes its query. The repository contains no benchmark that establishes sub-second search latency. Cold startup, cached search and initial indexing would need separate measurements.

There is also an edge case in result selection: search indexes the sorted list for every position up to limit. Asking for more results than the dataset contains raises an indexing error. Clamping the limit to the available result count would make that boundary explicit.

A cache can load successfully and still be wrong

SemanticSearch saves the document matrix to cache/movie_embeddings.npy. When it finds that file, it accepts the cached matrix if its row count matches the number of current documents.

That check catches a change in dataset size. It does not catch edited descriptions, a new record order or a different embedding model. Search pairs rows with documents using zip, so reordering the dataset can associate a movie with another movie's vector while the cache still passes validation.

A cache key based on the ordered document contents and model identity would address those cases. I would store that identity alongside the matrix and rebuild when it changes. The current cache paths are also relative to the working directory, while the dataset path is resolved from the source tree. Running the command from a different directory therefore changes where it looks for cached vectors.

Sentence windows are a separate experiment

ChunkedSemanticSearch extends the document search class with chunk embeddings and metadata files. Its builder splits each nonempty description at punctuation followed by whitespace, groups four sentences per chunk and overlaps adjacent windows by one sentence. The stride is three sentences.

For example, seven sentences produce two windows:

Chunk 1: sentences 1, 2, 3, 4
Chunk 2: sentences 4, 5, 6, 7

The shared sentence gives each window some context from its neighbor. This is rule-based splitting, despite the function's semantic_chunk name; it does not use embeddings to decide where a topic changes.

The standalone chunk command calls this sentence splitter too. Its argument help describes word counts, but --max-chunk-size and --overlap count sentences on that path. Both the sentence and word overlap helpers reject an overlap larger than the window, yet allow equality, which produces a zero stride and fails when constructing the range.

The chunk builder writes vectors to cache/chunk_embeddings.npy and metadata to cache/chunk_metadata.json. Each metadata entry tracks the movie ID and chunk index. Its total_chunks field currently uses the length of the chunk string, so it records a character count rather than the number of chunks for that movie.

The metadata also changes shape across runs: the builder keeps a list in memory, while the cache loader assigns the entire JSON wrapper to the same attribute. Before using those records in retrieval, I would make both paths return the same structure and validate it against the vector matrix.

Connecting chunks to retrieval

embed_chunks builds or loads chunk vectors, but search always constructs the base SemanticSearch class. No code scores chunks, groups their scores by movie or combines them with document scores. Max-pooling, hybrid retrieval and Reciprocal Rank Fusion are not implemented in this revision. The dependency file also contains no LangChain or Groq integration for rewriting a query.

A useful next experiment would score the chunks and return the best matching passage with each movie. I would compare that against the current document ranking on a fixed set of queries before choosing an aggregation strategy. Keeping the passage in the output would make a poor match inspectable, rather than leaving only a title and a similarity score to debug.