Semantic (vector) search catches meaning-based matches even with no shared vocabulary, but can miss exact terms — e.g., if a user asks about "invoice #4471," a pure embedding search might retrieve semantically-similar-sounding invoices instead of the exact one, because embeddings blur precise identifiers.
Keyword search (BM25) is the classical statistical text-matching algorithm (a refinement of TF-IDF) that scores documents based on exact term overlap, weighted by how rare/informative each term is. This is exactly what catches "invoice #4471" reliably — exact strings, numbers, codes, names.
Entity/graph matching boosts results that are connected, in the graph, to entities mentioned in the query — catching relational relevance that neither of the above two directly captures.
from rank_bm25 import BM25Okapi
documents = [
"User's invoice number is 4471, paid in full",
"User's favorite color is blue",
"User mentioned an allergy to peanuts",
]
tokenized_docs = [doc.lower().split() for doc in documents]
bm25 = BM25Okapi(tokenized_docs)
query = "invoice 4471".lower().split()
scores = bm25.get_scores(query)
print(scores) # the invoice doc should score highest
A simple, practical fusion approach — normalize each signal's scores to 0-1, then combine with weights:
def normalize(scores):
scores = np.array(scores)
if scores.max() == scores.min():
return np.zeros_like(scores)
return (scores - scores.min()) / (scores.max() - scores.min())
def hybrid_score(semantic_scores, bm25_scores, weight_semantic=0.6, weight_bm25=0.4):
sem_norm = normalize(semantic_scores)
bm25_norm = normalize(bm25_scores)
return weight_semantic * sem_norm + weight_bm25 * bm25_norm
More sophisticated production systems use Reciprocal Rank Fusion (RRF) instead of raw score averaging, since raw scores from different systems (cosine similarity vs. BM25) aren't on comparable scales — RRF instead combines based on each document's rank in each system's results, which is more robust.
Beyond just "is this relevant," retrieval needs to weight when a memory applies relative to the query's intent:
- Query implies "now" ("what's the user's current employer") → rank the most recent matching fact highest, potentially exclude older contradicting facts entirely
- Query implies history ("what did the user say last month") → rank by closeness to that specific time window, not recency
This requires your extraction pipeline (Module 5/6) to actually timestamp facts and, ideally, track validity ranges (Module 6's temporal graph concept) — retrieval quality on temporal questions is capped by whether your storage layer preserved that information in the first place.
- Build a small corpus of 15 facts including at least 3 with exact identifiers (numbers, codes, names) and run the same query through pure semantic search vs. pure BM25 vs. a simple fused combination — find a query where fusion clearly wins over either alone.
- Add timestamps to your facts, write a query that should prefer the most recent fact, and verify your fusion function (extended with a recency-boost term) actually surfaces the newest one first.