AI Agent Memory Systems

Given two vectors A and B, cosine similarity measures the angle between them:

cosine_similarity(A, B) = (A · B) / (||A|| × ||B||)

Where A · B is the dot product and ||A|| is the magnitude (length) of vector A. The result ranges from -1 (opposite meaning) to 1 (identical meaning), with 0 meaning unrelated.

import numpy as np

def cosine_similarity(a, b):
    a, b = np.array(a), np.array(b)
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

You almost never need to write this yourself in production (vector DBs do it internally, optimized), but you should understand it since it's the single most-used operation in this entire field.

from openai import OpenAI
client = OpenAI()

def embed(text):
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    return response.data[0].embedding

sentences = [
    "I love my dog",
    "My pet means the world to me",
    "The stock market crashed today",
    "Tech stocks fell sharply this morning"
]

vectors = [embed(s) for s in sentences]

for i in range(len(sentences)):
    for j in range(i+1, len(sentences)):
        sim = cosine_similarity(vectors[i], vectors[j])
        print(f"{sentences[i]!r} vs {sentences[j]!r}: {sim:.3f}")

Run this — you'll see the "dog"/"pet" pair score high (likely 0.6-0.8+) and the "dog" vs "stocks" pairs score noticeably lower. This confirms embeddings capture meaning, not just word overlap.

Once you have thousands or millions of embeddings, computing cosine similarity against every single one for every query (brute force) becomes too slow. Vector databases use approximate nearest neighbor (ANN) algorithms — most commonly HNSW (Hierarchical Navigable Small World graphs) — to find the top-k closest vectors in roughly logarithmic time instead of linear time, trading a tiny bit of accuracy for massive speed gains.

Common choices:

• Chroma — simplest, runs locally, zero setup, great for learning/prototyping
• pgvector — a Postgres extension, good if you already run Postgres and don't want a separate DB
• Qdrant / Weaviate / Pinecone — purpose-built, better at scale (millions+ vectors), managed hosting options

If you embed an entire 10-page document as one vector, that vector becomes a blurry "average" of everything in the document — a query about page 7 might not match well because the vector is diluted by pages 1-6 and 8-10. Chunking splits documents into smaller pieces (commonly 200-500 tokens, sometimes with overlap between chunks so you don't cut a sentence in half at a boundary) before embedding each piece separately.

def chunk_text(text, chunk_size=300, overlap=50):
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        end = start + chunk_size
        chunks.append(" ".join(words[start:end]))
        start += chunk_size - overlap
    return chunks