+1 (415) 649-9454
GraphGuru

Vector Search in Neo4j: From HNSW Index to the Native Vector Type

Neo4j has had vector search since the 5.11 HNSW index, and most GraphRAG tutorials still show the 2023 pattern: store the embedding as a list of floats, create a vector index, call db.index.vector.queryNodes. That still works. But since Neo4j 2025.10 there is a native Vector data type — embeddings as first-class typed values instead of LIST<FLOAT> — and it changes storage, validation, and how you write similarity code. This post walks from the index to the new type and the retrieval patterns we actually deploy.

The index: HNSW in one statement

CREATE VECTOR INDEX chunk_embeddings IF NOT EXISTS
FOR (c:Chunk) ON c.embedding
OPTIONS {
  indexConfig: {
    `vector.dimensions`: 1536,
    `vector.similarity_function`: 'cosine'
  }
};

Two choices live in that statement and both are hard to change later:

  • Dimensions must match the embedding model exactly. text-embedding-3-small is 1536; -large is 3072; open models vary. Changing models means re-embedding and rebuilding.
  • Similarity function: cosine for most text embeddings (they are usually normalised anyway), euclidean when the embedding model's documentation says so. Pick per model, not per taste.

HNSW tuning lives in the same indexConfig: vector.hnsw.m (graph connectivity, default 16) and vector.hnsw.ef_construction (build-time beam width, default 100). Higher values give better recall at the cost of build time and memory; we leave defaults unless an evaluation set says recall is the problem. The vector index docs list every option.

Querying:

CALL db.index.vector.queryNodes('chunk_embeddings', 10, $queryEmbedding)
YIELD node, score
RETURN node.text, score
ORDER BY score DESC;

The third argument is the query vector, the second the number of nearest neighbours (k). score is the similarity, already mapped so higher is better regardless of function.

The embedding pipeline

Wherever the embedding is computed — an application service, a neo4j-graphrag pipeline, a Spark job — the write is the same idempotent batch:

from neo4j import GraphDatabase
from openai import OpenAI

client = OpenAI()
driver = GraphDatabase.driver(URI, auth=AUTH)

def embed(texts: list[str]) -> list[list[float]]:
    resp = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return [d.embedding for d in resp.data]

def upsert_chunks(rows: list[dict]):
    vectors = embed([r["text"] for r in rows])
    for r, v in zip(rows, vectors):
        r["embedding"] = v
    driver.execute_query(
        """
        UNWIND $rows AS row
        MERGE (c:Chunk {id: row.id})
        SET c.text = row.text,
            c.embedding = row.embedding
        """,
        rows=rows,
    )

Batch by a few hundred chunks per embedding call and per transaction. Embed on content change only — hash the text and skip unchanged chunks — because re-embedding a corpus every night is the single most common way we see GenAI bills blow up.

The native Vector type (2025.10+)

Before 2025.10, c.embedding above is a LIST<FLOAT>: each element a 64-bit double, no dimensionality check, no type information. From 2025.10, with Cypher 25 and a 6.x driver, you can store a real vector:

CYPHER 25
UNWIND $rows AS row
MERGE (c:Chunk {id: row.id})
SET c.text = row.text,
    c.embedding = vector(row.embedding, 1536, FLOAT32);

vector(list, dimension, coordinateType) builds the typed value. What you get:

  • Half the storage. FLOAT32 coordinates are four bytes instead of eight. On a corpus of ten million chunks at 1536 dimensions, that is roughly 60 GB less on disk and in page cache. Smaller types (INTEGER8, INTEGER16) are available for quantised models.
  • Dimension validation at write time. A 3072-dimensional vector written to a 1536-dimensional property fails loudly, instead of silently sitting outside the index.
  • Typed similarity functions: vector.similarity.cosine(a, b) and vector.similarity.euclidean(a, b) for exact scoring between two vectors, which is what you need for re-ranking a candidate set from a graph traversal.
  • Driver support: the 6.x drivers return a vector value rather than a plain list, so application code needs the newer driver to read it back.

Migrating an existing property is a batched rewrite:

CYPHER 25
MATCH (c:Chunk)
WHERE c.embedding IS :: LIST<FLOAT>
CALL (c) {
  SET c.embedding = vector(c.embedding, 1536, FLOAT32)
} IN TRANSACTIONS OF 5000 ROWS;

Then drop and recreate the vector index so it is built over the typed values. Do this on a staging copy first and measure page-cache hit ratio before and after; the improvement is the business case.

Hybrid scoring patterns

Pure vector retrieval fails on exact tokens — an invoice number, a clause reference, a name the embedding model has never seen. The fix is to combine it with a fulltext index and, where the data is a graph, with traversal.

Vector + fulltext, merged

CALL () {
  CALL db.index.vector.queryNodes('chunk_embeddings', 20, $queryEmbedding)
  YIELD node, score
  RETURN node, score * $vectorWeight AS s
  UNION ALL
  CALL db.index.fulltext.queryNodes('chunk_text', $queryText)
  YIELD node, score
  RETURN node, score * $textWeight AS s
}
WITH node, sum(s) AS combined
RETURN node.text AS text, combined
ORDER BY combined DESC
LIMIT 10;

Fulltext scores (Lucene BM25) and vector scores live on different scales; normalise before weighting. In practice we rank each list, convert to reciprocal rank (1 / (60 + rank)), and sum — reciprocal rank fusion is robust and needs no tuning. The HybridRetriever in neo4j-graphrag does this for you.

Vector entry point, graph expansion, exact re-rank

CYPHER 25
CALL db.index.vector.queryNodes('chunk_embeddings', 10, $queryEmbedding)
YIELD node AS seed, score
MATCH (seed)-[:FROM_DOCUMENT]->(d:Document)<-[:FROM_DOCUMENT]-(neighbour:Chunk)
WHERE neighbour <> seed
WITH DISTINCT neighbour,
     vector.similarity.cosine(neighbour.embedding, vector($queryEmbedding, 1536, FLOAT32)) AS exact
RETURN neighbour.text, exact
ORDER BY exact DESC
LIMIT 10;

The index finds approximate entry points; the graph pulls in related chunks the index might not surface (same document, same entity, same author); the typed similarity function re-scores the expanded set exactly. This is the pattern that answers multi-hop questions, and it is only possible because the vectors live inside the graph.

Cost and recall tuning

  • k is a budget. Asking for 100 neighbours and keeping 10 costs more than asking for 20; measure recall at the k you use.
  • Recall is measured, not assumed. Build a set of 200 queries with exact-search ground truth (vector.similarity.cosine over every chunk, offline), and report recall@10 for your index settings. If it is under 0.9, raise ef_construction and rebuild before doing anything cleverer.
  • Page cache first. A vector index that does not fit in the page cache is slow regardless of parameters; the native type's smaller footprint is the cheapest win here.
  • Filter before, not after. Post-filtering 10 results by tenant_id leaves you with 2. Partition by label (:Chunk:TenantA) with an index per partition, or expand k and filter in the same query with a margin.
  • Re-rank the top 20 with a cross-encoder if answer quality matters more than latency; the graph expansion pattern already gives you the candidate set.

Where to go from here

If you are starting a new project in 2026 on a 2025.10+ server, use the native Vector type from day one; there is no reason to carry LIST<FLOAT> forward. If you are on 5.26 LTS, the index and hybrid patterns above are all available — only the type and its functions need the CalVer line. Either way, vector search is a component of retrieval, not the whole of it; the first GraphRAG pipeline tutorial shows the complete picture, and our GraphRAG consulting page describes how we help teams take it to production.