GraphGuru | Causal Consistency in Neo4j Clusters: Bookmarks, Routing, and Retries with the v6 Drivers

Every Neo4j cluster question we get asked in a health check eventually reduces to the same one: "we wrote a node, then immediately read it back from another service, and it wasn't there — is the cluster broken?" The cluster is almost never broken. What is broken is an assumption: that a cluster behaves like a single server. It does not, and the driver gives you exact tools to get the guarantees you actually need. This tutorial covers those tools — routing, bookmarks, managed transactions, and retry behaviour — with the v6 driver API, and shows the three failure modes that put read-your-own-writes bugs into production.

Prerequisites

A Neo4j cluster, or a single instance for the code (the patterns are identical; only the observable failures differ). The Kubernetes autonomous clustering tutorial sets up a real one.
A v6 driver. pip install "neo4j>=6" for Python, npm i neo4j-driver@6 for JavaScript, or the 6.x Java artifact.
A neo4j:// connection URI. If yours starts with bolt://, read the next section carefully, because that is failure mode number one.

What the cluster actually guarantees

A Neo4j autonomous cluster replicates writes from a primary to other primaries and to secondaries. A write transaction commits when a quorum of primaries has it. Replication to the remaining members — and to every secondary — happens asynchronously, shortly afterwards. So immediately after a successful commit there exist cluster members that have not yet applied your transaction.

The default consistency model is therefore eventual across sessions, with one important exception: within a single session, the driver tracks a bookmark — an opaque marker of the last transaction that session saw — and sends it with the next transaction. The server holding the next transaction waits until it has caught up to that bookmark before running your query. That is what makes read-your-own-writes work inside one session, and it is the mechanism you extend when you need the guarantee across services.

Three consequences to internalise:

Causal consistency is per-session, not per-cluster. A new session gets no bookmarks and may land on a lagging member.
Waiting has a cost. A bookmark on a heavily lagged secondary means your read blocks until it catches up, or times out.
You can choose not to wait. Dashboards and analytics usually should not; a "did my save work?" screen must.

Failure mode 1: `bolt://` instead of `neo4j://`

# WRONG on a cluster: pins every query to one member, no routing, no failover
driver = GraphDatabase.driver("bolt://core-1.neo4j.svc:7687", auth=auth)

# RIGHT: routing driver, discovers members, routes reads and writes
driver = GraphDatabase.driver("neo4j://neo4j.svc:7687", auth=auth)

The bolt:// scheme opens a direct connection to exactly the host you named. Every read and write goes there. When that member is the one being restarted during a rolling upgrade, your application goes down even though the cluster is healthy. neo4j:// asks the cluster for a routing table, refreshes it periodically, sends writes to a primary and reads to whichever member is appropriate, and survives members coming and going.

Use bolt:// deliberately and rarely: admin tooling that must talk to a specific instance, or a single-instance dev database where it makes no difference. Grep your codebase and your Helm values for bolt:// today; on most engagements we find at least one.

Failure mode 2: hand-rolled transactions instead of managed ones

# Fragile: one transient network blip and this raises to your user
with driver.session() as session:
    tx = session.begin_transaction()
    tx.run("MATCH (a:Account {id: $id}) SET a.tier = $tier", id=account_id, tier="gold")
    tx.commit()

# Resilient: retried automatically on transient errors (leader switch,
# deadlock, connectivity), with exponential backoff
def set_tier(tx, account_id, tier):
    tx.run("MATCH (a:Account {id: $id}) SET a.tier = $tier",
           id=account_id, tier=tier)

with driver.session() as session:
    session.execute_write(set_tier, account_id, "gold")

execute_write / execute_read (the v5+/v6 names for what used to be write_transaction / read_transaction) take a function and may call it more than once. That is the whole point, and it imposes one rule: the function must be idempotent and must contain no side effects outside the transaction. No HTTP calls, no Kafka publishes, no print you rely on, no incrementing an in-memory counter. Write Cypher that is safe to re-run — MERGE rather than CREATE where identity matters, absolute SET rather than SET x = x + 1.

For a single query with no client-side logic, v6's execute_query is shorter and still managed:

records, summary, keys = driver.execute_query(
    "MERGE (a:Account {id: $id}) SET a.tier = $tier RETURN a.id AS id",
    id=account_id, tier="gold",
    routing_=RoutingControl.WRITE,
    database_="neo4j",
)

Note routing_=RoutingControl.WRITE. execute_query defaults to write routing; pass RoutingControl.READ for read-only queries so they can be served by secondaries. Getting this wrong is the quietest scaling bug in the API: a read-heavy service that sends everything to the primary and then wonders why adding secondaries did nothing.

Failure mode 3: read-your-own-writes across two sessions

This is the bug from the opening paragraph. Service A writes; service B reads; the read misses.

# Reproduces the bug
with driver.session() as s1:
    s1.execute_write(lambda tx: tx.run(
        "CREATE (o:Order {id: $id, status: 'new'})", id=order_id))

with driver.session() as s2:                     # fresh session = no bookmarks
    rec = s2.execute_read(lambda tx: tx.run(
        "MATCH (o:Order {id: $id}) RETURN o.status AS status",
        id=order_id).single())
    print(rec)                                   # may be None on a cluster

The fix is to carry the bookmarks from the writing session into the reading one.

Same process

with driver.session() as s1:
    s1.execute_write(create_order, order_id)
    bookmarks = s1.last_bookmarks()

with driver.session(bookmarks=bookmarks) as s2:
    rec = s2.execute_read(read_order, order_id)   # guaranteed to see the order

Across services

Bookmarks are serialisable strings. Pass them where you already pass correlation IDs — an HTTP header, a message attribute, the job payload:

# producer
values = list(session.last_bookmarks().raw_values)
publish({"orderId": order_id, "neo4jBookmarks": values})

# consumer
from neo4j import Bookmarks
bm = Bookmarks.from_raw_values(msg["neo4jBookmarks"])
with driver.session(bookmarks=bm) as session:
    ...

Treat them as opaque. Do not parse, compare, or persist them long-term — a stale bookmark makes a member wait for a transaction that is long since consolidated, and you gain nothing but latency.

When you cannot thread them through

Use a BookmarkManager to share bookmarks across all sessions from one driver. It trades some latency for a much simpler contract, and it is the right default for a small service where every read should see every write:

from neo4j.api import bookmark_manager

bmm = bookmark_manager()
driver = GraphDatabase.driver(uri, auth=auth, bookmark_manager=bmm)
# execute_query uses the driver's bookmark manager automatically

Deliberately reading stale data

Not every read deserves to wait. A dashboard aggregating six months of orders does not care about the order committed 40 milliseconds ago, and making it care costs you the benefit of your secondaries.

with driver.session(
    default_access_mode=READ_ACCESS,
    bookmarks=None,                 # explicitly no causal guarantee
    database="neo4j",
) as session:
    session.execute_read(dashboard_query)

If your cluster has secondaries dedicated to analytics, target them with server-side routing policies rather than hostnames, so the topology stays an operations concern rather than an application one.

Measuring replication lag before it bites

Bookmark waits are invisible in your application metrics — they look like slow queries. Watch the cluster instead:

SHOW SERVERS YIELD name, address, state, health, hosting;

SHOW DATABASES YIELD name, serverID, currentStatus, requestedStatus,
     lastCommittedTxn, replicationLag
WHERE name = 'neo4j';

replicationLag is the field to alert on. Scrape it alongside the metrics from our observability tutorial and page when a member's lag exceeds the window your bookmark-bearing reads can tolerate. A secondary that has drifted minutes behind will turn every causal read routed to it into a timeout, and the stack trace will point at your query, not at the replica.

A checklist you can apply this week

Every application URI uses neo4j://; bolt:// appears only in admin tooling.
Every transaction goes through execute_read, execute_write, or execute_query.
Every transaction function is idempotent and side-effect-free.
Read-only queries are declared read-only, so secondaries can serve them.
Every write-then-read flow that crosses a session boundary either passes bookmarks or has a documented reason it tolerates stale reads.
replicationLag is scraped and alerted on.
Your integration tests run against a cluster, or at least assert that transaction functions are re-runnable. Testcontainers makes the second part cheap.

Most "the cluster lost my data" incidents we are called into are one of the three failure modes above, and all three are fixed in application code rather than in the database. If you would rather have someone audit your driver layer and cluster topology together, that is a standard part of our Neo4j consulting work — get in touch.

Causal Consistency in Neo4j Clusters: Bookmarks, Routing, and Retries with the v6 Drivers