fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (#1090)

Opening chroma.sqlite3 via Python's sqlite3.connect() against a live
ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next
PersistentClient call — the same failure mode tracked at #1090.

_fix_blob_seq_ids runs unconditionally on every make_client() call, so
every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite
open → corrupt → segfault cycle on palaces that have already completed
the 0.6.x → 1.5.x seq_id migration.

Guard with a .blob_seq_ids_migrated marker file in the palace directory:
- If marker exists, return immediately — skip sqlite entirely
- After successful migration (or confirmation that no BLOBs remain),
  write the marker so subsequent opens take the fast path
- Palaces that never had BLOB seq_ids also get the marker on first open,
  so they too avoid the redundant sqlite open after that
- Already-migrated palaces can touch the marker manually to opt in

Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh
palace; second call returns immediately because marker exists. 1094
existing tests pass.
This commit is contained in:
jp
2026-04-24 13:35:07 -07:00
parent 0d9929c0dd
commit bc24aa14e2
+20
View File
@@ -159,6 +159,9 @@ def _pin_hnsw_threads(collection) -> None:
logger.debug("_pin_hnsw_threads modify failed", exc_info=True) logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
def _fix_blob_seq_ids(palace_path: str) -> None: def _fix_blob_seq_ids(palace_path: str) -> None:
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER. """Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
@@ -168,10 +171,19 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
type INTEGER) is not compatible with SQL type BLOB". type INTEGER) is not compatible with SQL type BLOB".
Must run BEFORE PersistentClient is created (the compactor fires on init). Must run BEFORE PersistentClient is created (the compactor fires on init).
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
database leaves state that segfaults the next PersistentClient call. After
the migration has run once successfully, a marker file is written so
subsequent opens skip the sqlite connection entirely. Already-migrated
palaces can touch the marker manually to opt into the fast path.
""" """
db_path = os.path.join(palace_path, "chroma.sqlite3") db_path = os.path.join(palace_path, "chroma.sqlite3")
if not os.path.isfile(db_path): if not os.path.isfile(db_path):
return return
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
if os.path.isfile(marker):
return
try: try:
with sqlite3.connect(db_path) as conn: with sqlite3.connect(db_path) as conn:
for table in ("embeddings", "max_seq_id"): for table in ("embeddings", "max_seq_id"):
@@ -189,6 +201,14 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
conn.commit() conn.commit()
except Exception: except Exception:
logger.exception("Could not fix BLOB seq_ids in %s", db_path) logger.exception("Could not fix BLOB seq_ids in %s", db_path)
return
# Write marker whether or not rows needed migration — the palace is now
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
# sqlite3.connect() entirely.
try:
open(marker, "a").close()
except OSError:
logger.exception("Could not write migration marker %s", marker)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------