fix: best-effort HNSW thread-pin retrofit + drop dead attempt-cap constant

Addresses remaining PR #976 review items after rebase on develop.

`get_collection(create=False)` previously returned existing collections without
re-applying `hnsw:num_threads=1`, so palaces created before the fix kept the
unsafe parallel-insert path. Add `_pin_hnsw_threads()` helper that calls
`collection.modify(configuration=UpdateCollectionConfiguration(
hnsw=UpdateHNSWConfiguration(num_threads=1)))` best-effort on every
`get_collection` call (including the MCP server's `_get_collection`).

In chromadb 1.5.x the runtime config does not persist to disk across
`PersistentClient` reopens, so the retrofit is re-applied each process start
rather than being a one-shot migration. Fresh palaces keep the metadata-based
pin as primary defense; legacy palaces now also get per-session protection
without requiring `mempalace nuke` + re-mine.

After the rebase on develop, `hook_precompact` delegates to `_mine_sync` and
no longer emits `decision: block`, so the attempt-cap constant was orphaned.
Grep confirms 0 usages in the repo — remove it.

- `_pin_hnsw_threads` retrofits legacy collection (num_threads None -> 1)
- `_pin_hnsw_threads` swallows all errors (never raises)
- `ChromaBackend.get_collection(create=False)` applies retrofit on legacy palace
- 62 tests pass (10 backends + 6 palace locks + 46 hooks_cli)
This commit is contained in:
Felipe Truman
2026-04-17 20:04:37 -03:00
committed by Igor Lins e Silva
parent 40d7958ca1
commit 8df944a54d
4 changed files with 100 additions and 14 deletions
+32
View File
@@ -130,6 +130,37 @@ def quarantine_stale_hnsw(palace_path: str, stale_seconds: float = 3600.0) -> li
return moved
def _pin_hnsw_threads(collection) -> None:
"""Best-effort retrofit: pin ``hnsw:num_threads=1`` on an existing collection.
Fresh collections set this via ``metadata=`` at creation. Legacy palaces
built before that change keep the default (parallel insert) and can hit
the HNSW race described in #974/#965. ChromaDB's
``collection.modify(configuration=...)`` lets us re-apply ``num_threads=1``
in memory at load time so every new process is protected.
Note: in chromadb 1.5.x the modified ``configuration_json["hnsw"]`` does
not persist to disk across ``PersistentClient`` reopens, so this must
run on every ``get_collection`` call, not just once.
"""
try:
from chromadb.api.collection_configuration import (
UpdateCollectionConfiguration,
UpdateHNSWConfiguration,
)
except ImportError:
logger.debug("_pin_hnsw_threads skipped: chromadb too old", exc_info=True)
return
try:
collection.modify(
configuration=UpdateCollectionConfiguration(
hnsw=UpdateHNSWConfiguration(num_threads=1)
)
)
except Exception:
logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
def _fix_blob_seq_ids(palace_path: str) -> None:
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
@@ -572,6 +603,7 @@ class ChromaBackend(BaseBackend):
)
else:
collection = client.get_collection(collection_name, **ef_kwargs)
_pin_hnsw_threads(collection)
return ChromaCollection(collection)
def close_palace(self, palace) -> None: