fix: best-effort HNSW thread-pin retrofit + drop dead attempt-cap constant

Addresses remaining PR #976 review items after rebase on develop.

`get_collection(create=False)` previously returned existing collections without
re-applying `hnsw:num_threads=1`, so palaces created before the fix kept the
unsafe parallel-insert path. Add `_pin_hnsw_threads()` helper that calls
`collection.modify(configuration=UpdateCollectionConfiguration(
hnsw=UpdateHNSWConfiguration(num_threads=1)))` best-effort on every
`get_collection` call (including the MCP server's `_get_collection`).

In chromadb 1.5.x the runtime config does not persist to disk across
`PersistentClient` reopens, so the retrofit is re-applied each process start
rather than being a one-shot migration. Fresh palaces keep the metadata-based
pin as primary defense; legacy palaces now also get per-session protection
without requiring `mempalace nuke` + re-mine.

After the rebase on develop, `hook_precompact` delegates to `_mine_sync` and
no longer emits `decision: block`, so the attempt-cap constant was orphaned.
Grep confirms 0 usages in the repo — remove it.

- `_pin_hnsw_threads` retrofits legacy collection (num_threads None -> 1)
- `_pin_hnsw_threads` swallows all errors (never raises)
- `ChromaBackend.get_collection(create=False)` applies retrofit on legacy palace
- 62 tests pass (10 backends + 6 palace locks + 46 hooks_cli)
This commit is contained in:
Felipe Truman
2026-04-17 20:04:37 -03:00
committed by Igor Lins e Silva
parent 40d7958ca1
commit 8df944a54d
4 changed files with 100 additions and 14 deletions
+54
View File
@@ -16,6 +16,7 @@ from mempalace.backends.chroma import (
ChromaBackend,
ChromaCollection,
_fix_blob_seq_ids,
_pin_hnsw_threads,
quarantine_stale_hnsw,
)
@@ -443,3 +444,56 @@ def test_quarantine_stale_hnsw_skips_already_quarantined(tmp_path):
moved = quarantine_stale_hnsw(str(palace), stale_seconds=3600.0)
assert moved == []
assert drift.exists()
# ── _pin_hnsw_threads ─────────────────────────────────────────────────────
def test_pin_hnsw_threads_retrofits_legacy_collection(tmp_path):
"""Legacy collections (created without num_threads) get the retrofit applied."""
palace_path = tmp_path / "legacy-palace"
palace_path.mkdir()
client = chromadb.PersistentClient(path=str(palace_path))
col = client.create_collection(
"mempalace_drawers",
metadata={"hnsw:space": "cosine"}, # no num_threads — legacy
)
assert col.configuration_json.get("hnsw", {}).get("num_threads") is None
_pin_hnsw_threads(col)
assert col.configuration_json["hnsw"]["num_threads"] == 1
def test_pin_hnsw_threads_swallows_all_errors():
"""Retrofit never raises even when collection.modify explodes."""
class _ExplodingCollection:
def modify(self, *args, **kwargs):
raise RuntimeError("boom")
_pin_hnsw_threads(_ExplodingCollection()) # must not raise
def test_get_collection_applies_retrofit_on_existing_palace(tmp_path):
"""ChromaBackend.get_collection(create=False) applies the retrofit."""
palace_path = tmp_path / "palace"
palace_path.mkdir()
# Simulate a legacy palace: create collection without num_threads
bootstrap_client = chromadb.PersistentClient(path=str(palace_path))
bootstrap_client.create_collection(
"mempalace_drawers", metadata={"hnsw:space": "cosine"}
)
del bootstrap_client # drop reference so a fresh client reopens cleanly
wrapper = ChromaBackend().get_collection(
str(palace_path),
collection_name="mempalace_drawers",
create=False,
)
assert (
wrapper._collection.configuration_json["hnsw"]["num_threads"] == 1
)