Extends the init orchestrator to consume two new signal sources:
1. Claude Code conversation dirs: when the target is a
`~/.claude/projects/` root, convo_scanner contributes ProjectInfo
entries alongside the git/manifest projects. Dedup is by name,
preferring the entry with more user-authored activity.
2. Optional LLM refinement: when --llm is passed, discover_entities
constructs the provider, validates availability, and runs
llm_refine.refine_entities on the merged candidates. Status
summary (reclassified / dropped / cancelled / batch errors)
prints to stderr.
New init flags (opt-in, default remains zero-API):
- --llm: enable refinement
- --llm-provider: ollama (default) | openai-compat | anthropic
- --llm-model: default gemma4:e4b for Ollama
- --llm-endpoint: URL (required for openai-compat)
- --llm-api-key: falls back to env ($ANTHROPIC_API_KEY or
$OPENAI_API_KEY depending on provider)
Provider check_available runs before the scan, so the user sees an
immediate error ("Run: ollama pull <model>" or "ANTHROPIC_API_KEY not
set") rather than a mid-scan failure.
Takes the candidate set produced by phase-1 detection (manifests, git
authors, regex on prose) and asks an LLM to reclassify each candidate
as PERSON / PROJECT / TOPIC / COMMON_WORD / AMBIGUOUS.
Scale approach: never feed the raw corpus to the LLM. For each
candidate, collect up to 3 context lines from sampled prose, cap each
at 240 chars, batch 25 candidates per call. Keeps total input around
50-100K tokens even on large corpora and completes in a few minutes
on a 4B local model.
Interactive UX:
- Stderr progress bar with the current candidate name, updates
per-batch.
- Ctrl-C interrupts cleanly: returns a RefineResult with
`cancelled=True` and whatever was classified before the interrupt.
The partial result is safe to pass straight to confirm_entities.
- Per-batch errors (transport, parse) are recorded in `errors` and
don't abort the whole run.
Refinement scope: only `uncertain` and low-confidence `projects`
entries are sent. Manifest-backed projects (conf >= 0.95) and git-
authored people are already authoritative and skip the LLM.
Response parser is defensive — accepts `label` or `type` keys,
lowercase/uppercase variants, top-level list or wrapped object, and
strips markdown code fences. Unknown labels become AMBIGUOUS so the
user reviews them rather than silently accepting a bad classification.
`collect_corpus_text` provides a simple stratified prose sampler
(recent first, capped per-file) so callers don't need to build their
own corpus window.
28 tests with a FakeProvider (no network). Covers context collection,
prompt building, response parsing variants, classification apply,
end-to-end refine, and Ctrl-C partial-result behavior.
Three providers cover the useful space while keeping the zero-API
default:
- `ollama` (default): local models via http://localhost:11434. Works
fully offline. Tag-matching check accepts both `model` and
`model:latest` forms.
- `openai-compat`: any /v1/chat/completions endpoint. Covers
OpenRouter, LM Studio, llama.cpp server, vLLM, Groq, Together,
Fireworks, and most self-hosted frameworks. API key falls back to
$OPENAI_API_KEY. Endpoint normalization is forgiving about trailing
`/v1`.
- `anthropic`: Messages API v2023-06-01. API key falls back to
$ANTHROPIC_API_KEY. Concatenates multi-block text responses.
JSON mode is normalized across providers — Ollama uses
`format: "json"`, OpenAI-compat uses `response_format`, Anthropic uses
prompt-level instruction. Callers request JSON once; this module
handles the provider-specific plumbing.
No external SDK dependency; stdlib `urllib` throughout. HTTP errors
are wrapped into a single `LLMError` class so callers don't need to
distinguish transport, auth, and parse failures at the call site.
26 tests, all with mocked HTTP — suite runs offline with no real
provider required.
Claude Code stores sessions under `~/.claude/projects/<slug>/<id>.jsonl`
where `<slug>` is the original CWD with `/` replaced by `-`. That
encoding is lossy — can't distinguish `foo-bar` (one segment) from
`foo/bar` (two) — so slug-decoding alone produces wrong names for any
hyphenated project.
Fortunately, every message record carries a `cwd` field with the true
path. This scanner reads one record per session to recover the
accurate project name deterministically, falling back to slug-decoding
only if the JSONL is malformed or empty.
Output shape matches project_scanner.ProjectInfo so the discover
orchestrator can union results across sources. Session count doubles
as a density signal for ranking.
22 unit tests cover: root detection, cwd extraction with malformed
input tolerance, fallback slug decoding, name resolution using the
newest session (so renames win), and dedup when two encoded dirs
resolve to the same project.
`tomllib` is stdlib only in Python 3.11+. On Python 3.9/3.10 (and the
macOS runner) the scanner's toml parsing returned empty, so manifest
lookups for `pyproject.toml` / `Cargo.toml` produced no name. CI
surfaced this via 4 test_project_scanner.py failures on the 3.9 matrix.
Add `tomli>=2.0.0` as a conditional dependency for `python_version <
'3.11'` and fall back to it in `project_scanner.py`. The project still
declares `requires-python = ">=3.9"` so the fallback is the correct
shape.
`mempalace init` previously leaned entirely on regex-based entity
extraction from prose. That path works for text-only folders but wastes
signal in any codebase: the project's own name is already in
`package.json` / `pyproject.toml` / `Cargo.toml` / `go.mod`, and the
people who worked on it are in `git log`.
This adds `project_scanner.py`, which becomes the primary signal source
when real signal is available, with the regex detector preserved as the
fallback for prose-only folders (diaries, research notes, writing).
What it does:
- Walks the target directory, parses manifests for canonical project
names, and detects git repos by the presence of a `.git` directory.
- For each repo, reads `git log` for authors and filters obvious bots
(`[bot]`, `dependabot`, `renovate`, `github-actions`, names ending in
`bot`, `-autoroll`). Importantly does NOT filter
`@users.noreply.github.com` - that's GitHub's privacy-protected human
email, used by real contributors.
- Resolves author aliases with a union-find: commits that share a name
OR an email collapse into one person. Picks the most-frequent
real-name variant as display, ignoring handles and single-token
usernames.
- Flags "mine" projects: user is top-5 committer OR has >=10% of
commits OR >=20 commits. Ordered by user_commits in the UX.
- `discover_entities()` merges scanner results with the regex detector
case-insensitively (so `mempalace` from pyproject absorbs `MemPalace`
from docs), and suppresses the regex `uncertain` bucket when real
signal is already found - the user doesn't need to adjudicate prose
noise when the answer is already in git.
Integration: `cmd_init` now calls `discover_entities` instead of
running the regex detector directly. Same output shape, so
`confirm_entities` works unchanged.
Ships with 39 new tests covering manifest parsing, bot filtering,
union-find dedup, git repo discovery, scan integration, and
merge/fallback behavior. Existing 56 regex-detector tests all pass.
The pattern-matching detector had several systematic false positives that
crowded the init review with nonsense. Concrete fixes:
- CamelCase extraction: add `[A-Z][a-z]+(?:[A-Z][a-z]+|[A-Z]{2,})+` to
candidate patterns so `MemPalace`, `ChromaDB`, `OpenAI`, `ChatGPT` are
visible. Previously `MemPalace` fragmented into `Mem` + `Palace`.
- Dialogue `^NAME:\s` requires >=2 matches to count. A single metadata
line like `Created: 2026-04-21` was scoring as dialogue and classifying
`Created` as a person.
- Versioned/hyphenated pattern tightened to `\b{name}[-_]v?\d+(?:\.\d+)*\b`
(version-only). The previous `\b{name}[-v]\w+` matched `context-manager`,
`multi-word`, etc. - every hyphenated compound.
- Skip LICENSE/COPYING/NOTICE/AUTHORS/PATENTS files during scan. They
produce pure-English-prose noise (`Contributor`, `Software`, `Covered`,
`Before`).
- Extra SKIP_DIRS: `.terraform`, `vendor`, `target`.
- Expand stopword list with capitalized participles/descriptors that
commonly appear at sentence start: `created`, `updated`, `extracted`,
`processed`, `total`, `summary`, `auto`, `multi`, `hybrid`, `context`,
`bridge`, `batch`, `local`, `native`, `never`, `before`, `after`, etc.
- classify_entity: high-pronoun single-category signal now classifies as
person. A diary's main character gets referenced with pronouns, not
dialogue markers - requiring two signal categories demoted `Lu` (16
pronoun hits across 30 mentions) to uncertain. Gate on
`pronoun_hits >= 5 AND pronoun_hits / frequency >= 0.2` so common
sentence-start words (`Never`, `Before`) with incidental proximity
stay uncertain.
Two follow-up fixes from the v3.3.3 smoke test get folded into 3.3.3
before the tag is cut. Also syncs uv.lock with the 3.3.3 version
bump merged via #1144.
#1097 fixed mempalace_search to treat empty-string wing/room as
no filter, matching how LLM agents default to filling every optional
parameter with ''. The same pattern wasn't applied to diary_read:
passing wing='' defaulted to wing_<agent_name>, siloing away entries
that hooks had written to project-derived wings per #659.
When wing is empty/omitted, filter only on agent + room=diary so
callers get a unified view of the agent's journal across every wing
it has written to. Explicit wing=<name> continues to scope reads
to that wing only.
Adds test covering empty-wing read after writing to both the default
and a non-default wing.
_wing_from_transcript_path only matched '-Projects-<name>' segments,
so Linux users with code under ~/dev/, ~/code/, or ~/src/ fell through
to the wing_sessions fallback and lost the per-project diary scoping
introduced in #659.
Broaden the heuristic to derive the project from the final
dash-separated token of the encoded project-folder name under
.claude/projects/. Keeps the legacy -Projects- regex as a secondary
match for transcripts living outside the standard Claude Code path.
Covers macOS Users layout, Linux dev/code layouts, and deeper nested
source paths while preserving existing Projects/ behavior.
Restore-integrity release. Unbreaks fresh `pip install mempalace` from
v3.3.2 by re-tagging current develop, which carries both the plugin.json
consumer (shipped in 3.3.2) and the matching mempalace-mcp entry point
in pyproject.toml (added on develop ~10h after the 3.3.2 tag via #340
by @messelink). #1093 diagnosed by @jphein.
Bumps (all 5 sources agree per Version Guard / CLAUDE.md):
- mempalace/version.py 3.3.2 → 3.3.3
- pyproject.toml 3.3.2 → 3.3.3
- .claude-plugin/plugin.json 3.3.2 → 3.3.3
- .claude-plugin/marketplace.json 3.3.2 → 3.3.3
- .codex-plugin/plugin.json 3.3.2 → 3.3.3
- CHANGELOG.md new [3.3.3] entry
No code changes. The fix for #1093 is already on develop via merged PRs
#340, #1021, #851, #942, #833, #673, #661, #659, #1097, #1051, #1001,
#945.
Branch name intentionally outside the `release/*` ruleset so follow-up
CI-fix commits aren't gated behind a nested PR. (Supersedes #1143 —
closed for exactly that reason after it missed 3 of 5 version files.)
Smoke-tested locally from a fresh develop clone:
grep mempalace-mcp pyproject.toml .claude-plugin/plugin.json # both ✓
python -m build --wheel # ✓
pip install …-py3-none-any.whl # ✓
which mempalace-mcp # ✓
mempalace-mcp --help # ✓
* fix: add wing param to diary_write/diary_read, derive from transcript path
Without a wing override, all diary entries from the stop hook land in
wing_session-hook regardless of which project the session is in, making
per-project diary search impossible.
- tool_diary_write(): add optional `wing` param; sanitize and use it when
provided, fall back to wing_{agent_name} when omitted
- tool_diary_read(): add optional `wing` param for filtering by target wing
- TOOLS dict: expose `wing` in input_schema for both diary tools
- hooks_cli: add _wing_from_transcript_path() helper that extracts the
project name from Claude Code paths like
~/.claude/projects/-home-jp-Projects-kiyo-xhci-fix/... → kiyo-xhci-fix
- hook_stop: derive project wing and append wing= hint to block reason so
Claude writes diary entries to the correct per-project wing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: sanitize wing param, cross-platform paths, tighten test assertions
Addresses Copilot review feedback on #659.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: wing_ prefix + agent filter on diary_read
Addresses bensig's 2-issue review on this PR.
1. _wing_from_transcript_path() was returning bare project names
(e.g. "myproject") while all existing wings follow the wing_*
convention from AAAK_SPEC. Entries landed in wing="myproject"
while diary_read defaulted to wing="wing_<agent_name>" —
orphaning every diary entry written by the stop hook. Now
returns "wing_<project>" and falls back to "wing_sessions".
2. tool_diary_read() did not include agent_name in the ChromaDB
where filter when a custom wing was provided — any caller with
a shared wing could read entries written by other agents.
Add {"agent": agent_name} to the $and clause. Also flagged by
Qudo and left unresolved until now.
Tests updated to expect the wing_ prefix (6 tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- Swap Inter/Cormorant Garamond/Geist → Neue Machina/Satoshi/Onest (all free/web)
- Align color palette to Crystal Lattice decision (0002): void #080C18, cyan-vivid #38BDF8, ice #DBE7F5
- Update hero: "Memory *is* identity." with italic blue "is", white "identity"
- New hero subtext: "Every conversation, every idea, every small decision… held somewhere safe."
- JetBrains Mono unchanged (already OFL)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clean squash by jphein on 2026-04-21. Backwards-compatible via hook_silent_save config flag. Save marker now only advances after confirmed write — strictly safer than status quo.
Merging to fix active pagination crash — 3 users hit this on v3.3.2 in the last 20 hours (54K, 114K, 465K drawer palaces). None-guard regression fixed. 9-day production soak on 165K-drawer palace confirms stability.
Adds a `hook_silent_save` mode (default `true` in new installs) where
the stop and precompact hooks write diary entries directly via the
Python API — no AI block, no MCP tool roundtrip, no possibility of the
AI forgetting or ignoring the save instruction.
**Two modes, controlled by `hook_silent_save` in `~/.mempalace/config.json`:**
1. **Silent mode** (default): Direct call to `tool_diary_write()`. Plain
text, no AI involved, deterministic. Save marker advances only after
the write is confirmed, so mid-save failures do not lose exchanges.
Shows `"✦ N memories woven into the palace"` as a systemMessage
notification so the user knows the save fired.
2. **Block mode** (legacy): Returns `{"decision": "block"}` asking the
AI to call the MCP tool chain. Non-deterministic — the AI may ignore,
summarize lossy, or fail. Kept for backward compatibility.
**Extras rolled in:**
- Block reasons name "MemPalace" explicitly and instruct the AI not to
write to Claude Code's native auto-memory (.md files) — prevents the
two memory systems from stepping on each other.
- Codex transcript handling (`event_msg` payloads) in
`_count_human_messages` + `_extract_recent_messages`.
- Tightened stopword leak in diary summaries; docstring polish; test
hermeticity fixes (per-test `STATE_DIR` patching).
**Tests:** hooks_cli tests cover silent-save path, save-marker
advancement after confirmed write only, and systemMessage formatting.
Rebased fresh on upstream/develop. Only touches files germane to the
feature (hooks_cli.py, tests, hooks/README.md, HOOKS_TUTORIAL.md) —
stale fork-local `.sh` wrapper and plugin manifest changes dropped.
The legacy hook scripts `hooks/mempal_save_hook.sh` and
`hooks/mempal_precompact_hook.sh` shell out to `python3` for JSON
parsing and transcript-message counting. On macOS GUI launches of
Claude Code — `open -a`, Spotlight, the dock — the harness inherits
`PATH` from launchd (`/usr/bin:/bin:/usr/sbin:/sbin`), which may not
contain a `python3` at all, or may contain only a system Python that
lacks what the hook needs. The hook then fails silently in the
background log where users never look.
`mempalace` auto-ingest itself is unaffected — #340 switched that
path to the `mempalace` CLI entry point, which pipx/uv install on a
stable global PATH.
This PR adds a `MEMPAL_PYTHON` environment variable that users can
set to point the hook at any Python 3 interpreter. Resolution order
applied at each `python3` invocation site inside the two hooks:
1. $MEMPAL_PYTHON (if set and executable)
2. $(command -v python3) on PATH
3. bare `python3` as a last resort
The interpreter does not need `mempalace` installed in it — only the
standard-library `json` and `sys` modules. The hook's `mempalace mine`
call runs via the CLI, independent of this override.
hooks/README.md documents the macOS GUI PATH issue and the
MEMPAL_PYTHON override. tests/test_hooks_shell.py adds 3 regression
tests (Linux/macOS only, POSIX bash):
- MEMPAL_PYTHON override wins over PATH (proved via a
marker-emitting shim that proxies to the real interpreter).
- Non-executable MEMPAL_PYTHON falls back to PATH rather than
crashing on permission denied.
- Unset MEMPAL_PYTHON resolves via PATH.
`hooks_cli.py` (the Python implementation invoked via
`mempalace hook run ...`) already uses `sys.executable` and is
therefore trivially correct — no changes needed there.
Supersedes abandoned branch `fix/hook-bugs`.
Co-Authored-By: MSL <232237854+milla-jovovich@users.noreply.github.com>
Three assertions in test_mcp_command_* were still checking for the old
`python -m mempalace.mcp_server` output string. Update to match the new
`mempalace-mcp` command printed by cmd_mcp().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cmd_mcp() in cli.py was still printing `python -m mempalace.mcp_server`
as the setup command. Update to use the mempalace-mcp console entry
point added in the previous commit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
hooks/mempal_precompact_hook.sh and hooks/mempal_save_hook.sh used
python3 -m mempalace mine which fails when mempalace is installed via
pipx or uv. Switch to the mempalace CLI entry point which pipx/uv put
on PATH. Also removes the now-unused PYTHON variable from mempal_save_hook.sh.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hook scripts used `python3 -m mempalace` which fails when mempalace is
installed via pipx or uv. Using the `mempalace` CLI command directly
works for all installation methods. Dev users running from source should
use `pip install -e .` as documented in CONTRIBUTING.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The MCP server config used `python -m mempalace.mcp_server` which fails
when mempalace is installed via pipx or uv, since the system python
cannot find the module in the isolated venv. Adding a `mempalace-mcp`
console_scripts entry point ensures the MCP server works regardless of
installation method (pip, pipx, uv, conda).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Conflicts resolved by taking the 3.3.2 side for all version files:
- pyproject.toml, mempalace/version.py (3.3.2)
- .claude-plugin/marketplace.json, .claude-plugin/plugin.json (3.3.2)
- .codex-plugin/plugin.json (3.3.2)
- README.md version badge (3.3.2)
- uv.lock (3.3.2)
- CHANGELOG.md keeps [3.3.2] section on top of main's [3.3.1]
No source-code conflicts; main's 3.3.1 commit footprint is already in
develop's history via the earlier sync boundaries.
1033 tests pass on the merged tree.
Non-ASCII glyphs (regression of the #681 class of Windows UnicodeEncodeError):
- mempalace/cli.py: "✗" → "ERROR:", "⚠" → "WARNING:", em dash → "-"
- mempalace/sweeper.py: "⚠" → "WARNING:"
Backend arg validation:
- mempalace/backends/chroma.py: `_normalize_get_collection_args` now
raises TypeError on unexpected trailing positional args instead of
silently dropping them — surfaces call-site bugs early.
Docs site:
- website/.vitepress/config.mts: gate Google Analytics scripts behind
MEMPALACE_DOCS_GA_ID env var (default off). Self-hosters no longer
get GA injected unconditionally.
Landing page SPA hygiene:
- website/.vitepress/theme/landing/useLandingEffects.js: collect all
IntersectionObserver disconnects and removeEventListener thunks in a
shared `cleanups` registry; drain it in `onBeforeUnmount` so observers
and form/replay listeners don't leak across SPA navigations.
Version bumps across pyproject.toml, mempalace/version.py, README badge,
uv.lock, and plugin manifests (.claude-plugin/*, .codex-plugin/*).
CHANGELOG aligned with main (post-3.3.1) and a new [3.3.2] section added
covering the 11 PRs merged on develop since v3.3.1 — silent-transcript-drop
fix + tandem sweeper (#998), None-metadata guards (#999, #1013),
chromadb ≥1.5.4 for Py 3.13/3.14 (#1010), Windows Unicode (#681),
HNSW quarantine recovery (#1000), PID stacking guard (#1023), doc-path
cleanup (#996, #1012), and RFC 001/002 internal scaffolding (#995, #1014, #990).
Addresses bensig's review on PR #1021.
silent_guard was initialized to False, so when both MempalaceConfig
import and .hook_silent_save attribute access failed, silent_guard
stayed False. Then `if not silent_guard:` fired and returned empty —
silently dropping the save. In silent mode (the default since v3.3.0),
saves should ALWAYS proceed on config-read failure. Changing the
initial value to True makes that the safe default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upstream develop commit feba7e8 (2026-04-18) added `m = m or {}` to the
single-shot `for m in metas:` loop after this branch already rewrote
status() to paginate. Without porting the guard forward, merging this PR
would silently drop jp's fix and crash again on palaces with null-metadata
drawers.
Addresses bensig's review on #851.
copilot-swe-agent[bot]