cleanup and remote only

This commit is contained in:
2026-05-09 10:52:25 -05:00
parent 2fc47a52fc
commit 40e5e5e3cc
136 changed files with 1502 additions and 349529 deletions
+91 -83
View File
@@ -2,17 +2,51 @@
These hook scripts make MemPalace save automatically. No manual "save" commands needed.
This deployment ships only the **remote** hook variants — the palace runs as a Docker container on a server (e.g. Unraid), and hooks `curl` the active session transcript to the server's `/ingest/transcript` endpoint over HTTPS with bearer auth. Server-side, the existing `mine_convos` pipeline handles entity detection, room assignment, dedup, and idempotency. See [`deploy/unraid/README.md`](../deploy/unraid/README.md) for the server side.
## What They Do
| Hook | When It Fires | What Happens |
|------|--------------|-------------|
| **Save Hook** | Every 15 human messages | Auto-mines transcript (tool output included), then blocks the AI to save topics/decisions/quotes |
| **PreCompact Hook** | Right before context compaction | Auto-mines transcript, then emergency save — forces the AI to save EVERYTHING before losing context |
|---|---|---|
| **Save Hook** (`mempal_save_hook_remote.sh`) | Every 15 user messages (configurable via `SAVE_INTERVAL`) | Backgrounded `curl` POSTs the active transcript. Returns immediately so the AI doesn't stall. Idempotent — failed retries are safe. |
| **PreCompact Hook** (`mempal_precompact_hook_remote.sh`) | Right before context compaction | Synchronous `curl` POST. Blocks until the upload completes (or the hook timeout fires) so memory is durable before context shrinks. |
**Two-layer capture:** Hooks auto-mine the JSONL transcript directly into the palace (capturing raw tool output — Bash results, search findings, build errors). They also block the AI with a reason message telling it to save verbatim tool output and key context. Belt and suspenders — tool output gets stored even if the AI summarizes instead of quoting.
**Two-layer capture.** The save hook ships the JSONL transcript directly to the server (capturing raw tool output — Bash results, search findings, build errors), where the miner files it verbatim into the palace. Tool output gets stored even if the AI summarizes instead of quoting.
## Env-var contract
The scripts read all configuration from environment variables. There is no script-level config to edit; the same script works against any number of machines.
| Variable | Required | Purpose |
|---|---|---|
| `MEMPAL_REMOTE_URL` | yes | Base URL of the MemPalace server, e.g. `https://unraid.local:8443`. |
| `MEMPAL_REMOTE_TOKEN` | yes | Bearer token shared with the server's `MEMPAL_TOKEN`. |
| `MEMPAL_REMOTE_INSECURE` | no | Set to `1` to skip TLS verification. Use only when the server uses Caddy's `tls internal` self-signed cert and the client hasn't trusted the root CA. |
| `MEMPAL_REMOTE_WING` | no | Force a specific wing for this client's transcripts. Default: server derives wing from the session id. |
| `SAVE_INTERVAL` | no | Override the default of 15 user messages. |
| `MEMPAL_PYTHON` | no | Path to a Python 3 interpreter. Only needs `json` + `sys` from stdlib — mempalace does not need to be installed in it. Used to parse the hook's stdin JSON. |
Set these persistently:
**PowerShell (Windows):**
```powershell
[Environment]::SetEnvironmentVariable("MEMPAL_REMOTE_URL", "https://unraid.local:8443", "User")
[Environment]::SetEnvironmentVariable("MEMPAL_REMOTE_TOKEN", "<the-token>", "User")
[Environment]::SetEnvironmentVariable("MEMPAL_REMOTE_INSECURE", "1", "User") # if self-signed
```
**Bash/Zsh:** add the same exports to `~/.zshrc` / `~/.bashrc`.
If `MEMPAL_REMOTE_URL` or `MEMPAL_REMOTE_TOKEN` is unset, the scripts no-op and log a one-liner — they never block the AI from stopping. Safe to install on a machine that doesn't have a remote configured yet.
## Install — Claude Code
Make the scripts executable:
```bash
chmod +x hooks/mempal_save_hook_remote.sh hooks/mempal_precompact_hook_remote.sh
```
Add to `.claude/settings.local.json`:
```json
@@ -22,26 +56,21 @@ Add to `.claude/settings.local.json`:
"matcher": "*",
"hooks": [{
"type": "command",
"command": "/absolute/path/to/hooks/mempal_save_hook.sh",
"command": "/absolute/path/to/hooks/mempal_save_hook_remote.sh",
"timeout": 30
}]
}],
"PreCompact": [{
"hooks": [{
"type": "command",
"command": "/absolute/path/to/hooks/mempal_precompact_hook.sh",
"timeout": 30
"command": "/absolute/path/to/hooks/mempal_precompact_hook_remote.sh",
"timeout": 60
}]
}]
}
}
```
Make them executable:
```bash
chmod +x hooks/mempal_save_hook.sh hooks/mempal_precompact_hook.sh
```
## Install — Codex CLI (OpenAI)
Add to `.codex/hooks.json`:
@@ -50,132 +79,111 @@ Add to `.codex/hooks.json`:
{
"Stop": [{
"type": "command",
"command": "/absolute/path/to/hooks/mempal_save_hook.sh",
"command": "/absolute/path/to/hooks/mempal_save_hook_remote.sh",
"timeout": 30
}],
"PreCompact": [{
"type": "command",
"command": "/absolute/path/to/hooks/mempal_precompact_hook.sh",
"timeout": 30
"command": "/absolute/path/to/hooks/mempal_precompact_hook_remote.sh",
"timeout": 60
}]
}
```
## Configuration
Edit `mempal_save_hook.sh` to change:
- **`SAVE_INTERVAL=15`** — How many human messages between saves. Lower = more frequent saves, higher = less interruption.
- **`STATE_DIR`** — Where hook state is stored (defaults to `~/.mempalace/hook_state/`)
- **`MEMPAL_DIR`** — Optional **project directory** (code, notes, docs) to also mine on each save trigger, with `--mode projects`. The hook ALWAYS mines the active conversation transcript automatically with `--mode convos``MEMPAL_DIR` is purely additive, never an override. Leave blank if you don't want to ingest project files.
- **`MEMPALACE_PYTHON`** — Optional env var. Python interpreter with mempalace + chromadb installed. Auto-detects: `MEMPALACE_PYTHON` env var → repo `venv/bin/python3` → system `python3`. Set this if your venv is in a non-standard location.
### mempalace CLI
The relevant commands are:
```bash
mempalace mine <dir> # Mine all files in a directory
mempalace mine <dir> --mode convos # Mine conversation transcripts only
```
The hooks resolve the repo root automatically from their own path, so they work regardless of where you install the repo.
## How It Works (Technical)
## How it works
### Save Hook (Stop event)
```
User sends message → AI responds → Claude Code fires Stop hook
Hook counts human messages in JSONL transcript
Hook counts user messages in JSONL transcript
┌─── < 15 since last save ──→ echo "{}" (let AI stop)
┌─── < SAVE_INTERVAL since last save ──→ echo "{}" (let AI stop)
└─── ≥ 15 since last save
└─── ≥ SAVE_INTERVAL since last save
Auto-mine transcript → palace (tool output captured)
Background curl POST → server /ingest/transcript
{"decision": "block", "reason": "save tool output verbatim..."}
Hook returns {} immediately (AI stops normally)
AI saves to palace (topics, decisions, quotes)
AI tries to stop again
stop_hook_active = true
Hook sees flag → echo "{}" (let it through)
Server-side miner runs in background, files drawers
```
The `stop_hook_active` flag prevents infinite loops: block once → AI saves → tries to stop → flag is true → we let it through.
### PreCompact Hook
```
Context window getting full → Claude Code fires PreCompact
Find transcript (from input or session_id lookup)
Synchronous curl POST → server /ingest/transcript
Auto-mine transcript → palace (tool output captured)
Wait for 200 OK (or hook timeout)
{"decision": "block", "reason": "save tool output verbatim..."}
AI saves everything
Compaction proceeds
echo "{}" → Compaction proceeds
```
No counting needed — compaction always warrants a save. The auto-mine captures raw tool output before the AI gets a chance to summarize it away.
Synchronous on PreCompact is intentional — this is the safety net before context shrinks. The Claude Code hook timeout (set in `settings.local.json`) bounds how long we'll wait.
## Debugging
Check the hook log:
```bash
cat ~/.mempalace/hook_state/hook.log
tail -f ~/.mempalace/hook_state/hook.log
```
Example output:
Example:
```
[14:30:15] Session abc123: 12 exchanges, 12 since last save
[14:35:22] Session abc123: 15 exchanges, 15 since last save
[14:35:22] TRIGGERING SAVE at exchange 15
[14:40:01] Session abc123: 18 exchanges, 3 since last save
[14:35:22] ingest ok
[14:50:18] PRE-COMPACT triggered for session abc123
[14:50:19] PRE-COMPACT ingest ok
```
## Known Limitations
A 401 response means the bearer token is wrong. A connection error means the URL/cert is wrong (or the server is down). All curl output goes to the same log.
**Hooks require session restart after install.** Claude Code loads hooks from `settings.json` at session start only. If you run `mempalace init` or manually edit hook config mid-session, the hooks won't fire until you restart Claude Code. This is a Claude Code limitation.
## Known limitations
**`MEMPAL_PYTHON` override for the hook's internal Python calls.** The save hook parses its JSON input and counts transcript messages with `python3`. When the harness is launched from a GUI on macOS — `open -a`, Spotlight, the dock — its `PATH` is the minimal `/usr/bin:/bin:/usr/sbin:/sbin` inherited from `launchd`, not your shell PATH. If `python3` isn't on that PATH, those internal calls fail and the hook can't count exchanges.
**Hooks require session restart after install.** Claude Code loads hooks from `settings.json` at session start only. If you edit hook config mid-session, restart Claude Code to pick up changes.
Point the hook at any Python 3 interpreter to fix it:
**Python interpreter resolution.** The scripts parse hook stdin JSON with `python3`. When Claude Code is launched from a GUI on macOS (Spotlight, dock, `open -a`), its `PATH` is the minimal `/usr/bin:/bin:/usr/sbin:/sbin` inherited from `launchd` rather than your shell PATH. If `python3` isn't there, set `MEMPAL_PYTHON` to a known-good interpreter:
```bash
export MEMPAL_PYTHON="/usr/bin/python3" # system Python is fine
export MEMPAL_PYTHON="$HOME/.venvs/mempalace/bin/python" # or your venv
export MEMPAL_PYTHON="/usr/bin/python3"
# or:
export MEMPAL_PYTHON="$HOME/.venvs/x/bin/python"
```
Resolution priority: `$MEMPAL_PYTHON` (if set and executable)`$(command -v python3)` → bare `python3`. The interpreter only needs `json` and `sys` from the standard library — `mempalace` itself does not need to be installed in it.
Resolution priority: `$MEMPAL_PYTHON``$(command -v python3)` → bare `python3`. The interpreter only needs `json` and `sys` mempalace itself does not need to be installed.
Note: the `mempalace mine` auto-ingest runs via the `mempalace` CLI, so that command also needs to be on the hook's `PATH`. Installing with `pipx install mempalace` or `uv tool install mempalace` puts it on a stable global location; otherwise extend the hook environment's `PATH` to include your venv's `bin/`.
**`MineAlreadyRunning` collisions.** If two clients ingest simultaneously, the second one's request returns 500 because the server-side `mine_lock` is held. The save hook is idempotent — the next save catches up. If you see this constantly in the log, raise `SAVE_INTERVAL` on the chattier client.
## Backfill Past Conversations
## Backfilling past conversations
The hooks only capture conversations going forward. To mine **past** Claude Code sessions into your palace, run a one-time backfill:
The hooks only capture sessions going forward. To mine **past** sessions into the remote palace, loop `curl` over them:
```bash
mempalace mine ~/.claude/projects/ --mode convos
# Claude Code sessions
for f in ~/.claude/projects/**/*.jsonl; do
curl -k -X POST \
-H "Authorization: Bearer $MEMPAL_REMOTE_TOKEN" \
-H "X-Session-Id: $(basename "$f" .jsonl)" \
--data-binary @"$f" \
"$MEMPAL_REMOTE_URL/ingest/transcript"
done
# Codex CLI sessions
for f in ~/.codex/sessions/**/*.jsonl; do
curl -k -X POST \
-H "Authorization: Bearer $MEMPAL_REMOTE_TOKEN" \
-H "X-Session-Id: $(basename "$f" .jsonl)" \
--data-binary @"$f" \
"$MEMPAL_REMOTE_URL/ingest/transcript"
done
```
This scans all JSONL transcripts from previous sessions and files them into the `conversations` wing. On a typical developer machine with months of history, this can yield 50K200K drawers.
For Codex CLI sessions:
```bash
mempalace mine ~/.codex/sessions/ --mode convos
```
This only needs to be done once — after that, the hooks auto-mine each session as you go.
The server-side miner is idempotent — re-uploading the same transcript won't double-file. Drop `-k` once Caddy's root CA is trusted on the client.
## Cost
**Zero extra tokens.** The hooks notify the AI that saves happened in the background — the AI doesn't need to write anything in the chat. All filing is handled automatically. Previous versions asked the AI to write diary entries and drawer content in the chat window, which cost ~$1/session in retransmitted tokens.
**Zero extra tokens.** The hooks save in the background — the AI doesn't need to write anything in the chat window. All filing happens server-side after the upload returns.
-123
View File
@@ -1,123 +0,0 @@
#!/bin/bash
# MEMPALACE PRE-COMPACT HOOK — Emergency save before compaction
#
# Claude Code "PreCompact" hook. Fires RIGHT BEFORE the conversation
# gets compressed to free up context window space.
#
# This is the safety net. When compaction happens, the AI loses detailed
# context about what was discussed. This hook forces one final save of
# EVERYTHING before that happens.
#
# Unlike the save hook (which triggers every N exchanges), this ALWAYS
# blocks — because compaction is always worth saving before.
#
# === INSTALL ===
# Add to .claude/settings.local.json:
#
# "hooks": {
# "PreCompact": [{
# "hooks": [{
# "type": "command",
# "command": "/absolute/path/to/mempal_precompact_hook.sh",
# "timeout": 30
# }]
# }]
# }
#
# For Codex CLI, add to .codex/hooks.json:
#
# "PreCompact": [{
# "type": "command",
# "command": "/absolute/path/to/mempal_precompact_hook.sh",
# "timeout": 30
# }]
#
# === HOW IT WORKS ===
#
# Claude Code sends JSON on stdin with:
# session_id — unique session identifier
#
# We always return decision: "block" with a reason telling the AI
# to save everything. After the AI saves, compaction proceeds normally.
#
# === MEMPALACE CLI ===
# The hook ALWAYS mines the active conversation transcript synchronously
# before compaction (via `mempalace mine <transcript-dir> --mode convos`).
# MEMPAL_DIR is an *additional*, optional target for project files — it
# does not replace the conversation mine.
STATE_DIR="$HOME/.mempalace/hook_state"
mkdir -p "$STATE_DIR"
# Optional: project directory (code / notes / docs) to also mine before
# compaction. Mined with `--mode projects`. The conversation transcript
# is always mined regardless — this is purely additive.
# Example: MEMPAL_DIR="$HOME/projects/my_app"
MEMPAL_DIR=""
# Resolve the Python interpreter. Same contract as mempal_save_hook.sh:
# MEMPAL_PYTHON (explicit override) → $(command -v python3) → bare python3.
MEMPAL_PYTHON_BIN="${MEMPAL_PYTHON:-}"
if [ -z "$MEMPAL_PYTHON_BIN" ] || [ ! -x "$MEMPAL_PYTHON_BIN" ]; then
MEMPAL_PYTHON_BIN="$(command -v python3 2>/dev/null || echo python3)"
fi
# Read JSON input from stdin
INPUT=$(cat)
# Parse session_id and transcript_path in one call. Sanitize both, then
# read sanitized values from one-per-line stdout into shell variables —
# avoids ``eval`` on generated code (#1231 review). Same contract as
# mempal_save_hook.sh.
mapfile -t _mempal_parsed < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
import sys, json, re
data = json.load(sys.stdin)
sid = data.get('session_id', 'unknown')
tp = data.get('transcript_path', '')
safe = lambda s: re.sub(r'[^a-zA-Z0-9_/.\-~]', '', str(s))
print(safe(sid))
print(safe(tp))
" 2>/dev/null)
SESSION_ID="${_mempal_parsed[0]:-unknown}"
TRANSCRIPT_PATH="${_mempal_parsed[1]:-}"
# Expand ~ in path
TRANSCRIPT_PATH="${TRANSCRIPT_PATH/#\~/$HOME}"
# Validate that TRANSCRIPT_PATH looks like a transcript file. Mirrors
# mempalace.hooks_cli._validate_transcript_path so the shell hook
# rejects the same shapes the Python hook rejects (#1231 review).
is_valid_transcript_path() {
local path="$1"
[ -n "$path" ] || return 1
case "$path" in
*.json|*.jsonl) ;;
*) return 1 ;;
esac
case "/$path/" in
*/../*) return 1 ;;
esac
return 0
}
echo "[$(date '+%H:%M:%S')] PRE-COMPACT triggered for session $SESSION_ID" >> "$STATE_DIR/hook.log"
# Run ingest synchronously so memories land before compaction. Two
# independent targets — both run if both are set:
# 1. TRANSCRIPT_PATH (from Claude Code) → parent dir, --mode convos
# 2. MEMPAL_DIR → --mode projects
if is_valid_transcript_path "$TRANSCRIPT_PATH" && [ -f "$TRANSCRIPT_PATH" ]; then
mempalace mine "$(dirname "$TRANSCRIPT_PATH")" --mode convos \
>> "$STATE_DIR/hook.log" 2>&1
elif [ -n "$TRANSCRIPT_PATH" ]; then
echo "[$(date '+%H:%M:%S')] Skipping invalid transcript path: $TRANSCRIPT_PATH" \
>> "$STATE_DIR/hook.log"
fi
if [ -n "$MEMPAL_DIR" ] && [ -d "$MEMPAL_DIR" ]; then
mempalace mine "$MEMPAL_DIR" --mode projects \
>> "$STATE_DIR/hook.log" 2>&1
fi
# Silent: return empty JSON to not block. "decision": "allow" is invalid —
# only "block" or {} are recognized.
echo '{}'
+102
View File
@@ -0,0 +1,102 @@
#!/bin/bash
# MEMPALACE PRE-COMPACT HOOK (REMOTE) — emergency save before compaction.
#
# Drop-in replacement for mempal_precompact_hook.sh when MemPalace runs
# on a server. Always synchronous: we wait for the upload to complete
# before returning so the transcript is on the server before the
# conversation gets compressed.
#
# Required env vars (same as the save hook):
# MEMPAL_REMOTE_URL e.g. https://unraid.local:8443
# MEMPAL_REMOTE_TOKEN bearer token
# Optional:
# MEMPAL_REMOTE_WING explicit wing override
# MEMPAL_REMOTE_INSECURE "1" for self-signed cert
#
# === INSTALL ===
# Add to .claude/settings.local.json:
#
# "hooks": {
# "PreCompact": [{
# "hooks": [{
# "type": "command",
# "command": "/abs/path/to/mempal_precompact_hook_remote.sh",
# "timeout": 60
# }]
# }]
# }
set -u
STATE_DIR="$HOME/.mempalace/hook_state"
mkdir -p "$STATE_DIR"
MEMPAL_PYTHON_BIN="${MEMPAL_PYTHON:-}"
if [ -z "$MEMPAL_PYTHON_BIN" ] || [ ! -x "$MEMPAL_PYTHON_BIN" ]; then
MEMPAL_PYTHON_BIN="$(command -v python3 2>/dev/null || echo python3)"
fi
if [ -z "${MEMPAL_REMOTE_URL:-}" ] || [ -z "${MEMPAL_REMOTE_TOKEN:-}" ]; then
echo "[$(date '+%H:%M:%S')] PRE-COMPACT: MEMPAL_REMOTE_URL/TOKEN not set — skipping" \
>> "$STATE_DIR/hook.log"
echo "{}"
exit 0
fi
INPUT=$(cat)
mapfile -t _mempal_parsed < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
import sys, json, re
data = json.load(sys.stdin)
sid = data.get('session_id', 'unknown')
tp = data.get('transcript_path', '')
safe = lambda s: re.sub(r'[^a-zA-Z0-9_/.\-~]', '', str(s))
print(safe(sid))
print(safe(tp))
" 2>/dev/null)
SESSION_ID="${_mempal_parsed[0]:-unknown}"
TRANSCRIPT_PATH="${_mempal_parsed[1]:-}"
TRANSCRIPT_PATH="${TRANSCRIPT_PATH/#\~/$HOME}"
is_valid_transcript_path() {
local path="$1"
[ -n "$path" ] || return 1
case "$path" in
*.json|*.jsonl) ;;
*) return 1 ;;
esac
case "/$path/" in
*/../*) return 1 ;;
esac
return 0
}
echo "[$(date '+%H:%M:%S')] PRE-COMPACT triggered for session $SESSION_ID" \
>> "$STATE_DIR/hook.log"
# Synchronous upload — pre-compact is the safety net, blocking is correct
# here. The Claude Code hook timeout (set in settings.local.json) bounds
# how long we'll wait.
if is_valid_transcript_path "$TRANSCRIPT_PATH" && [ -f "$TRANSCRIPT_PATH" ]; then
CURL_OPTS=("-sS" "--max-time" "55" "-X" "POST")
[ "${MEMPAL_REMOTE_INSECURE:-0}" = "1" ] && CURL_OPTS+=("-k")
WING_HEADER=()
[ -n "${MEMPAL_REMOTE_WING:-}" ] && WING_HEADER=(-H "X-Wing: $MEMPAL_REMOTE_WING")
curl "${CURL_OPTS[@]}" \
-H "Authorization: Bearer $MEMPAL_REMOTE_TOKEN" \
-H "X-Session-Id: $SESSION_ID" \
-H "Content-Type: application/octet-stream" \
"${WING_HEADER[@]}" \
--data-binary "@$TRANSCRIPT_PATH" \
"$MEMPAL_REMOTE_URL/ingest/transcript" \
>> "$STATE_DIR/hook.log" 2>&1 \
&& echo "[$(date '+%H:%M:%S')] PRE-COMPACT ingest ok" >> "$STATE_DIR/hook.log" \
|| echo "[$(date '+%H:%M:%S')] PRE-COMPACT ingest FAILED — context will compact unsaved" \
>> "$STATE_DIR/hook.log"
elif [ -n "$TRANSCRIPT_PATH" ]; then
echo "[$(date '+%H:%M:%S')] PRE-COMPACT: invalid transcript path: $TRANSCRIPT_PATH" \
>> "$STATE_DIR/hook.log"
fi
echo "{}"
-223
View File
@@ -1,223 +0,0 @@
#!/bin/bash
# MEMPALACE SAVE HOOK — Auto-save every N exchanges
#
# Claude Code "Stop" hook. After every assistant response:
# 1. Counts human messages in the session transcript
# 2. Every SAVE_INTERVAL messages, BLOCKS the AI from stopping
# 3. Returns a reason telling the AI to save structured diary + palace entries
# 4. AI does the save (topics, decisions, code, quotes → organized into palace)
# 5. Next Stop fires with stop_hook_active=true → lets AI stop normally
#
# The AI does the classification — it knows what wing/hall/closet to use
# because it has context about the conversation. No regex needed.
#
# === INSTALL ===
# Add to .claude/settings.local.json:
#
# "hooks": {
# "Stop": [{
# "matcher": "*",
# "hooks": [{
# "type": "command",
# "command": "/absolute/path/to/mempal_save_hook.sh",
# "timeout": 30
# }]
# }]
# }
#
# For Codex CLI, add to .codex/hooks.json:
#
# "Stop": [{
# "type": "command",
# "command": "/absolute/path/to/mempal_save_hook.sh",
# "timeout": 30
# }]
#
# === HOW IT WORKS ===
#
# Claude Code sends JSON on stdin with these fields:
# session_id — unique session identifier
# stop_hook_active — true if AI is already in a save cycle (prevents infinite loop)
# transcript_path — path to the JSONL transcript file
#
# When we block, Claude Code shows our "reason" to the AI as a system message.
# The AI then saves to memory, and when it tries to stop again,
# stop_hook_active=true so we let it through. No infinite loop.
#
# === MEMPALACE CLI ===
# The hook ALWAYS mines the active conversation transcript automatically
# (via `mempalace mine <transcript-dir> --mode convos`). MEMPAL_DIR is an
# *additional*, optional target for project files — it does not replace
# the conversation mine.
#
# === CONFIGURATION ===
SAVE_INTERVAL=15 # Save every N human messages (adjust to taste)
STATE_DIR="$HOME/.mempalace/hook_state"
mkdir -p "$STATE_DIR"
# Optional: project directory (code / notes / docs) to also mine each
# save trigger. Mined with `--mode projects`. The conversation transcript
# is always mined regardless — this is purely additive.
# Example: MEMPAL_DIR="$HOME/projects/my_app"
MEMPAL_DIR=""
# Resolve the Python interpreter the hook should use.
#
# Why this is nontrivial: GUI-launched Claude Code on macOS (or any harness
# that doesn't inherit the user's shell PATH) may find a `python3` on PATH
# that lacks mempalace — e.g. /usr/bin/python3 while the user installed
# mempalace into a venv or pyenv. Users in that situation can point the
# hook at the right interpreter by exporting MEMPAL_PYTHON.
#
# Resolution order (first hit wins):
# 1. $MEMPAL_PYTHON — explicit user override (absolute path)
# 2. $(command -v python3) — first python3 on the hook's PATH
# 3. bare "python3" — last-resort fallback (hope the PATH has it)
MEMPAL_PYTHON_BIN="${MEMPAL_PYTHON:-}"
if [ -z "$MEMPAL_PYTHON_BIN" ] || [ ! -x "$MEMPAL_PYTHON_BIN" ]; then
MEMPAL_PYTHON_BIN="$(command -v python3 2>/dev/null || echo python3)"
fi
# Read JSON input from stdin
INPUT=$(cat)
# Parse all fields in a single Python call (3x faster than separate invocations)
# without invoking ``eval`` on generated code: Python prints one sanitized
# value per line, the shell reads them via ``mapfile`` and does plain
# variable assignment — same data, smaller blast radius if the sanitizer
# is ever bypassed (#1231 review).
mapfile -t _mempal_parsed < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
import sys, json, re
data = json.load(sys.stdin)
sid = data.get('session_id', 'unknown')
sha_raw = data.get('stop_hook_active', False)
tp = data.get('transcript_path', '')
# Shell-safe output — only allow alphanumeric, underscore, hyphen, slash, dot, tilde
safe = lambda s: re.sub(r'[^a-zA-Z0-9_/.\-~]', '', str(s))
# Coerce stop_hook_active to strict boolean string
sha = 'True' if sha_raw is True or str(sha_raw).lower() in ('true', '1', 'yes') else 'False'
print(safe(sid))
print(sha)
print(safe(tp))
" 2>/dev/null)
SESSION_ID="${_mempal_parsed[0]:-unknown}"
STOP_HOOK_ACTIVE="${_mempal_parsed[1]:-False}"
TRANSCRIPT_PATH="${_mempal_parsed[2]:-}"
# Expand ~ in path
TRANSCRIPT_PATH="${TRANSCRIPT_PATH/#\~/$HOME}"
# Validate that TRANSCRIPT_PATH looks like a transcript file:
# - non-empty
# - .jsonl or .json suffix
# - no traversal segments (.. components)
# Mirrors mempalace.hooks_cli._validate_transcript_path so the shell hook
# rejects the same shapes the Python hook rejects (#1231 review).
is_valid_transcript_path() {
local path="$1"
[ -n "$path" ] || return 1
case "$path" in
*.json|*.jsonl) ;;
*) return 1 ;;
esac
case "/$path/" in
*/../*) return 1 ;;
esac
return 0
}
# If we're already in a save cycle, let the AI stop normally
# This is the infinite-loop prevention: block once → AI saves → tries to stop again → we let it through
if [ "$STOP_HOOK_ACTIVE" = "True" ] || [ "$STOP_HOOK_ACTIVE" = "true" ]; then
echo "{}"
exit 0
fi
# Count human messages in the JSONL transcript
# SECURITY: Pass transcript path as sys.argv to avoid shell injection via crafted paths
if [ -f "$TRANSCRIPT_PATH" ]; then
EXCHANGE_COUNT=$("$MEMPAL_PYTHON_BIN" - "$TRANSCRIPT_PATH" <<'PYEOF'
import json, sys
count = 0
with open(sys.argv[1]) as f:
for line in f:
try:
entry = json.loads(line)
msg = entry.get('message', {})
if isinstance(msg, dict) and msg.get('role') == 'user':
content = msg.get('content', '')
if isinstance(content, str) and '<command-message>' in content:
continue
count += 1
except:
pass
print(count)
PYEOF
2>/dev/null)
else
EXCHANGE_COUNT=0
fi
# Track last save point for this session
LAST_SAVE_FILE="$STATE_DIR/${SESSION_ID}_last_save"
LAST_SAVE=0
if [ -f "$LAST_SAVE_FILE" ]; then
LAST_SAVE_RAW=$(cat "$LAST_SAVE_FILE")
# SECURITY: Validate as plain integer before arithmetic to prevent command injection
if [[ "$LAST_SAVE_RAW" =~ ^[0-9]+$ ]]; then
LAST_SAVE="$LAST_SAVE_RAW"
fi
fi
SINCE_LAST=$((EXCHANGE_COUNT - LAST_SAVE))
# Log for debugging (check ~/.mempalace/hook_state/hook.log)
echo "[$(date '+%H:%M:%S')] Session $SESSION_ID: $EXCHANGE_COUNT exchanges, $SINCE_LAST since last save" >> "$STATE_DIR/hook.log"
# Time to save?
if [ "$SINCE_LAST" -ge "$SAVE_INTERVAL" ] && [ "$EXCHANGE_COUNT" -gt 0 ]; then
# Update last save point
echo "$EXCHANGE_COUNT" > "$LAST_SAVE_FILE"
echo "[$(date '+%H:%M:%S')] TRIGGERING SAVE at exchange $EXCHANGE_COUNT" >> "$STATE_DIR/hook.log"
# Auto-mine. Two independent targets — both run if both are set:
# 1. TRANSCRIPT_PATH (from Claude Code) → parent dir, --mode convos
# (Claude Code session JSONL — must use the convo miner)
# 2. MEMPAL_DIR (user-configured project) → --mode projects
# (code, notes, docs)
# MEMPAL_DIR is *additive*, not an override: a user with MEMPAL_DIR
# pointed at their project still gets the active conversation mined.
if is_valid_transcript_path "$TRANSCRIPT_PATH" && [ -f "$TRANSCRIPT_PATH" ]; then
mempalace mine "$(dirname "$TRANSCRIPT_PATH")" --mode convos \
>> "$STATE_DIR/hook.log" 2>&1 &
elif [ -n "$TRANSCRIPT_PATH" ]; then
echo "[$(date '+%H:%M:%S')] Skipping invalid transcript path: $TRANSCRIPT_PATH" \
>> "$STATE_DIR/hook.log"
fi
if [ -n "$MEMPAL_DIR" ] && [ -d "$MEMPAL_DIR" ]; then
mempalace mine "$MEMPAL_DIR" --mode projects \
>> "$STATE_DIR/hook.log" 2>&1 &
fi
# MEMPAL_VERBOSE toggle:
# true = developer mode — block and show diaries/code in chat
# false = silent mode (default) — save in background, no chat clutter
# Set via: export MEMPAL_VERBOSE=true
if [ "$MEMPAL_VERBOSE" = "true" ] || [ "$MEMPAL_VERBOSE" = "1" ]; then
cat << 'HOOKJSON'
{
"decision": "block",
"reason": "MemPalace save checkpoint. Write a brief session diary entry covering key topics, decisions, and code changes since the last save. Use verbatim quotes where possible. Continue after saving."
}
HOOKJSON
else
# Silent mode: return empty JSON to not block. "decision": "allow" is
# not a valid value — only "block" or {} are recognized.
echo '{}'
fi
else
# Not time yet — let the AI stop normally
echo "{}"
fi
+170
View File
@@ -0,0 +1,170 @@
#!/bin/bash
# MEMPALACE SAVE HOOK (REMOTE) — Auto-save every N exchanges to a remote palace.
#
# Drop-in replacement for mempal_save_hook.sh when MemPalace runs on a
# server (e.g. Unraid) instead of the dev machine. Same trigger logic
# (count human messages, fire every SAVE_INTERVAL), but instead of running
# `mempalace mine` locally it POSTs the active transcript to the server's
# /ingest/transcript endpoint.
#
# Required env vars:
# MEMPAL_REMOTE_URL Base URL of the MemPalace server, e.g.
# https://unraid.local:8443
# MEMPAL_REMOTE_TOKEN Bearer token (same one configured in the server's
# .env / MEMPAL_TOKEN).
#
# Optional env vars:
# MEMPAL_REMOTE_WING Wing name to file under (defaults to the
# session-id-derived inbox name on the server).
# MEMPAL_REMOTE_INSECURE "1" to skip TLS verification — needed when
# the server uses Caddy's self-signed `tls
# internal` cert and the client hasn't trusted
# the Caddy root CA.
# SAVE_INTERVAL Override the default of 15 messages.
#
# === INSTALL ===
# Add to .claude/settings.local.json (Claude Code):
#
# "hooks": {
# "Stop": [{
# "matcher": "*",
# "hooks": [{
# "type": "command",
# "command": "/abs/path/to/mempal_save_hook_remote.sh",
# "timeout": 30
# }]
# }]
# }
#
# For Codex CLI, add the same shape to .codex/hooks.json.
set -u
SAVE_INTERVAL="${SAVE_INTERVAL:-15}"
STATE_DIR="$HOME/.mempalace/hook_state"
mkdir -p "$STATE_DIR"
# Resolve Python — used only for parsing the hook's stdin JSON.
MEMPAL_PYTHON_BIN="${MEMPAL_PYTHON:-}"
if [ -z "$MEMPAL_PYTHON_BIN" ] || [ ! -x "$MEMPAL_PYTHON_BIN" ]; then
MEMPAL_PYTHON_BIN="$(command -v python3 2>/dev/null || echo python3)"
fi
# Pre-flight: bail with a clean no-op if config is missing. Returning {}
# lets Claude Code stop normally; we log the reason for the user to find.
if [ -z "${MEMPAL_REMOTE_URL:-}" ] || [ -z "${MEMPAL_REMOTE_TOKEN:-}" ]; then
echo "[$(date '+%H:%M:%S')] MEMPAL_REMOTE_URL/TOKEN not set — skipping" \
>> "$STATE_DIR/hook.log"
echo "{}"
exit 0
fi
INPUT=$(cat)
# Parse session_id, stop_hook_active, transcript_path in one Python call —
# same sanitization shape as the local hook.
mapfile -t _mempal_parsed < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
import sys, json, re
data = json.load(sys.stdin)
sid = data.get('session_id', 'unknown')
sha_raw = data.get('stop_hook_active', False)
tp = data.get('transcript_path', '')
safe = lambda s: re.sub(r'[^a-zA-Z0-9_/.\-~]', '', str(s))
sha = 'True' if sha_raw is True or str(sha_raw).lower() in ('true', '1', 'yes') else 'False'
print(safe(sid))
print(sha)
print(safe(tp))
" 2>/dev/null)
SESSION_ID="${_mempal_parsed[0]:-unknown}"
STOP_HOOK_ACTIVE="${_mempal_parsed[1]:-False}"
TRANSCRIPT_PATH="${_mempal_parsed[2]:-}"
TRANSCRIPT_PATH="${TRANSCRIPT_PATH/#\~/$HOME}"
is_valid_transcript_path() {
local path="$1"
[ -n "$path" ] || return 1
case "$path" in
*.json|*.jsonl) ;;
*) return 1 ;;
esac
case "/$path/" in
*/../*) return 1 ;;
esac
return 0
}
if [ "$STOP_HOOK_ACTIVE" = "True" ] || [ "$STOP_HOOK_ACTIVE" = "true" ]; then
echo "{}"
exit 0
fi
# Count human messages (same logic as local hook).
if [ -f "$TRANSCRIPT_PATH" ]; then
EXCHANGE_COUNT=$("$MEMPAL_PYTHON_BIN" - "$TRANSCRIPT_PATH" <<'PYEOF'
import json, sys
count = 0
with open(sys.argv[1]) as f:
for line in f:
try:
entry = json.loads(line)
msg = entry.get('message', {})
if isinstance(msg, dict) and msg.get('role') == 'user':
content = msg.get('content', '')
if isinstance(content, str) and '<command-message>' in content:
continue
count += 1
except Exception:
pass
print(count)
PYEOF
2>/dev/null)
else
EXCHANGE_COUNT=0
fi
LAST_SAVE_FILE="$STATE_DIR/${SESSION_ID}_last_save"
LAST_SAVE=0
if [ -f "$LAST_SAVE_FILE" ]; then
LAST_SAVE_RAW=$(cat "$LAST_SAVE_FILE")
if [[ "$LAST_SAVE_RAW" =~ ^[0-9]+$ ]]; then
LAST_SAVE="$LAST_SAVE_RAW"
fi
fi
SINCE_LAST=$((EXCHANGE_COUNT - LAST_SAVE))
echo "[$(date '+%H:%M:%S')] Session $SESSION_ID: $EXCHANGE_COUNT exchanges, $SINCE_LAST since last save" \
>> "$STATE_DIR/hook.log"
if [ "$SINCE_LAST" -ge "$SAVE_INTERVAL" ] && [ "$EXCHANGE_COUNT" -gt 0 ]; then
if is_valid_transcript_path "$TRANSCRIPT_PATH" && [ -f "$TRANSCRIPT_PATH" ]; then
echo "$EXCHANGE_COUNT" > "$LAST_SAVE_FILE"
CURL_OPTS=("-sS" "--max-time" "30" "-X" "POST")
[ "${MEMPAL_REMOTE_INSECURE:-0}" = "1" ] && CURL_OPTS+=("-k")
WING_HEADER=()
[ -n "${MEMPAL_REMOTE_WING:-}" ] && WING_HEADER=(-H "X-Wing: $MEMPAL_REMOTE_WING")
# Background the upload so we don't block the AI's stop. The hook
# exits immediately with {} — the next save retry will catch any
# transient failure (the miner is idempotent server-side).
(
curl "${CURL_OPTS[@]}" \
-H "Authorization: Bearer $MEMPAL_REMOTE_TOKEN" \
-H "X-Session-Id: $SESSION_ID" \
-H "Content-Type: application/octet-stream" \
"${WING_HEADER[@]}" \
--data-binary "@$TRANSCRIPT_PATH" \
"$MEMPAL_REMOTE_URL/ingest/transcript" \
>> "$STATE_DIR/hook.log" 2>&1 \
&& echo "[$(date '+%H:%M:%S')] ingest ok" >> "$STATE_DIR/hook.log" \
|| echo "[$(date '+%H:%M:%S')] ingest failed (will retry next save)" \
>> "$STATE_DIR/hook.log"
) &
disown
elif [ -n "$TRANSCRIPT_PATH" ]; then
echo "[$(date '+%H:%M:%S')] Skipping invalid transcript path: $TRANSCRIPT_PATH" \
>> "$STATE_DIR/hook.log"
fi
fi
echo "{}"