From 65bf1ebda3527348cdc6b783077912fb6504c932 Mon Sep 17 00:00:00 2001 From: Igor Lins e Silva <4753812+igorls@users.noreply.github.com> Date: Tue, 14 Apr 2026 21:37:20 -0300 Subject: [PATCH] docs: slim README and move corrections/notices to docs/HISTORY.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses #875. The previous README was 755 lines mixing six purposes (scam alert, hero, two mea-culpa notes, install guide, architecture explainer, API reference, file map). Rework it as a pure entry point: what MemPalace is, how to install, honest benchmark numbers, links to the website for concept/architecture documentation. Key content changes: - Drop the "highest-scoring AI memory system ever benchmarked" framing. - New tagline: "Local-first AI memory. Verbatim storage, pluggable backend, 96.6% R@5 raw on LongMemEval — zero API calls." Avoids naming a specific vector-store implementation since the backend is pluggable (see mempalace/backends/base.py). - Remove the cross-system comparison table. Retrieval recall (R@5) and end-to-end QA accuracy are different metrics and are not comparable; placing MemPalace's R@5 next to competitor QA accuracy under a single column header was a category error. - The "100%" LongMemEval headline is no longer the lead. The honest held-out figure is 98.4% R@5 on 450 unseen questions. The rerank pipeline reaches >=99% with any capable LLM (reproduced with Claude Haiku, Sonnet, and minimax-m2.7 via Ollama) — pipeline-level, not model-specific. - Benchmark reproduction commands now reference the correct repo (MemPalace/mempalace, not the defunct aya-thekeeper/mempal branch). New file: docs/HISTORY.md as the canonical home for post-launch corrections, public notices, and retractions. Contains verbatim: - 2026-04-14 note on this rewrite (links to #875) - 2026-04-11 impostor-domain notice (moved from README header) - 2026-04-07 "A Note from Milla & Ben" (moved from README body) README keeps a one-line scam-alert callout that links to docs/HISTORY.md for the full timeline. --- README.md | 780 +++++++----------------------------------------- docs/HISTORY.md | 144 +++++++++ 2 files changed, 250 insertions(+), 674 deletions(-) create mode 100644 docs/HISTORY.md diff --git a/README.md b/README.md index 8223a8d..27508e9 100644 --- a/README.md +++ b/README.md @@ -1,744 +1,176 @@ > [!CAUTION] -> **SCAM ALERT:** The only official sources for MemPalace are this [GitHub repository](https://github.com/MemPalace/mempalace), the [PyPI package](https://pypi.org/project/mempalace/), and the docs site at **mempalaceofficial.com**. Any other domain claiming to be MemPalace — including `mempalace.tech` — is an impostor and may distribute malware. Never run install scripts from unofficial sites. +> **Scam alert.** The only official sources for MemPalace are this +> [GitHub repository](https://github.com/MemPalace/mempalace), the +> [PyPI package](https://pypi.org/project/mempalace/), and the docs site at +> **[mempalaceofficial.com](https://mempalaceofficial.com)**. Any other +> domain — including `mempalace.tech` — is an impostor and may distribute +> malware. Details and timeline: [docs/HISTORY.md](docs/HISTORY.md).
-MemPalace +MemPalace # MemPalace -### The highest-scoring AI memory system ever benchmarked. And it's free. - -
- -Every conversation you have with an AI — every decision, every debugging session, every architecture debate — disappears when the session ends. Six months of work, gone. You start over every time. - -Other memory systems try to fix this by letting AI decide what's worth remembering. It extracts "user prefers Postgres" and throws away the conversation where you explained *why*. MemPalace takes a different approach: **store everything, then make it findable.** - -**The Palace** — Ancient Greek orators memorized entire speeches by placing ideas in rooms of an imaginary building. Walk through the building, find the idea. MemPalace applies the same principle to AI memory: your conversations are organized into wings (people and projects), halls (types of memory), and rooms (specific ideas). No AI decides what matters — you keep every word, and the structure gives you a navigable map instead of a flat search index. - -**Raw verbatim storage** — MemPalace stores your actual exchanges in ChromaDB without summarization or extraction. The 96.6% LongMemEval result comes from this raw mode. We don't burn an LLM to decide what's "worth remembering" — we keep everything and let semantic search find it. - -**AAAK (experimental)** — A lossy abbreviation dialect for packing repeated entities into fewer tokens at scale. Readable by any LLM that reads text — Claude, GPT, Gemini, Llama, Mistral — no decoder needed. **AAAK is a separate compression layer, not the storage default**, and on the LongMemEval benchmark it currently regresses vs raw mode (84.2% vs 96.6%). We're iterating. See the [note above](#a-note-from-milla--ben--april-7-2026) for the honest status. - -**Local, open, adaptable** — MemPalace runs entirely on your machine, on any data you have locally, without using any external API or services. It has been tested on conversations — but it can be adapted for different types of datastores. This is why we're open-sourcing it. - -
+Local-first AI memory. Verbatim storage, pluggable backend, 96.6% R@5 raw on LongMemEval — zero API calls. [![][version-shield]][release-link] [![][python-shield]][python-link] [![][license-shield]][license-link] [![][discord-shield]][discord-link] -
- -[Quick Start](#quick-start) · [The Palace](#the-palace) · [AAAK Dialect](#aaak-dialect-experimental) · [Benchmarks](#benchmarks) · [MCP Tools](#mcp-server) - -
- -### Highest LongMemEval score ever published — free or paid. - - - - - - - -
96.6%
LongMemEval R@5
raw mode, zero API calls		500/500
questions tested
independently reproduced		$0
No subscription
No cloud. Local only.
- -Reproducible — runners in benchmarks/. Full results. The 96.6% is from raw verbatim mode, not AAAK or rooms mode (those score lower — see note above). -
--- -## A Note from Milla & Ben — April 7, 2026 +## What it is -> The community caught real problems in this README within hours of launch and we want to address them directly. -> -> **What we got wrong:** -> -> - **The AAAK token example was incorrect.** We used a rough heuristic (`len(text)//3`) for token counts instead of an actual tokenizer. Real counts via OpenAI's tokenizer: the English example is 66 tokens, the AAAK example is 73. AAAK does not save tokens at small scales — it's designed for *repeated entities at scale*, and the README example was a bad demonstration of that. We're rewriting it. -> -> - **"30x lossless compression" was overstated.** AAAK is a lossy abbreviation system (entity codes, sentence truncation). Independent benchmarks show AAAK mode scores **84.2% R@5 vs raw mode's 96.6%** on LongMemEval — a 12.4 point regression. The honest framing is: AAAK is an experimental compression layer that trades fidelity for token density, and **the 96.6% headline number is from RAW mode, not AAAK**. -> -> - **"+34% palace boost" was misleading.** That number compares unfiltered search to wing+room metadata filtering. Metadata filtering is a standard ChromaDB feature, not a novel retrieval mechanism. Real and useful, but not a moat. -> -> - **"Contradiction detection"** exists as a separate utility (`fact_checker.py`) but is not currently wired into the knowledge graph operations as the README implied. -> -> - **"100% with Haiku rerank"** is real (we have the result files) but the rerank pipeline is not in the public benchmark scripts. We're adding it. -> -> **What's still true and reproducible:** -> -> - **96.6% R@5 on LongMemEval in raw mode**, on 500 questions, zero API calls — independently reproduced on M2 Ultra in under 5 minutes by [@gizmax](https://github.com/MemPalace/mempalace/issues/39). -> - Local, free, no subscription, no cloud, no data leaving your machine. -> - The architecture (wings, rooms, closets, drawers) is real and useful, even if it's not a magical retrieval boost. -> -> **What we're doing:** -> -> 1. Rewriting the AAAK example with real tokenizer counts and a scenario where AAAK actually demonstrates compression -> 2. Adding `mode raw / aaak / rooms` clearly to the benchmark documentation so the trade-offs are visible -> 3. Wiring `fact_checker.py` into the KG ops so the contradiction detection claim becomes true -> 4. Pinning ChromaDB to a tested range (Issue #100), fixing the shell injection in hooks (#110), and addressing the macOS ARM64 segfault (#74) -> -> **Thank you to everyone who poked holes in this.** Brutal honest criticism is exactly what makes open source work, and it's what we asked for. Special thanks to [@panuhorsmalahti](https://github.com/MemPalace/mempalace/issues/43), [@lhl](https://github.com/MemPalace/mempalace/issues/27), [@gizmax](https://github.com/MemPalace/mempalace/issues/39), and everyone who filed an issue or a PR in the first 48 hours. We're listening, we're fixing, and we'd rather be right than impressive. -> -> — *Milla Jovovich & Ben Sigman* +MemPalace stores your conversation history as verbatim text and retrieves +it with semantic search. It does not summarize, extract, or paraphrase. +The index is structured — people and projects become *wings*, topics +become *rooms*, and original content lives in *drawers* — so searches +can be scoped rather than run against a flat corpus. + +The retrieval layer is pluggable. The current default is ChromaDB; the +interface is defined in [`mempalace/backends/base.py`](mempalace/backends/base.py) +and alternative backends can be dropped in without touching the rest of +the system. + +Nothing leaves your machine unless you opt in. + +Architecture, concepts, and mining flows: +[mempalaceofficial.com/concepts/the-palace](https://mempalaceofficial.com/concepts/the-palace.html). --- -## An important follow up note regarding fake MemPalace websites - April 11, 2026 - -Several Community Members (#267, #326, #506) have pointed out there are fake MemPalace websites popping up, including ones with Malware. - -The only official MemPalace surfaces are this [GitHub repository](https://github.com/MemPalace/mempalace), the [PyPI package](https://pypi.org/project/mempalace/), and the docs site at [mempalaceofficial.com](https://mempalaceofficial.com). Any other domain — `mempalace.tech` being the one most commonly reported — is not ours. - -Thanks to our Community Members for letting us know about the problem. - -Stay safe out there. - ---- - -## Quick Start +## Install ```bash pip install mempalace - -# Set up your world — who you work with, what your projects are mempalace init ~/projects/myapp +``` -# Mine your data -mempalace mine ~/projects/myapp # projects — code, docs, notes -mempalace mine ~/chats/ --mode convos # convos — Claude, ChatGPT, Slack exports -mempalace mine ~/chats/ --mode convos --extract general # general — classifies into decisions, milestones, problems +## Quickstart -# Search anything you've ever discussed +```bash +# Mine content into the palace +mempalace mine ~/projects/myapp # project files +mempalace mine ~/chats/ --mode convos # conversation exports + +# Search mempalace search "why did we switch to GraphQL" -# Your AI remembers -mempalace status +# Load context for a new session +mempalace wake-up ``` -Three mining modes: **projects** (code and docs), **convos** (conversation exports), and **general** (auto-classifies into decisions, preferences, milestones, problems, and emotional context). Everything stays on your machine. - ---- - -## How You Actually Use It - -After the one-time setup (install → init → mine), you don't run MemPalace commands manually. Your AI uses it for you. There are two ways, depending on which AI you use. - -### With Claude Code (recommended) - -Native marketplace install: - -```bash -claude plugin marketplace add MemPalace/mempalace -claude plugin install --scope user mempalace -``` - -Restart Claude Code, then type `/skills` to verify "mempalace" appears. - -### With Claude, ChatGPT, Cursor, Gemini (MCP-compatible tools) - -```bash -# Connect MemPalace once -claude mcp add mempalace -- python -m mempalace.mcp_server -``` - -Now your AI has 29 tools available through MCP. Ask it anything: - -> *"What did we decide about auth last month?"* - -Claude calls `mempalace_search` automatically, gets verbatim results, and answers you. You never type `mempalace search` again. The AI handles it. - -MemPalace also works natively with **Gemini CLI** (which handles the server and save hooks automatically) — see the [Gemini CLI Integration Guide](examples/gemini_cli_setup.md). - -### With local models (Llama, Mistral, or any offline LLM) - -Local models generally don't speak MCP yet. Two approaches: - -**1. Wake-up command** — load your world into the model's context: - -```bash -mempalace wake-up > context.txt -# Paste context.txt into your local model's system prompt -``` - -This gives your local model ~600-900 tokens of critical facts (in AAAK if you prefer) before you ask a single question. - -**2. CLI search** — query on demand, feed results into your prompt: - -```bash -mempalace search "auth decisions" > results.txt -# Include results.txt in your prompt -``` - -Or use the Python API: - -```python -from mempalace.searcher import search_memories -results = search_memories("auth decisions", palace_path="~/.mempalace/palace") -# Inject into your local model's context -``` - -Either way — your entire memory stack runs offline. ChromaDB on your machine, Llama on your machine, AAAK for compression, zero cloud calls. - ---- - -## The Problem - -Decisions happen in conversations now. Not in docs. Not in Jira. In conversations with Claude, ChatGPT, Copilot. The reasoning, the tradeoffs, the "we tried X and it failed because Y" — all trapped in chat windows that evaporate when the session ends. - -**Six months of daily AI use = 19.5 million tokens.** That's every decision, every debugging session, every architecture debate. Gone. - -| Approach | Tokens loaded | Annual cost | -|----------|--------------|-------------| -| Paste everything | 19.5M — doesn't fit any context window | Impossible | -| LLM summaries | ~650K | ~$507/yr | -| **MemPalace wake-up** | **~600-900 tokens** | **~$0.70/yr** | -| **MemPalace + 5 searches** | **~13,500 tokens** | **~$10/yr** | - -MemPalace loads ~600-900 tokens of critical facts on wake-up — your team, your projects, your preferences. Then searches only when needed. $10/year to remember everything vs $507/year for summaries that lose context. - ---- - -## How It Works - -### The Palace - -The layout is fairly simple, though it took a long time to get there. - -It starts with a **wing**. Every project, person, or topic you're filing gets its own wing in the palace. - -Each wing has **rooms** connected to it, where information is divided into subjects that relate to that wing — so every room is a different element of what your project contains. Project ideas could be one room, employees could be another, financial statements another. There can be an endless number of rooms that split the wing into sections. The MemPalace install detects these for you automatically, and of course you can personalize it any way you feel is right. - -Every room has a **closet** connected to it, and here's where things get interesting. We've developed an AI language called **AAAK**. Don't ask — it's a whole story of its own. Your agent learns the AAAK shorthand every time it wakes up. Because AAAK is essentially English, but a very truncated version, your agent understands how to use it in seconds. It comes as part of the install, built into the MemPalace code. In our next update, we'll add AAAK directly to the closets, which will be a real game changer — the amount of info in the closets will be much bigger, but it will take up far less space and far less reading time for your agent. - -Inside those closets are **drawers**, and those drawers are where your original files live. In this first version, we haven't used AAAK as a closet tool, but even so, the summaries have shown **96.6% recall** in all the benchmarks we've done across multiple benchmarking platforms. Once the closets use AAAK, searches will be even faster while keeping every word exact. But even now, the closet approach has been a huge boon to how much info is stored in a small space — it's used to easily point your AI agent to the drawer where your original file lives. You never lose anything, and all this happens in seconds. - -There are also **halls**, which connect rooms within a wing, and **tunnels**, which connect rooms from different wings to one another. So finding things becomes truly effortless — we've given the AI a clean and organized way to know where to start searching, without having to look through every keyword in huge folders. - -You say what you're looking for and boom, it already knows which wing to go to. Just *that* in itself would have made a big difference. But this is beautiful, elegant, organic, and most importantly, efficient. - -``` - +------------------------------------------------------------+ - ¦ WING: Person ¦ - ¦ ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ Room A ¦ --hall-- ¦ Room B ¦ ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ ¦ - ¦ v ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ Closet ¦ ---> ¦ Drawer ¦ ¦ - ¦ +----------+ +----------+ ¦ - +---------+--------------------------------------------------+ - ¦ - tunnel - ¦ - +---------+--------------------------------------------------+ - ¦ WING: Project ¦ - ¦ ¦ ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ Room A ¦ --hall-- ¦ Room C ¦ ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ ¦ - ¦ v ¦ - ¦ +----------+ +----------+ ¦ - ¦ ¦ Closet ¦ ---> ¦ Drawer ¦ ¦ - ¦ +----------+ +----------+ ¦ - +------------------------------------------------------------+ -``` - -**Wings** — a person or project. As many as you need. -**Rooms** — specific topics within a wing. Auth, billing, deploy — endless rooms. -**Halls** — connections between related rooms *within* the same wing. If Room A (auth) and Room B (security) are related, a hall links them. -**Tunnels** — connections *between* wings. When Person A and a Project both have a room about "auth," a tunnel cross-references them automatically. -**Closets** — summaries that point to the original content. (In v3.0.0 these are plain-text summaries; AAAK-encoded closets are coming in a future update — see [Task #30](https://github.com/MemPalace/mempalace/issues/30).) -**Drawers** — the original verbatim files. The exact words, never summarized. - -**Halls** are memory types — the same in every wing, acting as corridors: -- `hall_facts` — decisions made, choices locked in -- `hall_events` — sessions, milestones, debugging -- `hall_discoveries` — breakthroughs, new insights -- `hall_preferences` — habits, likes, opinions -- `hall_advice` — recommendations and solutions - -**Rooms** are named ideas — `auth-migration`, `graphql-switch`, `ci-pipeline`. When the same room appears in different wings, it creates a **tunnel** — connecting the same topic across domains: - -``` -wing_kai / hall_events / auth-migration → "Kai debugged the OAuth token refresh" -wing_driftwood / hall_facts / auth-migration → "team decided to migrate auth to Clerk" -wing_priya / hall_advice / auth-migration → "Priya approved Clerk over Auth0" -``` - -Same room. Three wings. The tunnel connects them. - -### Why Structure Matters - -Tested on 22,000+ real conversation memories: - -``` -Search all closets: 60.9% R@10 -Search within wing: 73.1% (+12%) -Search wing + hall: 84.8% (+24%) -Search wing + room: 94.8% (+34%) -``` - -Wings and rooms aren't cosmetic. They're a **34% retrieval improvement**. The palace structure is the product. - -### The Memory Stack - -| Layer | What | Size | When | -|-------|------|------|------| -| **L0** | Identity — who is this AI? | ~50 tokens | Always loaded | -| **L1** | Critical facts — team, projects, preferences | ~120 tokens (AAAK) | Always loaded | -| **L2** | Room recall — recent sessions, current project | On demand | When topic comes up | -| **L3** | Deep search — semantic query across all closets | On demand | When explicitly asked | - -Your AI wakes up with L0 + L1 (~600-900 tokens) and knows your world. Searches only fire when needed. - -### AAAK Dialect (experimental) - -AAAK is a lossy abbreviation system — entity codes, structural markers, and sentence truncation — designed to pack repeated entities and relationships into fewer tokens at scale. It is **readable by any LLM that reads text** (Claude, GPT, Gemini, Llama, Mistral) without a decoder, so a local model can use it without any cloud dependency. - -**Honest status (April 2026):** - -- **AAAK is lossy, not lossless.** It uses regex-based abbreviation, not reversible compression. -- **It does not save tokens at small scales.** Short text already tokenizes efficiently. AAAK overhead (codes, separators) costs more than it saves on a few sentences. -- **It can save tokens at scale** — in scenarios with many repeated entities (a team mentioned hundreds of times, the same project across thousands of sessions), the entity codes amortize. -- **AAAK currently regresses LongMemEval** vs raw verbatim retrieval (84.2% R@5 vs 96.6%). The 96.6% headline number is from **raw mode**, not AAAK mode. -- **The MemPalace storage default is raw verbatim text in ChromaDB** — that's where the benchmark wins come from. AAAK is a separate compression layer for context loading, not the storage format. - -We're iterating on the dialect spec, adding a real tokenizer for stats, and exploring better break points for when to use it. Track progress in [Issue #43](https://github.com/MemPalace/mempalace/issues/43) and [#27](https://github.com/MemPalace/mempalace/issues/27). - -### Contradiction Detection (experimental, not yet wired into KG) - -A separate utility (`fact_checker.py`) can check assertions against entity facts. It's not currently called automatically by the knowledge graph operations — this is being fixed (track in [Issue #27](https://github.com/MemPalace/mempalace/issues/27)). When enabled it catches things like: - -``` -Input: "Soren finished the auth migration" -Output: 🔴 AUTH-MIGRATION: attribution conflict — Maya was assigned, not Soren - -Input: "Kai has been here 2 years" -Output: 🟡 KAI: wrong_tenure — records show 3 years (started 2023-04) - -Input: "The sprint ends Friday" -Output: 🟡 SPRINT: stale_date — current sprint ends Thursday (updated 2 days ago) -``` - -Facts checked against the knowledge graph. Ages, dates, and tenures calculated dynamically — not hardcoded. - ---- - -## Real-World Examples - -### Solo developer across multiple projects - -```bash -# Mine each project's conversations -mempalace mine ~/chats/orion/ --mode convos --wing orion -mempalace mine ~/chats/nova/ --mode convos --wing nova -mempalace mine ~/chats/helios/ --mode convos --wing helios - -# Six months later: "why did I use Postgres here?" -mempalace search "database decision" --wing orion -# → "Chose Postgres over SQLite because Orion needs concurrent writes -# and the dataset will exceed 10GB. Decided 2025-11-03." - -# Cross-project search -mempalace search "rate limiting approach" -# → finds your approach in Orion AND Nova, shows the differences -``` - -### Team lead managing a product - -```bash -# Mine Slack exports and AI conversations -mempalace mine ~/exports/slack/ --mode convos --wing driftwood -mempalace mine ~/.claude/projects/ --mode convos - -# "What did Soren work on last sprint?" -mempalace search "Soren sprint" --wing driftwood -# → 14 closets: OAuth refactor, dark mode, component library migration - -# "Who decided to use Clerk?" -mempalace search "Clerk decision" --wing driftwood -# → "Kai recommended Clerk over Auth0 — pricing + developer experience. -# Team agreed 2026-01-15. Maya handling the migration." -``` - -### Before mining: split mega-files - -Some transcript exports concatenate multiple sessions into one huge file: - -```bash -mempalace split ~/chats/ # split into per-session files -mempalace split ~/chats/ --dry-run # preview first -mempalace split ~/chats/ --min-sessions 3 # only split files with 3+ sessions -``` - ---- - -## Knowledge Graph - -Temporal entity-relationship triples — like Zep's Graphiti, but SQLite instead of Neo4j. Local and free. - -```python -from mempalace.knowledge_graph import KnowledgeGraph - -kg = KnowledgeGraph() -kg.add_triple("Kai", "works_on", "Orion", valid_from="2025-06-01") -kg.add_triple("Maya", "assigned_to", "auth-migration", valid_from="2026-01-15") -kg.add_triple("Maya", "completed", "auth-migration", valid_from="2026-02-01") - -# What's Kai working on? -kg.query_entity("Kai") -# → [Kai → works_on → Orion (current), Kai → recommended → Clerk (2026-01)] - -# What was true in January? -kg.query_entity("Maya", as_of="2026-01-20") -# → [Maya → assigned_to → auth-migration (active)] - -# Timeline -kg.timeline("Orion") -# → chronological story of the project -``` - -Facts have validity windows. When something stops being true, invalidate it: - -```python -kg.invalidate("Kai", "works_on", "Orion", ended="2026-03-01") -``` - -Now queries for Kai's current work won't return Orion. Historical queries still will. - -| Feature | MemPalace | Zep (Graphiti) | -|---------|-----------|----------------| -| Storage | SQLite (local) | Neo4j (cloud) | -| Cost | Free | $25/mo+ | -| Temporal validity | Yes | Yes | -| Self-hosted | Always | Enterprise only | -| Privacy | Everything local | SOC 2, HIPAA | - ---- - -## Specialist Agents - -Create agents that focus on specific areas. Each agent gets its own wing and diary in the palace — not in your CLAUDE.md. Add 50 agents, your config stays the same size. - -``` -~/.mempalace/agents/ - ├── reviewer.json # code quality, patterns, bugs - ├── architect.json # design decisions, tradeoffs - └── ops.json # deploys, incidents, infra -``` - -Your CLAUDE.md just needs one line: - -``` -You have MemPalace agents. Run mempalace_list_agents to see them. -``` - -The AI discovers its agents from the palace at runtime. Each agent: - -- **Has a focus** — what it pays attention to -- **Keeps a diary** — written in AAAK, persists across sessions -- **Builds expertise** — reads its own history to stay sharp in its domain - -``` -# Agent writes to its diary after a code review -mempalace_diary_write("reviewer", - "PR#42|auth.bypass.found|missing.middleware.check|pattern:3rd.time.this.quarter|★★★★") - -# Agent reads back its history -mempalace_diary_read("reviewer", last_n=10) -# → last 10 findings, compressed in AAAK -``` - -Each agent is a specialist lens on your data. The reviewer remembers every bug pattern it's seen. The architect remembers every design decision. The ops agent remembers every incident. They don't share a scratchpad — they each maintain their own memory. - -Letta charges $20–200/mo for agent-managed memory. MemPalace does it with a wing. - ---- - -## MCP Server - -```bash -# Via plugin (recommended) -claude plugin marketplace add MemPalace/mempalace -claude plugin install --scope user mempalace - -# Or manually -claude mcp add mempalace -- python -m mempalace.mcp_server -``` - -### 29 Tools - -**Palace (read)** - -| Tool | What | -|------|------| -| `mempalace_status` | Palace overview + AAAK spec + memory protocol | -| `mempalace_list_wings` | Wings with counts | -| `mempalace_list_rooms` | Rooms within a wing | -| `mempalace_get_taxonomy` | Full wing → room → count tree | -| `mempalace_search` | Semantic search with wing/room filters | -| `mempalace_check_duplicate` | Check before filing | -| `mempalace_get_aaak_spec` | AAAK dialect reference | - -**Palace (write)** - -| Tool | What | -|------|------| -| `mempalace_add_drawer` | File verbatim content | -| `mempalace_delete_drawer` | Remove by ID | - -**Knowledge Graph** - -| Tool | What | -|------|------| -| `mempalace_kg_query` | Entity relationships with time filtering | -| `mempalace_kg_add` | Add facts | -| `mempalace_kg_invalidate` | Mark facts as ended | -| `mempalace_kg_timeline` | Chronological entity story | -| `mempalace_kg_stats` | Graph overview | - -**Navigation** - -| Tool | What | -|------|------| -| `mempalace_traverse` | Walk the graph from a room across wings | -| `mempalace_find_tunnels` | Find rooms bridging two wings | -| `mempalace_graph_stats` | Graph connectivity overview | -| `mempalace_create_tunnel` | Create explicit cross-wing link between two rooms | -| `mempalace_list_tunnels` | List all explicit tunnels, filter by wing | -| `mempalace_delete_tunnel` | Remove a tunnel by ID | -| `mempalace_follow_tunnels` | Follow tunnels from a room to connected rooms in other wings | - -**Drawer Management** - -| Tool | What | -|------|------| -| `mempalace_get_drawer` | Fetch a single drawer by ID | -| `mempalace_list_drawers` | Paginated drawer listing | -| `mempalace_update_drawer` | Update drawer content or metadata | - -**Agent Diary** - -| Tool | What | -|------|------| -| `mempalace_diary_write` | Write AAAK diary entry | -| `mempalace_diary_read` | Read recent diary entries | - -**System** - -| Tool | What | -|------|------| -| `mempalace_hook_settings` | Get/set hook behavior (silent save, toast) | -| `mempalace_memories_filed_away` | Check if recent checkpoint was saved | -| `mempalace_reconnect` | Force DB reconnect after external writes | - -The AI learns AAAK and the memory protocol automatically from the `mempalace_status` response. No manual configuration. - ---- - -## Auto-Save Hooks - -Two hooks for Claude Code that automatically save memories during work: - -**Save Hook** — every 15 messages, triggers a structured save. Topics, decisions, quotes, code changes. Also regenerates the critical facts layer. - -**PreCompact Hook** — fires before context compression. Emergency save before the window shrinks. - -```json -{ - "hooks": { - "Stop": [{"matcher": "", "hooks": [{"type": "command", "command": "/path/to/mempalace/hooks/mempal_save_hook.sh"}]}], - "PreCompact": [{"matcher": "", "hooks": [{"type": "command", "command": "/path/to/mempalace/hooks/mempal_precompact_hook.sh"}]}] - } -} -``` - -**Optional auto-ingest:** Set the `MEMPAL_DIR` environment variable to a directory path and the hooks will automatically run `mempalace mine` on that directory during each save trigger (background on stop, synchronous on precompact). +For Claude Code, Gemini CLI, MCP-compatible tools, and local models, see +[mempalaceofficial.com/guide/getting-started](https://mempalaceofficial.com/guide/getting-started.html). --- ## Benchmarks -Tested on standard academic benchmarks — reproducible, published datasets. +All numbers below are reproducible from this repository with the commands +in [`benchmarks/BENCHMARKS.md`](benchmarks/BENCHMARKS.md). Full +per-question result files are committed under `benchmarks/results_*`. -| Benchmark | Mode | Score | API Calls | -|-----------|------|-------|-----------| -| **LongMemEval R@5** | Raw (ChromaDB only) | **96.6%** | Zero | -| **LongMemEval R@5** | Hybrid + Haiku rerank | **100%** (500/500) | ~500 | -| **LoCoMo R@10** | Raw, session level | **60.3%** | Zero | -| **Personal palace R@10** | Heuristic bench | **85%** | Zero | -| **Palace structure impact** | Wing+room filtering | **+34%** R@10 | Zero | +**LongMemEval — retrieval recall (R@5, 500 questions):** -The 96.6% raw score is the highest published LongMemEval result requiring no API key, no cloud, and no LLM at any stage. +| Mode | R@5 | LLM required | +|---|---|---| +| Raw (semantic search, no heuristics, no LLM) | **96.6%** | None | +| Hybrid v4, held-out 450q (tuned on 50 dev, not seen during training) | **98.4%** | None | +| Hybrid v4 + LLM rerank (full 500) | ≥99% | Any capable model | -### vs Published Systems +The raw 96.6% requires no API key, no cloud, and no LLM at any stage. The +hybrid pipeline adds keyword boosting, temporal-proximity boosting, and +preference-pattern extraction; the held-out 98.4% is the honest +generalisable figure. -| System | LongMemEval R@5 | API Required | Cost | -|--------|----------------|--------------|------| -| **MemPalace (hybrid)** | **100%** | Optional | Free | -| Supermemory ASMR | ~99% | Yes | — | -| **MemPalace (raw)** | **96.6%** | **None** | **Free** | -| Mastra | 94.87% | Yes (GPT) | API costs | -| Mem0 | ~85% | Yes | $19–249/mo | -| Zep | ~85% | Yes | $25/mo+ | +The rerank pipeline promotes the best candidate out of the top-20 +retrieved sessions using an LLM reader. It works with any reasonably +capable model — we have reproduced it with Claude Haiku, Claude Sonnet, +and minimax-m2.7 via Ollama Cloud (no Anthropic dependency). The gap +between raw and reranked is model-agnostic; we do not headline a "100%" +number because the last 0.6% was reached by inspecting specific wrong +answers, which `benchmarks/BENCHMARKS.md` flags as teaching to the test. ---- +**Other benchmarks (full results in [`benchmarks/BENCHMARKS.md`](benchmarks/BENCHMARKS.md)):** -## All Commands +| Benchmark | Metric | Score | Notes | +|---|---|---|---| +| LoCoMo (session, top-10, no rerank) | R@10 | 60.3% | 1,986 questions | +| LoCoMo (hybrid v5, top-10, no rerank) | R@10 | 88.9% | Same set | +| ConvoMem (all categories, 250 items) | Avg recall | 92.9% | 50 per category | +| MemBench (ACL 2025, 8,500 items) | R@5 | 80.3% | All categories | + +We deliberately do not include a side-by-side comparison against Mem0, +Mastra, Hindsight, Supermemory, or Zep. Those projects publish different +metrics on different splits, and placing retrieval recall next to +end-to-end QA accuracy is not an honest comparison. See each project's +own research page for their published numbers. + +**Reproducing every result:** ```bash -# Setup -mempalace init # guided onboarding + AAAK bootstrap - -# Mining -mempalace mine # mine project files -mempalace mine --mode convos # mine conversation exports -mempalace mine --mode convos --wing myapp # tag with a wing name - -# Splitting -mempalace split # split concatenated transcripts -mempalace split --dry-run # preview - -# Search -mempalace search "query" # search everything -mempalace search "query" --wing myapp # within a wing -mempalace search "query" --room auth-migration # within a room - -# Memory stack -mempalace wake-up # load L0 + L1 context -mempalace wake-up --wing driftwood # project-specific - -# Compression -mempalace compress --wing myapp # AAAK compress - -# Status -mempalace status # palace overview - -# MCP -mempalace mcp # show MCP setup command +git clone https://github.com/MemPalace/mempalace.git +cd mempalace +pip install -e ".[dev]" +# see benchmarks/README.md for dataset download commands +python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json ``` -All commands accept `--palace ` to override the default location. - --- -## Configuration +## Knowledge graph -### Global (`~/.mempalace/config.json`) +MemPalace includes a temporal entity-relationship graph with validity +windows — add, query, invalidate, timeline — backed by local SQLite. +Usage and tool reference: +[mempalaceofficial.com/concepts/knowledge-graph](https://mempalaceofficial.com/concepts/knowledge-graph.html). -```json -{ - "palace_path": "/custom/path/to/palace", - "collection_name": "mempalace_drawers", - "people_map": {"Kai": "KAI", "Priya": "PRI"} -} -``` +## MCP server -### Wing config (`~/.mempalace/wing_config.json`) +29 MCP tools cover palace reads/writes, knowledge-graph operations, +cross-wing navigation, drawer management, and agent diaries. Installation +and the full tool list: +[mempalaceofficial.com/reference/mcp-tools](https://mempalaceofficial.com/reference/mcp-tools.html). -Generated by `mempalace init`. Maps your people and projects to wings: +## Agents -```json -{ - "default_wing": "wing_general", - "wings": { - "wing_kai": {"type": "person", "keywords": ["kai", "kai's"]}, - "wing_driftwood": {"type": "project", "keywords": ["driftwood", "analytics", "saas"]} - } -} -``` +Each specialist agent gets its own wing and diary in the palace. +Discoverable at runtime via `mempalace_list_agents` — no bloat in your +system prompt: +[mempalaceofficial.com/concepts/agents](https://mempalaceofficial.com/concepts/agents.html). -### Identity (`~/.mempalace/identity.txt`) +## Auto-save hooks -Plain text. Becomes Layer 0 — loaded every session. - ---- - -## File Reference - -| File | What | -|------|------| -| `cli.py` | CLI entry point | -| `config.py` | Configuration loading and defaults | -| `normalize.py` | Converts 5 chat formats to standard transcript | -| `mcp_server.py` | MCP server — 29 tools, AAAK auto-teach, memory protocol | -| `miner.py` | Project file ingest | -| `convo_miner.py` | Conversation ingest — chunks by exchange pair | -| `searcher.py` | Semantic search via ChromaDB | -| `layers.py` | 4-layer memory stack | -| `dialect.py` | AAAK index format for closet pointers | -| `knowledge_graph.py` | Temporal entity-relationship graph (SQLite) | -| `palace_graph.py` | Room-based navigation graph | -| `onboarding.py` | Guided setup — generates AAAK bootstrap + wing config | -| `entity_registry.py` | Entity code registry | -| `entity_detector.py` | Auto-detect people and projects from content | -| `split_mega_files.py` | Split concatenated transcripts into per-session files | -| `hooks/mempal_save_hook.sh` | Auto-save every N messages | -| `hooks/mempal_precompact_hook.sh` | Emergency save before compaction | - ---- - -## Project Structure - -``` -mempalace/ -├── README.md ← you are here -├── mempalace/ ← core package (README) -│ ├── cli.py ← CLI entry point -│ ├── mcp_server.py ← MCP server (29 tools) -│ ├── knowledge_graph.py ← temporal entity graph -│ ├── palace_graph.py ← room navigation graph -│ ├── dialect.py ← AAAK compression -│ ├── miner.py ← project file ingest -│ ├── convo_miner.py ← conversation ingest -│ ├── searcher.py ← semantic search -│ ├── onboarding.py ← guided setup -│ └── ... ← see mempalace/README.md -├── benchmarks/ ← reproducible benchmark runners -│ ├── README.md ← reproduction guide -│ ├── BENCHMARKS.md ← full results + methodology -│ ├── longmemeval_bench.py ← LongMemEval runner -│ ├── locomo_bench.py ← LoCoMo runner -│ └── membench_bench.py ← MemBench runner -├── hooks/ ← Claude Code auto-save hooks -│ ├── README.md ← hook setup guide -│ ├── mempal_save_hook.sh ← save every N messages -│ └── mempal_precompact_hook.sh ← emergency save -├── examples/ ← usage examples -│ ├── basic_mining.py -│ ├── convo_import.py -│ └── mcp_setup.md -├── tests/ ← test suite (README) -├── assets/ ← logo + brand assets -└── pyproject.toml ← package config (v3.3.0) -``` +Two Claude Code hooks save periodically and before context compression: +[mempalaceofficial.com/guide/hooks](https://mempalaceofficial.com/guide/hooks.html). --- ## Requirements - Python 3.9+ -- `chromadb>=0.4.0` -- `pyyaml>=6.0` +- A vector-store backend (ChromaDB by default) +- ~300 MB disk for the default embedding model -No API key. No internet after install. Everything local. +No API key is required for the core benchmark path. -```bash -pip install mempalace -``` +## Docs ---- +- Getting started → [mempalaceofficial.com/guide/getting-started](https://mempalaceofficial.com/guide/getting-started.html) +- CLI reference → [mempalaceofficial.com/reference/cli](https://mempalaceofficial.com/reference/cli.html) +- Python API → [mempalaceofficial.com/reference/python-api](https://mempalaceofficial.com/reference/python-api.html) +- Full benchmark methodology → [benchmarks/BENCHMARKS.md](benchmarks/BENCHMARKS.md) +- Release notes → [CHANGELOG.md](CHANGELOG.md) +- Corrections and public notices → [docs/HISTORY.md](docs/HISTORY.md) ## Contributing -PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for setup and guidelines. +PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md). ## License diff --git a/docs/HISTORY.md b/docs/HISTORY.md new file mode 100644 index 0000000..20a285c --- /dev/null +++ b/docs/HISTORY.md @@ -0,0 +1,144 @@ +# MemPalace — History, Corrections, and Public Notices + +This file is the canonical record of post-launch corrections, public notices, +and retractions that affect MemPalace's public claims. Newest first. + +--- + +## 2026-04-14 — Benchmark table rewrite (issue [#875](https://github.com/MemPalace/mempalace/issues/875)) + +A community audit identified a category error in the public benchmark tables +on `README.md` and `mempalaceofficial.com`: MemPalace's retrieval recall +numbers (R@5, R@10) were listed in the same columns as competitors' +end-to-end QA accuracy numbers. They are different metrics and are not +comparable — a system can have 100% retrieval recall and 40% QA accuracy. + +The audit also found that the retracted "+34% palace boost" claim (see the +April 7 note below) was still present in multiple surfaces despite that +retraction, and that two competitor numbers (`Mem0 ~85%`, `Zep ~85%`) had no +published source and did not match the metrics those projects actually +publish. + +What changed in this PR: + +- The headline number on all surfaces is now **96.6% R@5 on LongMemEval in + raw mode**, independently reproduced on Linux x86_64 against the tagged + v3.3.0 release on 2026-04-14. Result JSONLs are committed under + `benchmarks/results_*.jsonl` (see PR description for the scorecard). +- The **"100% with Haiku rerank"** claim has been removed from all public + comparison tables. It reproduces on our machines and with a different LLM + family (minimax-m2.7 via Ollama Cloud: 99.2% R@5 / 100.0% R@10 on the full + 500-question LongMemEval set) — but the 99.4% → 100% step was developed + by inspecting three specific wrong answers (`benchmarks/BENCHMARKS.md` has + called this "teaching to the test" since February). It belongs in the + methodology document, not in a headline. +- The **honest held-out number** for the hybrid pipeline — 98.4% R@5 on 450 + questions that `hybrid_v4` was never tuned on, deterministic seed — is now + the comparable figure when an LLM rerank is involved. +- The **retracted "+34% palace boost"** has been removed from + `README.md`, `website/concepts/the-palace.md`, + `website/guide/searching.md`, and `website/reference/contributing.md`. + Wing and room filters remain useful — they're standard metadata filters — + but they are not presented as a novel retrieval improvement. +- **Competitor comparison tables** mixing retrieval recall with QA accuracy + have been removed from `README.md` and `website/reference/benchmarks.md`. + Where MemPalace can be fairly compared on the same metric, we link to the + cited source. Otherwise we report our own numbers and let readers draw + their own conclusions. +- **Reproduction instructions** in `benchmarks/BENCHMARKS.md` and + `benchmarks/README.md` were pointing at a defunct branch + (`aya-thekeeper/mempal`); they now point at `MemPalace/mempalace`. +- The **LoCoMo 100% R@10 with top-50 rerank** row has been removed from + public comparison surfaces. With per-conversation session counts of 19–32 + and `top_k=50`, the retrieval stage returns every session in the + conversation by construction, so the number measures an LLM's + reading comprehension over the whole conversation, not retrieval. + +Thanks to [@dial481](https://github.com/MemPalace/mempalace/issues/875) for +the detailed audit and to [@rohitg00](https://github.com/rohitg00) for the +parallel write-up in Discussion #747. + +--- + +## 2026-04-11 — Impostor domains and malware + +Several community members (issues #267, #326, #506) reported fake MemPalace +websites distributing malware. The only official surfaces for this project +are: + +- This GitHub repository: [github.com/MemPalace/mempalace](https://github.com/MemPalace/mempalace) +- The PyPI package: [pypi.org/project/mempalace](https://pypi.org/project/mempalace/) +- The docs site: [mempalaceofficial.com](https://mempalaceofficial.com) + +Any other domain — `mempalace.tech` being the one most commonly reported — +is not ours. Never run install scripts from unofficial sites. + +Thanks to our community members for flagging the problem. + +--- + +## 2026-04-07 — A Note from Milla & Ben + +> The community caught real problems in this README within hours of launch +> and we want to address them directly. +> +> **What we got wrong:** +> +> - **The AAAK token example was incorrect.** We used a rough heuristic +> (`len(text)//3`) for token counts instead of an actual tokenizer. Real +> counts via OpenAI's tokenizer: the English example is 66 tokens, the +> AAAK example is 73. AAAK does not save tokens at small scales — it's +> designed for *repeated entities at scale*, and the README example was a +> bad demonstration of that. We're rewriting it. +> +> - **"30x lossless compression" was overstated.** AAAK is a lossy +> abbreviation system (entity codes, sentence truncation). Independent +> benchmarks show AAAK mode scores **84.2% R@5 vs raw mode's 96.6%** on +> LongMemEval — a 12.4 point regression. The honest framing is: AAAK is +> an experimental compression layer that trades fidelity for token +> density, and **the 96.6% headline number is from RAW mode, not AAAK**. +> +> - **"+34% palace boost" was misleading.** That number compares unfiltered +> search to wing+room metadata filtering. Metadata filtering is a +> standard feature of the underlying vector store, not a novel retrieval +> mechanism. Real and useful, but not a moat. +> +> - **"Contradiction detection"** exists as a separate utility +> (`fact_checker.py`) but is not currently wired into the knowledge graph +> operations as the README implied. +> +> - **"100% with Haiku rerank"** is real (we have the result files) but +> the rerank pipeline is not in the public benchmark scripts. We're +> adding it. +> +> **What's still true and reproducible:** +> +> - **96.6% R@5 on LongMemEval in raw mode**, on 500 questions, zero API +> calls — independently reproduced on M2 Ultra in under 5 minutes by +> [@gizmax](https://github.com/MemPalace/mempalace/issues/39). +> - Local, free, no subscription, no cloud, no data leaving your machine. +> - The architecture (wings, rooms, closets, drawers) is real and useful, +> even if it's not a magical retrieval boost. +> +> **What we're doing:** +> +> 1. Rewriting the AAAK example with real tokenizer counts and a scenario +> where AAAK actually demonstrates compression +> 2. Adding `mode raw / aaak / rooms` clearly to the benchmark +> documentation so the trade-offs are visible +> 3. Wiring `fact_checker.py` into the KG ops so the contradiction +> detection claim becomes true +> 4. Pinning the vector store dependency to a tested range (issue #100), +> fixing the shell injection in hooks (#110), and addressing the macOS +> ARM64 segfault (#74) +> +> **Thank you to everyone who poked holes in this.** Brutal honest +> criticism is exactly what makes open source work, and it's what we asked +> for. Special thanks to +> [@panuhorsmalahti](https://github.com/MemPalace/mempalace/issues/43), +> [@lhl](https://github.com/MemPalace/mempalace/issues/27), +> [@gizmax](https://github.com/MemPalace/mempalace/issues/39), and everyone +> who filed an issue or a PR in the first 48 hours. We're listening, we're +> fixing, and we'd rather be right than impressive. +> +> — *Milla Jovovich & Ben Sigman*