Igor Lins e Silva
552e9927b7
Lands the read-side contract so third-party adapter authors (@Perseusxrltd, @JakobSachs, @adv3nt3, @zendesk-thittesdorf, @mfhens, @roip, @MrDys) have a stable target matching what RFC 001 §10 landed on the write side in #995. Scope (this PR): - mempalace/sources/base.py: BaseSourceAdapter ABC with kwargs-only ingest() / describe_schema() and default is_current() / source_summary() / close() (§1.1–1.2). Typed records: SourceRef, SourceItemMetadata, DrawerRecord, RouteHint, SourceSummary, AdapterSchema, FieldSpec (§1.3, §5.2). Error classes: SourceNotFoundError, AuthRequiredError, AdapterClosedError, TransformationViolationError, SchemaConformanceError (§2.7). Class-level identity contract: name / adapter_version / capabilities / supported_modes / declared_transformations / default_privacy_class (§2.1, §1.4, §1.5, §6). - mempalace/sources/transforms.py: reference implementations of the 13 reserved transformations (§1.4) — utf8_replace_invalid, newline_normalize, whitespace_trim, whitespace_collapse_internal, line_trim, line_join_spaces, blank_line_drop — as pure functions, plus identity shims for the six adapter-specific ones (strip_tool_chrome, tool_result_truncate, tool_result_omitted, spellcheck_user, synthesized_marker, speaker_role_assignment) that the conversations adapter will override when migrated. get_transformation(name) resolves by reserved name. - mempalace/sources/registry.py: entry-point discovery via importlib.metadata.entry_points(group="mempalace.sources") + explicit register()/unregister() surface (§3.1–3.2). resolve_adapter_for_source() implements the §3.3 priority order; crucially, no auto-detection on the read side (§3.3 is explicit about that — user intent never inferred from on-disk artifacts). - mempalace/sources/context.py: PalaceContext facade (§9) bundling the drawer/closet collections, knowledge graph, palace path, adapter identity, and progress hooks core passes into adapter.ingest(). upsert_drawer() applies the spec-mandated adapter_name/adapter_version stamps from §5.1. skip_current_item() signals laziness; emit() dispatches to hooks and swallows hook exceptions. - mempalace/knowledge_graph.py: add_triple() gains optional source_drawer_id and adapter_name kwargs (§5.5). Backwards-compatible column migration auto-adds the new columns on open of a pre-RFC 002 palace (PRAGMA table_info then ALTER TABLE ADD COLUMN), matching the pattern used for any new palace-side provenance fields. - pyproject.toml: mempalace.sources entry-point group declared. Empty on the first-party side for now — miners migrate in a follow-up; the group being present means third-party packages can begin registering today. Out of scope (explicit follow-ups): - miner.py → mempalace/sources/filesystem.py. Behavior-preserving rename that also moves READABLE_EXTENSIONS, detect_room(), detect_hall() into the adapter (§9). Larger refactor; lands separately. - convo_miner.py + normalize.py → mempalace/sources/conversations.py. The format-detection if-chain in normalize.py becomes per-format plugins; declared_transformations enumerates what the current pipeline already does to source bytes (§1.4 existing-code mapping). - Closet post-step wired into the conversations adapter (§1.7). - CLI --source flag + --mode deprecation alias (§3.3). - MCP mempalace_mine tool source parameter. - AbstractSourceAdapterContractSuite (§7.1–7.3): byte-preservation round- trip and declared-transformation round-trip tests. - Privacy-class floor enforcement (§6.2); depends on #389 for secrets_possible scanning. Tests: 1018 passed (up from ~990 on develop), +27 targeted tests covering the ABC instantiation rules, typed records, all reserved transformations, the registry register/get/unregister surface, PalaceContext upsert + skip + emit semantics, and both the new KG provenance kwargs and backwards- compatible legacy-schema migration. Refs: #989 (RFC 002 tracking), #990 (RFC 002 spec), #995 (RFC 001 §10 cleanup — sibling PR on the write side).
98 lines
2.6 KiB
TOML
98 lines
2.6 KiB
TOML
[project]
|
|
name = "mempalace"
|
|
version = "3.3.0"
|
|
description = "Give your AI a memory — mine projects and conversations into a searchable palace. No API key required."
|
|
readme = "README.md"
|
|
requires-python = ">=3.9"
|
|
license = "MIT"
|
|
authors = [
|
|
{name = "milla-jovovich"},
|
|
]
|
|
keywords = [
|
|
"ai", "memory", "llm", "rag", "chromadb", "mcp",
|
|
"vector-database", "claude", "chatgpt", "embeddings",
|
|
]
|
|
classifiers = [
|
|
"Development Status :: 4 - Beta",
|
|
"Environment :: Console",
|
|
"Intended Audience :: Developers",
|
|
"Programming Language :: Python :: 3",
|
|
"Programming Language :: Python :: 3.9",
|
|
"Programming Language :: Python :: 3.10",
|
|
"Programming Language :: Python :: 3.11",
|
|
"Programming Language :: Python :: 3.12",
|
|
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
|
"Topic :: Utilities",
|
|
]
|
|
dependencies = [
|
|
"chromadb>=0.5.0",
|
|
"pyyaml>=6.0,<7",
|
|
]
|
|
|
|
[project.urls]
|
|
Homepage = "https://github.com/MemPalace/mempalace"
|
|
Repository = "https://github.com/MemPalace/mempalace"
|
|
"Bug Tracker" = "https://github.com/MemPalace/mempalace/issues"
|
|
|
|
[project.scripts]
|
|
mempalace = "mempalace.cli:main"
|
|
|
|
[project.entry-points."mempalace.backends"]
|
|
chroma = "mempalace.backends.chroma:ChromaBackend"
|
|
|
|
# RFC 002 source-adapter entry-point group. Core publishes no first-party
|
|
# adapters under this group yet; ``miner.py`` and ``convo_miner.py`` migrate
|
|
# onto ``BaseSourceAdapter`` in a follow-up PR. Third-party adapter packages
|
|
# (``mempalace-source-cursor``, ``mempalace-source-git``, …) register here.
|
|
[project.entry-points."mempalace.sources"]
|
|
|
|
[project.optional-dependencies]
|
|
dev = ["pytest>=7.0", "pytest-cov>=4.0", "ruff>=0.4.0", "psutil>=5.9"]
|
|
spellcheck = ["autocorrect>=2.0"]
|
|
|
|
[dependency-groups]
|
|
dev = ["pytest>=7.0", "pytest-cov>=4.0", "ruff>=0.4.0", "psutil>=5.9"]
|
|
|
|
[build-system]
|
|
requires = ["hatchling"]
|
|
build-backend = "hatchling.build"
|
|
|
|
[tool.hatch.build.targets.wheel]
|
|
packages = ["mempalace"]
|
|
|
|
[tool.ruff]
|
|
line-length = 100
|
|
target-version = "py39"
|
|
extend-exclude = ["benchmarks"]
|
|
|
|
[tool.ruff.lint]
|
|
select = ["E", "F", "W", "C901"]
|
|
ignore = ["E501"]
|
|
|
|
[tool.ruff.lint.mccabe]
|
|
max-complexity = 25
|
|
|
|
[tool.ruff.format]
|
|
quote-style = "double"
|
|
|
|
[tool.pytest.ini_options]
|
|
testpaths = ["tests"]
|
|
pythonpath = ["."]
|
|
addopts = "-m 'not benchmark and not slow and not stress'"
|
|
markers = [
|
|
"benchmark: scale/performance benchmark tests",
|
|
"slow: tests that take more than 30 seconds",
|
|
"stress: destructive scale tests (100K+ drawers)",
|
|
]
|
|
|
|
[tool.coverage.run]
|
|
source = ["mempalace"]
|
|
|
|
[tool.coverage.report]
|
|
fail_under = 85
|
|
show_missing = true
|
|
exclude_lines = [
|
|
"if __name__",
|
|
"pragma: no cover",
|
|
]
|