2 Commits

Author SHA1 Message Date
shahondin1624 eb1063da28 add cross-agent memory extension
A pi extension for reading and writing persistent memory across agent systems.

Reads its own canonical store (project ./.pi/memory/ + user ~/.pi/memory/, typed
markdown with frontmatter — Claude's user/feedback/project/reference taxonomy)
AND other agents' memory verbatim (Claude CLAUDE.md + ~/.claude/projects/<slug>/
memory/, Copilot, AGENTS.md, GEMINI.md, Aider CONVENTIONS.md). Writes only ever
touch the canonical store; foreign files are never modified.

Surfaces: memory_read / memory_search / memory_write / memory_forget tools,
/memory-* commands, and a canonical index appended to the system prompt at
agent start.

Enum-valued args (type/scope/system) are lenient strings normalized in-tool
rather than anyOf/const unions: some local models emit const-union values
JSON-quoted (e.g. "\"feedback\""), which strict schema validation rejected
before the tool ran. normalizeEnumArg strips the stray quoting; memory_read by
name also falls back to a foreign file of that name.

store.ts / sources.ts import only Node built-ins so tests load them directly
under --experimental-strip-types. 31 unit tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 08:23:39 +02:00
shahondin1624 3876968bfa make llama.cpp base URL configurable via settings + document live-symlink dev setup
Resolve the local llama.cpp provider's server URL from LLAMA_BASE_URL env →
localLlama.baseUrl in settings.json → built-in default, reading settings inline
(node:fs) so the flat-copy test build stays self-contained. A PI_SETTINGS_PATH
override keeps the suite deterministic across hosts.

Document the live-development workflow of symlinking each extension dir AND
shared/ into ~/.pi/agent/extensions/, with a warning that a symlinked extension
paired with a stale copied shared/ silently loads the wrong helpers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 08:23:25 +02:00
9 changed files with 1745 additions and 10 deletions
+1
View File
@@ -11,6 +11,7 @@
!/ai-server/
!/dark-mechanicus/
!/llama.cpp/
!/memory/
!/scripts/
!/session-handoff/
!/shared/
+45 -5
View File
@@ -14,8 +14,9 @@ Warhammer 40k "Dark Mechanicum" flavoring on top of pi's interactive TUI.
|---|---|
| [`ai-server/`](ai-server/) | Remote llama.cpp provider over mTLS. Dynamic model discovery. Admin slash commands (load / unload / ctx / preset / restart / refresh). Custom SSE stream implementation with tool calls, reasoning, cache token reporting. See [ai-server/README.md](ai-server/README.md) for the full setup. |
| [`token-stats/`](token-stats/) + [`shared/token-stats.ts`](shared/token-stats.ts) | Footer owner for context-window + token-rate display. Tracks prefill and generation speed, including reasoning/thinking tokens, and reads `tokenStats.enabled` from `~/.pi/agent/settings.json`. |
| [`memory/`](memory/) | Cross-agent memory access. Reads this extension's own canonical store (project `./.pi/memory/` + user `~/.pi/memory/`, typed markdown with frontmatter — Claude's `user`/`feedback`/`project`/`reference` taxonomy) **and** other agents' memory verbatim (Claude `CLAUDE.md` + `~/.claude/projects/<slug>/memory/`, Copilot `.github/instructions`, `AGENTS.md`, `GEMINI.md`, Aider `CONVENTIONS.md`). Writes only ever touch the canonical store. Exposes `memory_read`/`memory_search`/`memory_write`/`memory_forget` tools, `/memory-*` commands, and appends the canonical index to the system prompt at agent start. |
| [`dark-mechanicus/`](dark-mechanicus/) | TUI customization bundle for the dark-mechanicus theme — loaded as one extension via `index.ts`. Includes: `indicator.ts` (working indicator: `⚙ <quote> · <elapsed>`, pulsing cog, 45-quote pool), `banner.ts` (cog-and-skull header art), `status-line.ts` (third footer line with rotating flavor text), `session-names.ts` (auto `<adj>-<noun> · <NNN>` session names + tab title), `thinking-label.ts` (`Cogitating...` for folded thinking blocks), `markdown-body-color.ts` (forces lavender body text). Display toggles now come from `darkMechanicus` settings instead of slash commands. |
| [`llama.cpp/`](llama.cpp/) | Local llama.cpp provider extension. Dynamic `/v1/models` discovery, fallback model registration, slash commands, and a custom streaming adapter that preserves `piTokenStats`. |
| [`llama.cpp/`](llama.cpp/) | Local llama.cpp provider extension. Dynamic `/v1/models` discovery, fallback model registration, slash commands, and a custom streaming adapter that preserves `piTokenStats`. Server base URL resolves from `LLAMA_BASE_URL` env → `localLlama.baseUrl` in `~/.pi/agent/settings.json` → built-in default. |
## Theme
@@ -39,7 +40,7 @@ A full commented sample config is in [`settings.sample.jsonc`](settings.sample.j
## Tests
Seventy-four tests total, no external dependencies. Runs with Node 22+'s
114 tests total, no external dependencies. Runs with Node 22+'s
built-in test runner:
```bash
@@ -49,10 +50,11 @@ node --experimental-strip-types --test tests/*.test.ts llama.cpp/llama.cpp.test.
| File | Coverage |
|---|---|
| `tests/messages.test.ts` | 15 unit tests over `ai-server/messages.ts` — pi Context → OpenAI payload conversion (system prompts, user/assistant/tool-result roles, tool calls, image-only messages). |
| `tests/router-utils.test.ts` | 12 unit tests over `ai-server/router-utils.ts``extractCtxSize`, `isShardArtefact`, and reasoning-model detection helpers. |
| `tests/router-utils.test.ts` | 14 unit tests over `ai-server/router-utils.ts``extractCtxSize`, `isShardArtefact`, and reasoning-model detection helpers. |
| `tests/integration.test.ts` | 6 live-endpoint tests: `/health`, `/models`, model-entry shape, mTLS enforcement, publicly-trusted cert (Let's Encrypt contract), chat completion usage shape including `prompt_tokens_details.cached_tokens`. Auto-skip if the server is unreachable. |
| `tests/token-stats.test.ts` | 6 unit tests over `shared/token-stats.ts` — timing metadata parsing and rate calculation, including thinking-token-aware generation speed. |
| `llama.cpp/llama.cpp.test.mjs` | 35 tests over the split local llama.cpp extension — reasoning-model detection, model discovery, provider registration, compat flags, slash commands, env overrides, and streaming token-stats behavior. |
| `tests/token-stats.test.ts` | 10 unit tests over `shared/token-stats.ts` — timing metadata parsing and rate calculation, including thinking-token-aware generation speed and displayable-turn fallback. |
| `tests/memory.test.ts` | 31 tests over `memory/store.ts` + `memory/sources.ts` — slug/frontmatter round-trip, canonical CRUD against temp dirs, index build + sort, keyword search ranking, enum-arg normalization (strips model double-quoting), foreign lookup by name, Claude project-slug derivation, and foreign-source discovery/source-tagging. |
| `llama.cpp/llama.cpp.test.mjs` | 38 tests over the split local llama.cpp extension — reasoning-model detection, model discovery, provider registration, compat flags, slash commands, env + `localLlama.baseUrl` settings resolution, and streaming token-stats behavior. |
Stream-parsing edge cases (SSE framing, tool-call splits across chunks,
reasoning deltas, abort mid-stream) remain deferred — they need a mock
@@ -73,6 +75,40 @@ scripts/install-client.sh
# /settings in pi, pick "dark-mechanicus"
```
## Local development (syncing the repo into pi)
pi loads extensions from `~/.pi/agent/extensions/`. `install-client.sh` *copies*
files there, but for active development it's easier to **symlink** each
extension (and `shared/`) so edits in this repo take effect on the next pi
restart — no re-copy needed.
```bash
REPO="$HOME/Projects/pi-extensions" # this checkout
EXT="$HOME/.pi/agent/extensions"
mkdir -p "$EXT"
# Symlink every tracked extension directory + shared/ into the load dir.
for d in ai-server dark-mechanicus llama.cpp memory session-handoff token-stats shared; do
rm -rf "$EXT/$d" # remove any stale copy/symlink first
ln -s "$REPO/$d" "$EXT/$d"
done
# Sanity check
ls -la "$EXT" # each entry should be a symlink -> the repo
```
> **Important:** `shared/` **must** be symlinked too, not left as a copy.
> Extensions import sibling helpers via `../shared/*.js`, and pi's loader
> resolves those relative to the *install* path (it does not canonicalize
> symlinks). A symlinked extension paired with a stale copied `shared/` will
> silently load the wrong helpers — e.g. an extension can import a function the
> copy doesn't have yet, throw at render, and (for the footer) blank out
> entirely. Keep them in lockstep by symlinking both.
After changing symlinks, **restart pi** to reload extensions. To go back to a
copy-based install, delete the symlinks and re-run
`scripts/install-client.sh`.
## Layout
```
@@ -87,6 +123,10 @@ pi-extensions/
│ └── README.md full mTLS + systemd + Caddy setup notes
├── token-stats/
│ └── index.ts footer owner for context + token rate display
├── memory/ cross-agent memory access (multi-file)
│ ├── index.ts entry — tools, /memory-* commands, index injection
│ ├── store.ts canonical typed store (frontmatter, CRUD, search)
│ └── sources.ts read-only foreign adapters (CLAUDE.md, AGENTS.md, …)
├── settings.sample.jsonc commented template for ~/.pi/agent/settings.json
├── dark-mechanicus/ theme TUI bundle (one extension, multi-file)
│ ├── index.ts entry — sequences each module's registrar
+34 -2
View File
@@ -1,13 +1,45 @@
/**
* Configuration constants for the llama.cpp provider extension.
*
* All values are configurable via environment variables. Defaults are
* The server base URL resolves in this order:
* 1. LLAMA_BASE_URL environment variable
* 2. `localLlama.baseUrl` in ~/.pi/agent/settings.json
* 3. Built-in default
* All other values are configurable via environment variables. Defaults are
* suitable for a typical LAN-based llama.cpp server.
*/
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
// ─── Settings lookup ────────────────────────────────────────────────────
const HOME = process.env.HOME ?? process.env.USERPROFILE ?? "";
// PI_SETTINGS_PATH lets tests point at an isolated settings file (or a
// nonexistent one) so resolution is deterministic regardless of the host.
const SETTINGS_PATH = process.env.PI_SETTINGS_PATH ?? join(HOME, ".pi", "agent", "settings.json");
/** Read `localLlama.baseUrl` (or `local-llama.baseUrl`) from pi's settings.json. */
function baseUrlFromSettings(): string | undefined {
try {
if (!SETTINGS_PATH || !existsSync(SETTINGS_PATH)) {
return undefined;
}
const settings = JSON.parse(readFileSync(SETTINGS_PATH, "utf8")) as Record<string, unknown>;
const section = (settings.localLlama ?? settings["local-llama"]) as
| Record<string, unknown>
| undefined;
const url = section?.baseUrl;
return typeof url === "string" && url.length > 0 ? url : undefined;
} catch {
return undefined;
}
}
// ─── Server configuration ───────────────────────────────────────────────
export const BASE_URL = process.env.LLAMA_BASE_URL ?? "http://192.168.2.35:8123/v1";
export const BASE_URL =
process.env.LLAMA_BASE_URL ?? baseUrlFromSettings() ?? "http://192.168.2.35:8123/v1";
// ─── Fallback model ─────────────────────────────────────────────────────
+50
View File
@@ -82,6 +82,9 @@ function cleanLlamaEnv() {
delete process.env.LLAMA_MODEL_ID;
delete process.env.LLAMA_CTX;
delete process.env.LLAMA_MAX_OUT;
// Point settings resolution at a nonexistent file so BASE_URL falls through
// to the built-in default, independent of the developer's real settings.json.
process.env.PI_SETTINGS_PATH = join(tmpdir(), "llama-test-no-such-settings.json");
}
// ─── Mock PI ────────────────────────────────────────────────────────────────
@@ -811,6 +814,53 @@ test("extension entry: registers slash commands", async () => {
}
});
test("config: reads baseUrl from localLlama settings when env unset", async () => {
const { outputDir } = buildCompiledModule();
const settingsDir = mkdtempSync(join(tmpdir(), "llama-settings-"));
const settingsPath = join(settingsDir, "settings.json");
writeFileSync(
settingsPath,
JSON.stringify({ localLlama: { baseUrl: "http://10.0.0.9:8123/v1" } }),
"utf8",
);
try {
cleanLlamaEnv();
process.env.PI_SETTINGS_PATH = settingsPath;
const { pi, state } = createMockPI();
const mod = await importModule(outputDir);
mod.registerProviderWithModels(pi, [{ id: "m" }]);
assert.equal(state.providers[0].config.baseUrl, "http://10.0.0.9:8123/v1");
} finally {
cleanLlamaEnv();
rmSync(settingsDir, { recursive: true, force: true });
rmSync(outputDir, { recursive: true, force: true });
}
});
test("config: LLAMA_BASE_URL env overrides localLlama settings", async () => {
const { outputDir } = buildCompiledModule();
const settingsDir = mkdtempSync(join(tmpdir(), "llama-settings-"));
const settingsPath = join(settingsDir, "settings.json");
writeFileSync(
settingsPath,
JSON.stringify({ localLlama: { baseUrl: "http://10.0.0.9:8123/v1" } }),
"utf8",
);
try {
cleanLlamaEnv();
process.env.PI_SETTINGS_PATH = settingsPath;
process.env.LLAMA_BASE_URL = "http://env-host:9999/v1";
const { pi, state } = createMockPI();
const mod = await importModule(outputDir);
mod.registerProviderWithModels(pi, [{ id: "m" }]);
assert.equal(state.providers[0].config.baseUrl, "http://env-host:9999/v1");
} finally {
cleanLlamaEnv();
rmSync(settingsDir, { recursive: true, force: true });
rmSync(outputDir, { recursive: true, force: true });
}
});
test("extension entry: uses env overrides for BASE_URL", async () => {
const { outputDir } = buildCompiledModule();
try {
+509
View File
@@ -0,0 +1,509 @@
/**
* memory — cross-agent memory access for pi.
*
* READS span this extension's own canonical store (project + user scope) AND
* other agent systems' memory files (Claude, Copilot, AGENTS.md, Gemini,
* Aider), surfaced verbatim and source-tagged. WRITES only ever touch the
* canonical store (see ./store.ts) — foreign files are never modified.
*
* Surfaces
* ────────
* Tools (LLM-callable): memory_search · memory_read · memory_write · memory_forget
* Commands (human): /memory-list · /memory-search · /memory-show
* /memory-write · /memory-forget
* Context injection: on before_agent_start, the canonical index (one line
* per memory) is appended to the system prompt so the
* model always knows what it can pull. Only the
* canonical store is indexed — foreign systems already
* load their own files, so we don't double-inject them.
*
* Settings (~/.pi/agent/settings.json, key "memory")
* enabled boolean master switch (default true; read at load)
* injectIndex boolean append the index to the system prompt (default true)
* readForeign boolean include foreign agent memory in reads (default true)
*/
import { homedir } from "node:os";
import type {
ExtensionAPI,
ExtensionCommandContext,
ExtensionContext,
} from "@mariozechner/pi-coding-agent";
import { Type } from "typebox";
import { readExtensionBooleanSetting } from "../shared/pi-settings.js";
import {
buildIndex,
type CanonicalMemory,
findByName,
isMemoryScope,
isMemoryType,
listMemories,
type MemoryScope,
MEMORY_SCOPES,
type MemoryType,
MEMORY_TYPES,
deleteMemory,
normalizeEnumArg,
searchMemories,
storeDirForScope,
writeMemory,
} from "./store.js";
import {
discoverForeignSources,
findForeignByName,
type ForeignSource,
FOREIGN_SYSTEMS,
formatForeignDigest,
formatForeignFull,
isForeignSystem,
} from "./sources.js";
const EXTENSION_NAME = "memory";
// ── settings ──────────────────────────────────────────────────────────────
const isEnabled = () => readExtensionBooleanSetting(EXTENSION_NAME, "enabled", true);
const shouldInjectIndex = () => readExtensionBooleanSetting(EXTENSION_NAME, "injectIndex", true);
const shouldReadForeign = () => readExtensionBooleanSetting(EXTENSION_NAME, "readForeign", true);
// ── parameter schemas ───────────────────────────────────────────────────────
// These are closed string vocabularies, but they are declared as plain strings
// (allowed values spelled out in the description) rather than anyOf/const
// unions on purpose: some local models emit const-union values JSON-quoted
// (e.g. `"\"feedback\""`), which strict schema validation then rejects BEFORE
// the tool runs. Keeping them lenient lets the call reach `execute`, where
// normalizeEnumArg strips the stray quoting and we validate against the
// vocabulary ourselves — with a helpful error instead of a hard schema reject.
const memoryTypeSchema = Type.String({
description: `Memory type — one of: ${MEMORY_TYPES.join(" | ")}.`,
});
const writeScopeSchema = Type.String({
description: `Store scope — one of: ${MEMORY_SCOPES.join(" | ")} (default project).`,
});
const scopeFilterSchema = Type.String({
description: `Scope filter — one of: ${MEMORY_SCOPES.join(" | ")} | all (default all).`,
});
const foreignSystemSchema = Type.String({
description: `Foreign agent system — one of: ${FOREIGN_SYSTEMS.join(" | ")}.`,
});
// ── small helpers ───────────────────────────────────────────────────────
function text(body: string) {
return { content: [{ type: "text" as const, text: body }], details: undefined };
}
function dirs(ctx: ExtensionContext | ExtensionCommandContext) {
const cwd = ctx.cwd;
const home = homedir();
return { cwd, home };
}
/** Canonical memories across scopes, optionally narrowed by a scope filter. */
function loadCanonical(
ctx: ExtensionContext | ExtensionCommandContext,
scopeFilter: MemoryScope | "all" = "all",
): CanonicalMemory[] {
const { cwd, home } = dirs(ctx);
const out: CanonicalMemory[] = [];
for (const scope of MEMORY_SCOPES) {
if (scopeFilter !== "all" && scopeFilter !== scope) continue;
out.push(...listMemories(storeDirForScope(scope, cwd, home), scope));
}
return out;
}
function formatMemory(m: CanonicalMemory): string {
const meta = `[${m.scope}/${m.type}] ${m.name}`;
const stamps = [m.created && `created ${m.created}`, m.updated && `updated ${m.updated}`]
.filter(Boolean)
.join(" · ");
const desc = m.description ? `\ndescription: ${m.description}` : "";
return `${meta}${desc}${stamps ? `\n${stamps}` : ""}\n\n${m.body}`;
}
function foreignSnippet(content: string, lowerQuery: string): string {
const idx = content.toLowerCase().indexOf(lowerQuery);
if (idx === -1) return "";
const start = Math.max(0, idx - 40);
const end = Math.min(content.length, idx + lowerQuery.length + 60);
const prefix = start > 0 ? "…" : "";
const suffix = end < content.length ? "…" : "";
return `${prefix}${content.slice(start, end).replace(/\s+/g, " ").trim()}${suffix}`;
}
function searchForeign(sources: readonly ForeignSource[], query: string) {
const lowerQuery = query.trim().toLowerCase();
if (!lowerQuery) return [];
return sources
.filter((s) => s.content.toLowerCase().includes(lowerQuery))
.map((s) => ({ source: s, snippet: foreignSnippet(s.content, lowerQuery) }));
}
/**
* Argument completer for /memory-show and /memory-forget. Completers don't get a
* ctx, so we best-effort use process.cwd() (the session cwd in practice) to list
* canonical memory names across both scopes.
*/
function completeMemoryName(prefix: string) {
try {
const cwd = process.cwd();
const home = homedir();
const names = new Set<string>();
for (const scope of MEMORY_SCOPES) {
for (const m of listMemories(storeDirForScope(scope, cwd, home), scope)) {
names.add(m.name);
}
}
const hits = [...names]
.filter((n) => n.toLowerCase().startsWith(prefix.toLowerCase()))
.map((n) => ({ value: n, label: n }));
return hits.length > 0 ? hits : null;
} catch {
return null;
}
}
// ── extension ───────────────────────────────────────────────────────────
export default function (pi: ExtensionAPI) {
if (!isEnabled()) return;
// ── Context injection ──────────────────────────────────────────────────
// Append the canonical index to the system prompt on every agent loop start.
// before_agent_start lets us chain onto the assembled prompt; we add a clearly
// delimited block and skip entirely when the store is empty (no token cost).
pi.on("before_agent_start", async (event, ctx) => {
try {
if (!shouldInjectIndex()) return undefined;
const index = buildIndex(loadCanonical(ctx));
if (!index) return undefined;
const block = [
"<persistent-memory>",
"You have a persistent memory store (project + user scope). The entries below",
"are available — call `memory_read` with a `name` to load a full body,",
"`memory_search` to search by keyword, `memory_write` to save a new/updated",
"memory, and `memory_forget` to remove one. Memory from other agent tools",
"(CLAUDE.md, AGENTS.md, etc.) is also readable via `memory_read`/`memory_search`.",
"",
index,
"</persistent-memory>",
].join("\n");
return { systemPrompt: `${event.systemPrompt}\n\n${block}` };
} catch {
return undefined;
}
});
// ── Tools ────────────────────────────────────────────────────────────
pi.registerTool({
name: "memory_search",
label: "Memory Search",
description:
"Keyword/substring search across persistent memory — this agent's canonical " +
"store (project + user) and, unless disabled, other agent systems' memory " +
"(CLAUDE.md, Copilot, AGENTS.md, Gemini, Aider). Case-insensitive.",
promptSnippet: "Search persistent memory by keyword",
parameters: Type.Object({
query: Type.String({ description: "Case-insensitive substring to search for." }),
scope: Type.Optional(
scopeFilterSchema,
),
type: Type.Optional(memoryTypeSchema),
includeForeign: Type.Optional(
Type.Boolean({
description:
"Also search other agent systems' memory files. Defaults to true.",
}),
),
}),
async execute(_id, params, _signal, _onUpdate, ctx) {
const scopeArg = normalizeEnumArg(params.scope);
const scope: MemoryScope | "all" = isMemoryScope(scopeArg) ? scopeArg : "all";
let canonical = loadCanonical(ctx, scope);
const typeArg = normalizeEnumArg(params.type);
if (isMemoryType(typeArg)) {
canonical = canonical.filter((m) => m.type === typeArg);
}
const hits = searchMemories(canonical, params.query);
const includeForeign = params.includeForeign !== false && shouldReadForeign();
const { cwd, home } = dirs(ctx);
const foreignHits = includeForeign
? searchForeign(discoverForeignSources(cwd, home), params.query)
: [];
if (hits.length === 0 && foreignHits.length === 0) {
return text(`No memory matches "${params.query}".`);
}
const lines: string[] = [];
if (hits.length > 0) {
lines.push(`Canonical matches (${hits.length}):`);
for (const h of hits) {
lines.push(`- [${h.memory.scope}/${h.memory.type}] ${h.memory.name}${h.snippet}`);
}
}
if (foreignHits.length > 0) {
if (lines.length) lines.push("");
lines.push(`Foreign matches (${foreignHits.length}):`);
for (const f of foreignHits) {
lines.push(`- [${f.source.system}/${f.source.scope}] ${f.source.path}${f.snippet}`);
}
}
return text(lines.join("\n"));
},
});
pi.registerTool({
name: "memory_read",
label: "Memory Read",
description:
"Read persistent memory. With `name`: return that canonical memory's full body — " +
"or, if the name matches a foreign file (e.g. CLAUDE.md, MEMORY.md, project_x), that " +
"file verbatim. With `system`: dump that agent system's memory files verbatim. With " +
"neither: return a map of all available memory (canonical index + foreign source list).",
promptSnippet: "Read a stored memory or list what is available",
parameters: Type.Object({
name: Type.Optional(
Type.String({
description:
"Name of a canonical memory, or a foreign memory file name like " +
"'MEMORY.md' / 'CLAUDE.md', to read in full.",
}),
),
system: Type.Optional(foreignSystemSchema),
scope: Type.Optional(scopeFilterSchema),
}),
async execute(_id, params, _signal, _onUpdate, ctx) {
const { cwd, home } = dirs(ctx);
const scopeArg = normalizeEnumArg(params.scope);
const scope: MemoryScope | "all" = isMemoryScope(scopeArg) ? scopeArg : "all";
if (params.name) {
const found = findByName(loadCanonical(ctx, scope), params.name);
if (found) return text(formatMemory(found));
// Fall back to a foreign file of the same name (CLAUDE.md, MEMORY.md, …).
if (shouldReadForeign()) {
const foreign = findForeignByName(discoverForeignSources(cwd, home), params.name);
if (foreign) return text(formatForeignFull([foreign]));
}
return text(`No memory named "${params.name}" (canonical or foreign).`);
}
const systemArg = normalizeEnumArg(params.system);
if (systemArg) {
if (!isForeignSystem(systemArg)) {
return text(
`Unknown system "${params.system}". Valid systems: ${FOREIGN_SYSTEMS.join(", ")}.`,
);
}
if (!shouldReadForeign()) {
return text("Foreign memory reading is disabled (memory.readForeign = false).");
}
const sources = discoverForeignSources(cwd, home, [systemArg]);
if (sources.length === 0) {
return text(`No ${systemArg} memory found for this project.`);
}
return text(formatForeignFull(sources));
}
// The map: canonical index + foreign digest.
const index = buildIndex(loadCanonical(ctx, scope));
const parts: string[] = [];
parts.push(index ? `Canonical memory:\n${index}` : "Canonical memory: (empty)");
if (shouldReadForeign()) {
const digest = formatForeignDigest(discoverForeignSources(cwd, home));
parts.push(digest ? `\nForeign memory sources:\n${digest}` : "\nForeign memory sources: (none found)");
}
return text(parts.join("\n"));
},
});
pi.registerTool({
name: "memory_write",
label: "Memory Write",
description:
"Create or update a canonical memory. Writing a name that matches an existing " +
"memory updates it. Saved to the project store by default; pass scope='user' for " +
"a machine-global memory. Use the typed taxonomy: user, feedback, project, reference.",
promptSnippet: "Save a durable memory to the canonical store",
promptGuidelines: [
"Save a memory when you learn something durable and reusable across sessions: a " +
"user preference or correction (type 'feedback'), facts about the user (type " +
"'user'), ongoing project context (type 'project'), or a pointer to an external " +
"system (type 'reference'). Keep `description` a one-line hook; put detail in `content`.",
"Prefer updating an existing memory (same name) over creating a near-duplicate.",
],
parameters: Type.Object({
name: Type.String({ description: "Short human-readable title; also the memory's identity." }),
description: Type.String({ description: "One-line summary used in the index and search." }),
type: memoryTypeSchema,
content: Type.String({ description: "The memory body (markdown)." }),
scope: Type.Optional(writeScopeSchema),
}),
async execute(_id, params, _signal, _onUpdate, ctx) {
const { cwd, home } = dirs(ctx);
const scope: MemoryScope = isMemoryScope(normalizeEnumArg(params.scope))
? (normalizeEnumArg(params.scope) as MemoryScope)
: "project";
const typeArg = normalizeEnumArg(params.type);
if (!isMemoryType(typeArg)) {
return text(`Invalid type "${params.type}". Valid types: ${MEMORY_TYPES.join(", ")}.`);
}
const type: MemoryType = typeArg;
const dir = storeDirForScope(scope, cwd, home);
const result = writeMemory(dir, {
name: params.name,
type,
description: params.description,
body: params.content,
});
const verb = result.created ? "Created" : "Updated";
return {
...text(`${verb} ${scope} memory "${result.memory.name}" (${type}) at ${result.memory.path}`),
details: { scope, ...result },
};
},
});
pi.registerTool({
name: "memory_forget",
label: "Memory Forget",
description:
"Delete a canonical memory by name. Only the canonical store is affected — foreign " +
"agent memory is never modified. Defaults to the project scope.",
promptSnippet: "Delete a stored memory",
parameters: Type.Object({
name: Type.String({ description: "Name of the canonical memory to delete." }),
scope: Type.Optional(writeScopeSchema),
}),
async execute(_id, params, _signal, _onUpdate, ctx) {
const { cwd, home } = dirs(ctx);
const scopeArg = normalizeEnumArg(params.scope);
const scope: MemoryScope = isMemoryScope(scopeArg) ? scopeArg : "project";
const removed = deleteMemory(storeDirForScope(scope, cwd, home), params.name);
return text(
removed
? `Forgot ${scope} memory "${params.name}".`
: `No ${scope} memory named "${params.name}" to forget.`,
);
},
});
// ── Slash commands ─────────────────────────────────────────────────────
pi.registerCommand("memory-list", {
description: "List canonical memories and discovered foreign memory sources",
handler: async (_args, ctx) => {
const { cwd, home } = dirs(ctx);
const index = buildIndex(loadCanonical(ctx));
const lines = [index ? `Canonical memory:\n${index}` : "Canonical memory: (empty)"];
if (shouldReadForeign()) {
const digest = formatForeignDigest(discoverForeignSources(cwd, home));
lines.push(digest ? `\nForeign sources:\n${digest}` : "\nForeign sources: (none found)");
}
ctx.ui.notify(lines.join("\n"), "info");
},
});
pi.registerCommand("memory-search", {
description: "Search memory by keyword (usage: /memory-search <query>)",
handler: async (args, ctx) => {
const query = args.trim();
if (!query) {
ctx.ui.notify("Usage: /memory-search <query>", "error");
return;
}
const { cwd, home } = dirs(ctx);
const hits = searchMemories(loadCanonical(ctx), query);
const foreignHits = shouldReadForeign()
? searchForeign(discoverForeignSources(cwd, home), query)
: [];
if (hits.length === 0 && foreignHits.length === 0) {
ctx.ui.notify(`No memory matches "${query}".`, "info");
return;
}
const lines: string[] = [];
for (const h of hits) {
lines.push(`[${h.memory.scope}/${h.memory.type}] ${h.memory.name}${h.snippet}`);
}
for (const f of foreignHits) {
lines.push(`[${f.source.system}/${f.source.scope}] ${f.source.path}${f.snippet}`);
}
ctx.ui.notify(lines.join("\n"), "info");
},
});
pi.registerCommand("memory-show", {
description: "Show a canonical memory in full (usage: /memory-show <name>)",
getArgumentCompletions: completeMemoryName,
handler: async (args, ctx) => {
const name = args.trim();
if (!name) {
ctx.ui.notify("Usage: /memory-show <name>", "error");
return;
}
const found = findByName(loadCanonical(ctx), name);
ctx.ui.notify(found ? formatMemory(found) : `No canonical memory named "${name}".`, found ? "info" : "error");
},
});
pi.registerCommand("memory-write", {
description: "Interactively create or update a canonical memory",
handler: async (_args, ctx) => {
const scope = await ctx.ui.select("Memory scope", [...MEMORY_SCOPES]);
if (!scope || !isMemoryScope(scope)) return;
const type = await ctx.ui.select("Memory type", [...MEMORY_TYPES]);
if (!type || !isMemoryType(type)) return;
const name = await ctx.ui.input("Memory name", "short title");
if (!name || !name.trim()) return;
const description = await ctx.ui.input("One-line description");
if (description === undefined) return;
const body = await ctx.ui.editor(`Memory body — ${name.trim()}`);
if (body === undefined) return;
const { cwd, home } = dirs(ctx);
try {
const result = writeMemory(storeDirForScope(scope, cwd, home), {
name,
type,
description: description ?? "",
body,
});
ctx.ui.notify(
`${result.created ? "Created" : "Updated"} ${scope} memory "${result.memory.name}"`,
"info",
);
} catch (err) {
ctx.ui.notify(`Write failed: ${(err as Error).message}`, "error");
}
},
});
pi.registerCommand("memory-forget", {
description: "Delete a canonical memory (usage: /memory-forget <name> [project|user])",
getArgumentCompletions: completeMemoryName,
handler: async (args, ctx) => {
const parts = args.trim().split(/\s+/).filter(Boolean);
const name = parts[0];
if (!name) {
ctx.ui.notify("Usage: /memory-forget <name> [project|user]", "error");
return;
}
const scopeArg = normalizeEnumArg(parts[1]);
const scope: MemoryScope = isMemoryScope(scopeArg) ? scopeArg : "project";
const ok = await ctx.ui.confirm(
"Forget memory?",
`Delete ${scope} memory "${name}"? This removes the file.`,
);
if (!ok) return;
const { cwd, home } = dirs(ctx);
const removed = deleteMemory(storeDirForScope(scope, cwd, home), name);
ctx.ui.notify(
removed ? `Forgot ${scope} memory "${name}".` : `No ${scope} memory named "${name}".`,
removed ? "info" : "warning",
);
},
});
}
+220
View File
@@ -0,0 +1,220 @@
/**
* memory/sources.ts — read-only adapters for OTHER agent systems' memory.
*
* This extension never writes here. It surfaces what already exists on disk,
* verbatim, each block tagged with the system + scope + path it came from
* ("raw, source-tagged" — no lossy re-parsing into our typed model). The model
* always sees exactly what is on disk.
*
* Covered systems
* ───────────────
* claude CLAUDE.md / CLAUDE.local.md (project), ~/.claude/CLAUDE.md (user),
* and the structured auto-memory store
* ~/.claude/projects/<slug>/memory/*.md (project-associated).
* copilot .github/copilot-instructions.md and .github/instructions/*.instructions.md.
* agents AGENTS.md (the generic cross-tool convention; Codex, Amp, etc.).
* gemini GEMINI.md (project) and ~/.gemini/GEMINI.md (user).
* aider CONVENTIONS.md (Aider's conventions file).
*
* Import-free of pi-coding-agent / typebox (Node built-ins only) so the test
* suite can load it with `node --experimental-strip-types`.
*/
import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";
import { join } from "node:path";
export const FOREIGN_SYSTEMS = ["claude", "copilot", "agents", "gemini", "aider"] as const;
export type ForeignSystem = (typeof FOREIGN_SYSTEMS)[number];
export type ForeignScope = "project" | "user";
export interface ForeignSource {
system: ForeignSystem;
scope: ForeignScope;
/** Absolute path of the file the content came from. */
path: string;
/** Verbatim file content. */
content: string;
}
export function isForeignSystem(value: unknown): value is ForeignSystem {
return typeof value === "string" && (FOREIGN_SYSTEMS as readonly string[]).includes(value);
}
// ── Claude auto-memory project slug ──────────────────────────────────────
/**
* Claude Code stores per-project auto-memory under
* ~/.claude/projects/<slug>/memory/, where <slug> is the project's absolute
* path with separators replaced. The exact encoding has shifted across
* versions, so we return CANDIDATES (most-likely first) and let discovery pick
* the one that actually exists on disk:
*
* 1. "/" → "-" e.g. /home/u/proj → -home-u-proj
* 2. every non-alphanumeric → "-" folds dots, spaces, etc.
*
* Pure (no fs) so it is unit-testable in isolation.
*/
export function claudeProjectSlugCandidates(cwd: string): string[] {
const bySlash = cwd.replace(/\//g, "-");
const byNonAlnum = cwd.replace(/[^a-zA-Z0-9]/g, "-");
return [...new Set([bySlash, byNonAlnum])];
}
/** Resolve the Claude auto-memory dir for this cwd, or null if none exists. */
export function claudeMemoryDir(cwd: string, home: string): string | null {
for (const slug of claudeProjectSlugCandidates(cwd)) {
const dir = join(home, ".claude", "projects", slug, "memory");
if (safeIsDir(dir)) return dir;
}
return null;
}
// ── fs helpers ────────────────────────────────────────────────────────────
function safeIsFile(path: string): boolean {
try {
return statSync(path).isFile();
} catch {
return false;
}
}
function safeIsDir(path: string): boolean {
try {
return statSync(path).isDirectory();
} catch {
return false;
}
}
function readIfFile(
system: ForeignSystem,
scope: ForeignScope,
path: string,
out: ForeignSource[],
): void {
if (!safeIsFile(path)) return;
try {
const content = readFileSync(path, "utf8");
if (content.trim().length > 0) out.push({ system, scope, path, content });
} catch {
// Unreadable — skip silently; memory reading must never hard-fail.
}
}
function readMatchingInDir(
system: ForeignSystem,
scope: ForeignScope,
dir: string,
match: (filename: string) => boolean,
out: ForeignSource[],
): void {
if (!safeIsDir(dir)) return;
let names: string[];
try {
names = readdirSync(dir).sort();
} catch {
return;
}
for (const name of names) {
if (match(name)) readIfFile(system, scope, join(dir, name), out);
}
}
// ── Discovery ─────────────────────────────────────────────────────────────
/**
* Discover and read every foreign memory source reachable from `cwd` / `home`.
* Only existing, non-empty, readable files are returned. Optional `systems`
* restricts the set (default: all). Order is stable: project before user, and
* grouped by system in FOREIGN_SYSTEMS order.
*/
export function discoverForeignSources(
cwd: string,
home: string,
systems: readonly ForeignSystem[] = FOREIGN_SYSTEMS,
): ForeignSource[] {
const want = new Set(systems);
const out: ForeignSource[] = [];
if (want.has("claude")) {
// Guidance files.
readIfFile("claude", "project", join(cwd, "CLAUDE.md"), out);
readIfFile("claude", "project", join(cwd, "CLAUDE.local.md"), out);
readIfFile("claude", "user", join(home, ".claude", "CLAUDE.md"), out);
// Structured auto-memory store for this project.
const memDir = claudeMemoryDir(cwd, home);
if (memDir) {
readMatchingInDir(
"claude",
"project",
memDir,
(n) => n.toLowerCase().endsWith(".md"),
out,
);
}
}
if (want.has("copilot")) {
readIfFile("copilot", "project", join(cwd, ".github", "copilot-instructions.md"), out);
readMatchingInDir(
"copilot",
"project",
join(cwd, ".github", "instructions"),
(n) => n.toLowerCase().endsWith(".instructions.md"),
out,
);
}
if (want.has("agents")) {
readIfFile("agents", "project", join(cwd, "AGENTS.md"), out);
}
if (want.has("gemini")) {
readIfFile("gemini", "project", join(cwd, "GEMINI.md"), out);
readIfFile("gemini", "user", join(home, ".gemini", "GEMINI.md"), out);
}
if (want.has("aider")) {
readIfFile("aider", "project", join(cwd, "CONVENTIONS.md"), out);
}
return out;
}
/**
* Find a discovered foreign source by file name. Matches the path's basename
* case-insensitively, with or without a trailing `.md`, so a caller can ask for
* "MEMORY.md", "memory", or "project_nextcloud". Returns the first match or null.
*/
export function findForeignByName(
sources: readonly ForeignSource[],
name: string,
): ForeignSource | null {
const want = name.trim().toLowerCase();
if (!want) return null;
const wantNoExt = want.replace(/\.md$/i, "");
for (const s of sources) {
const base = (s.path.split(/[\\/]/).pop() ?? "").toLowerCase();
if (base === want || base.replace(/\.md$/i, "") === wantNoExt) return s;
}
return null;
}
// ── Formatting ────────────────────────────────────────────────────────────
/** One line per source: `- [system/scope] path (N chars)`. For read-maps. */
export function formatForeignDigest(sources: readonly ForeignSource[]): string {
if (sources.length === 0) return "";
return sources
.map((s) => `- [${s.system}/${s.scope}] ${s.path} (${s.content.length} chars)`)
.join("\n");
}
/** Full source-tagged dump: a heading per source followed by its verbatim body. */
export function formatForeignFull(sources: readonly ForeignSource[]): string {
return sources
.map((s) => `### ${s.system} · ${s.scope} · ${s.path}\n\n${s.content.trim()}`)
.join("\n\n");
}
+413
View File
@@ -0,0 +1,413 @@
/**
* memory/store.ts — the canonical, typed memory store.
*
* This is where this extension WRITES. Reads can span foreign agent systems
* too (see ./sources.ts), but every write lands here, in one of two scopes:
*
* project → <cwd>/.pi/memory/ (travels with the repo)
* user → ~/.pi/memory/ (global, machine-wide)
*
* Each memory is a single markdown file with YAML-ish frontmatter, modelled on
* Claude Code's auto-memory format (the four types user/feedback/project/
* reference). The on-disk shape:
*
* ---
* name: short-kebab-slug
* description: one-line summary used for the index + search
* type: user | feedback | project | reference
* created: 2026-05-28T10:00:00.000Z
* updated: 2026-05-28T10:00:00.000Z
* ---
*
* <body>
*
* The index that gets surfaced at session start is BUILT from these files'
* frontmatter on demand (buildIndex). There is deliberately no separate
* MEMORY.md index file to drift out of sync.
*
* Import-free of pi-coding-agent / typebox on purpose: tests load this module
* directly with `node --experimental-strip-types` (the repo has no local
* node_modules), so only Node built-ins are allowed here.
*/
import {
existsSync,
mkdirSync,
readdirSync,
readFileSync,
rmSync,
statSync,
writeFileSync,
} from "node:fs";
import { join } from "node:path";
// ── Vocabulary ──────────────────────────────────────────────────────────
/** Claude Code's auto-memory taxonomy, reused verbatim. */
export const MEMORY_TYPES = ["user", "feedback", "project", "reference"] as const;
export type MemoryType = (typeof MEMORY_TYPES)[number];
export const MEMORY_SCOPES = ["project", "user"] as const;
export type MemoryScope = (typeof MEMORY_SCOPES)[number];
export function isMemoryType(value: unknown): value is MemoryType {
return typeof value === "string" && (MEMORY_TYPES as readonly string[]).includes(value);
}
export function isMemoryScope(value: unknown): value is MemoryScope {
return typeof value === "string" && (MEMORY_SCOPES as readonly string[]).includes(value);
}
/**
* Normalize a model-supplied enum-ish argument before matching it against a
* known vocabulary. Some local models, when handed a closed string set, emit
* the value JSON-quoted (e.g. `"\"feedback\""`) or with stray whitespace/casing
* rather than the bare token. Strip any wrapping single/double quotes (even
* nested), trim, and lowercase so `isMemoryType`/`isMemoryScope`/foreign-system
* checks see a clean value. Non-strings normalize to "".
*/
export function normalizeEnumArg(value: unknown): string {
if (typeof value !== "string") return "";
let v = value.trim();
let prev: string;
do {
prev = v;
v = v.replace(/^(["'])([\s\S]*)\1$/, "$2").trim();
} while (v !== prev);
return v.toLowerCase();
}
// ── Store locations ───────────────────────────────────────────────────────
/** Project-scoped store: lives inside the working directory so it travels with the repo. */
export function projectStoreDir(cwd: string): string {
return join(cwd, ".pi", "memory");
}
/** User-scoped store: machine-global, alongside pi's own ~/.pi tree. */
export function userStoreDir(home: string): string {
return join(home, ".pi", "memory");
}
export function storeDirForScope(scope: MemoryScope, cwd: string, home: string): string {
return scope === "project" ? projectStoreDir(cwd) : userStoreDir(home);
}
// ── Slugs ───────────────────────────────────────────────────────────────
/**
* Filename-safe slug derived from a memory name. Lowercased, runs of
* non-alphanumerics collapsed to a single dash, dashes trimmed. The slug is
* the file identity within a scope — writing with a name that slugifies to an
* existing file is an UPDATE of that memory.
*/
export function slugify(name: string): string {
return name
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "");
}
// ── Frontmatter ────────────────────────────────────────────────────────────
export interface Frontmatter {
meta: Record<string, string>;
body: string;
}
const FRONTMATTER_FENCE = "---";
/**
* Parse a markdown file with optional leading `--- ... ---` frontmatter into a
* flat string→string map plus the remaining body. Intentionally tiny: values
* are single-line strings (optionally quoted). No nested structures — the
* memory model only needs scalar fields. Files without frontmatter return an
* empty meta map and the whole text as the body.
*/
export function parseFrontmatter(raw: string): Frontmatter {
const normalized = raw.replace(/\r\n/g, "\n");
if (!normalized.startsWith(`${FRONTMATTER_FENCE}\n`)) {
return { meta: {}, body: normalized.replace(/^\n+/, "") };
}
const rest = normalized.slice(FRONTMATTER_FENCE.length + 1);
const closeIdx = rest.indexOf(`\n${FRONTMATTER_FENCE}`);
if (closeIdx === -1) {
// Unterminated fence — treat the whole thing as body, don't throw.
return { meta: {}, body: normalized };
}
const block = rest.slice(0, closeIdx);
const afterFence = rest.slice(closeIdx + 1 + FRONTMATTER_FENCE.length);
const body = afterFence.replace(/^\n+/, "");
const meta: Record<string, string> = {};
for (const line of block.split("\n")) {
if (!line.trim()) continue;
const sep = line.indexOf(":");
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (!key) continue;
meta[key] = unquote(line.slice(sep + 1).trim());
}
return { meta, body };
}
function unquote(value: string): string {
if (value.length >= 2) {
const first = value[0];
const last = value[value.length - 1];
if ((first === '"' && last === '"') || (first === "'" && last === "'")) {
return value.slice(1, -1);
}
}
return value;
}
/** Collapse newlines so a value stays on one frontmatter line. Quote when needed. */
function frontmatterValue(value: string): string {
const oneLine = value.replace(/\s*\n\s*/g, " ").trim();
// Quote if it could be misread (leading/trailing space, or a leading char
// YAML treats specially). Cheap and conservative.
if (oneLine === "" || /^[\s"'>|@`%#&*!?{}\[\],]/.test(oneLine)) {
return JSON.stringify(oneLine);
}
return oneLine;
}
// ── Canonical memory model ──────────────────────────────────────────────
export interface CanonicalMemory {
scope: MemoryScope;
/** Original human-readable name from frontmatter (NOT the slug). */
name: string;
type: MemoryType;
description: string;
body: string;
created: string;
updated: string;
/** Absolute path of the backing file. */
path: string;
}
export function serializeMemory(mem: {
name: string;
type: MemoryType;
description: string;
body: string;
created: string;
updated: string;
}): string {
const lines = [
FRONTMATTER_FENCE,
`name: ${frontmatterValue(mem.name)}`,
`description: ${frontmatterValue(mem.description)}`,
`type: ${mem.type}`,
`created: ${mem.created}`,
`updated: ${mem.updated}`,
FRONTMATTER_FENCE,
"",
mem.body.trim(),
"",
];
return lines.join("\n");
}
/** Read + parse one canonical memory file. Returns null if it can't be read. */
export function readMemoryFile(path: string, scope: MemoryScope): CanonicalMemory | null {
let raw: string;
try {
raw = readFileSync(path, "utf8");
} catch {
return null;
}
const { meta, body } = parseFrontmatter(raw);
const filenameName = path
.split(/[\\/]/)
.pop()!
.replace(/\.md$/i, "");
const name = meta.name?.trim() || filenameName;
const type = isMemoryType(meta.type) ? meta.type : "project";
return {
scope,
name,
type,
description: meta.description?.trim() ?? "",
body: body.trim(),
created: meta.created?.trim() || "",
updated: meta.updated?.trim() || "",
path,
};
}
/** List every canonical memory in one scope's directory. Missing dir → []. */
export function listMemories(dir: string, scope: MemoryScope): CanonicalMemory[] {
if (!existsSync(dir)) return [];
let names: string[];
try {
names = readdirSync(dir);
} catch {
return [];
}
const out: CanonicalMemory[] = [];
for (const entry of names) {
if (!entry.toLowerCase().endsWith(".md")) continue;
const full = join(dir, entry);
try {
if (!statSync(full).isFile()) continue;
} catch {
continue;
}
const mem = readMemoryFile(full, scope);
if (mem) out.push(mem);
}
return out;
}
export interface WriteResult {
memory: CanonicalMemory;
created: boolean;
}
/**
* Upsert a canonical memory. The slug of `name` is the file identity, so
* writing a name that resolves to an existing file updates it (preserving the
* original `created` timestamp). `now` is injectable for deterministic tests.
*/
export function writeMemory(
dir: string,
input: { name: string; type: MemoryType; description: string; body: string },
now: string = new Date().toISOString(),
): WriteResult {
const slug = slugify(input.name);
if (!slug) {
throw new Error("memory name must contain at least one alphanumeric character");
}
mkdirSync(dir, { recursive: true });
const path = join(dir, `${slug}.md`);
const existing = existsSync(path) ? readMemoryFile(path, "project") : null;
const created = existing?.created || now;
const serialized = serializeMemory({
name: input.name.trim(),
type: input.type,
description: input.description.trim(),
body: input.body,
created,
updated: now,
});
writeFileSync(path, serialized, "utf8");
return {
created: existing === null,
memory: {
scope: "project",
name: input.name.trim(),
type: input.type,
description: input.description.trim(),
body: input.body.trim(),
created,
updated: now,
path,
},
};
}
/** Delete a canonical memory by name. Returns true if a file was removed. */
export function deleteMemory(dir: string, name: string): boolean {
const slug = slugify(name);
if (!slug) return false;
const path = join(dir, `${slug}.md`);
if (!existsSync(path)) return false;
rmSync(path);
return true;
}
/** Find a memory by name within a pre-listed set (slug-insensitive match). */
export function findByName(
memories: readonly CanonicalMemory[],
name: string,
): CanonicalMemory | undefined {
const slug = slugify(name);
return memories.find((m) => slugify(m.name) === slug);
}
// ── Index ───────────────────────────────────────────────────────────────
/**
* Build the human/LLM-facing index of canonical memories. One line per memory:
* - [scope/type] name — description
* Sorted by scope, then type, then name. Returns "" for an empty store so
* callers can cheaply decide whether to inject anything.
*/
export function buildIndex(memories: readonly CanonicalMemory[]): string {
if (memories.length === 0) return "";
const sorted = [...memories].sort(
(a, b) =>
a.scope.localeCompare(b.scope) ||
a.type.localeCompare(b.type) ||
a.name.localeCompare(b.name),
);
return sorted
.map((m) => {
const desc = m.description ? `${m.description}` : "";
return `- [${m.scope}/${m.type}] ${m.name}${desc}`;
})
.join("\n");
}
// ── Search ──────────────────────────────────────────────────────────────
export interface SearchHit {
memory: CanonicalMemory;
/** Number of case-insensitive substring occurrences across searched fields. */
score: number;
/** A short body/description excerpt around the first match. */
snippet: string;
}
function countOccurrences(haystack: string, needle: string): number {
if (!needle) return 0;
let count = 0;
let idx = haystack.indexOf(needle);
while (idx !== -1) {
count++;
idx = haystack.indexOf(needle, idx + needle.length);
}
return count;
}
function makeSnippet(memory: CanonicalMemory, lowerQuery: string): string {
const fields = [memory.description, memory.body];
for (const field of fields) {
const idx = field.toLowerCase().indexOf(lowerQuery);
if (idx === -1) continue;
const start = Math.max(0, idx - 40);
const end = Math.min(field.length, idx + lowerQuery.length + 60);
const prefix = start > 0 ? "…" : "";
const suffix = end < field.length ? "…" : "";
return `${prefix}${field.slice(start, end).replace(/\s+/g, " ").trim()}${suffix}`;
}
return memory.description || memory.name;
}
/**
* Case-insensitive substring search across name, description, type and body.
* Ranked by total occurrence count, then most-recently-updated. Empty query
* returns nothing.
*/
export function searchMemories(
memories: readonly CanonicalMemory[],
query: string,
): SearchHit[] {
const lowerQuery = query.trim().toLowerCase();
if (!lowerQuery) return [];
const hits: SearchHit[] = [];
for (const memory of memories) {
const haystack = [memory.name, memory.description, memory.type, memory.body]
.join("\n")
.toLowerCase();
const score = countOccurrences(haystack, lowerQuery);
if (score > 0) {
hits.push({ memory, score, snippet: makeSnippet(memory, lowerQuery) });
}
}
hits.sort((a, b) => b.score - a.score || b.memory.updated.localeCompare(a.memory.updated));
return hits;
}
+13 -3
View File
@@ -17,6 +17,7 @@
# ai-server the pi extension (mTLS llama.cpp router provider)
# dark-mechanicus theme TUI bundle (banner, indicator, status line, etc.)
# token-stats footer owner for context + token rate display
# memory cross-agent memory access extension (memory/)
# themes JSON theme palette files (themes/*.json)
# local-llama split local llama.cpp extension (llama.cpp/)
# ai-complete shell CLI for direct llama-server access (installed to
@@ -64,7 +65,7 @@ SKIP_VERIFY=0
LEGACY_NO_CERTS=0
LEGACY_NO_SSH=0
ALL_COMPONENTS=(certs ssh ai-server dark-mechanicus token-stats themes local-llama ai-complete shared)
ALL_COMPONENTS=(certs ssh ai-server dark-mechanicus token-stats memory themes local-llama ai-complete shared)
usage() { sed -n '2,/^$/p' "$0" | sed 's/^#\{0,1\} \{0,1\}//'; exit 0; }
@@ -111,8 +112,8 @@ if [[ $LEGACY_NO_SSH -eq 1 ]]; then
COMPONENTS=$(echo "$COMPONENTS" | tr ',' '\n' | grep -vx 'ssh' | paste -sd, -)
fi
# Auto-add 'shared' if dark-mechanicus or token-stats is selected (they import from ../shared/ or ./shared/)
if [[ ",$COMPONENTS," == *",dark-mechanicus,"* || ",$COMPONENTS," == *",token-stats,"* ]]; then
# Auto-add 'shared' if dark-mechanicus, token-stats, or memory is selected (they import from ../shared/ or ./shared/)
if [[ ",$COMPONENTS," == *",dark-mechanicus,"* || ",$COMPONENTS," == *",token-stats,"* || ",$COMPONENTS," == *",memory,"* ]]; then
if [[ ",$COMPONENTS," != *",shared,"* ]]; then
COMPONENTS="$COMPONENTS,shared"
fi
@@ -196,6 +197,15 @@ if want token-stats; then
fi
fi
if want memory; then
if [[ -d memory ]]; then
echo; echo "==> [memory] syncing cross-agent memory extension"
rsync "${RSYNC_FLAGS[@]}" --exclude='.git' memory "$PI_DIR/extensions/"
else
echo " (memory/ not in repo, skipping)"
fi
fi
if want shared; then
if [[ -d shared ]]; then
echo; echo "==> [shared] syncing shared TS utils"
+460
View File
@@ -0,0 +1,460 @@
/**
* Unit tests for the memory extension's pure + fs-backed modules.
*
* node --experimental-strip-types --test tests/memory.test.ts
*
* store.ts and sources.ts import only Node built-ins, so they load directly
* under --experimental-strip-types (the repo has no local node_modules). fs
* tests run against throwaway temp directories.
*/
import assert from "node:assert/strict";
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { test } from "node:test";
import {
buildIndex,
type CanonicalMemory,
deleteMemory,
findByName,
isMemoryScope,
isMemoryType,
listMemories,
normalizeEnumArg,
parseFrontmatter,
projectStoreDir,
searchMemories,
serializeMemory,
slugify,
storeDirForScope,
userStoreDir,
writeMemory,
} from "../memory/store.ts";
import {
claudeMemoryDir,
claudeProjectSlugCandidates,
discoverForeignSources,
findForeignByName,
formatForeignDigest,
formatForeignFull,
isForeignSystem,
} from "../memory/sources.ts";
// ── helpers ───────────────────────────────────────────────────────────────
function tempRoot(): string {
return mkdtempSync(join(tmpdir(), "pi-memory-"));
}
function mem(overrides: Partial<CanonicalMemory> = {}): CanonicalMemory {
return {
scope: "project",
name: "sample",
type: "project",
description: "a sample",
body: "body text",
created: "2026-01-01T00:00:00.000Z",
updated: "2026-01-01T00:00:00.000Z",
path: "/tmp/sample.md",
...overrides,
};
}
// ── slugify ─────────────────────────────────────────────────────────────
test("slugify: lowercases and dashes non-alphanumerics", () => {
assert.equal(slugify("Testing Policy"), "testing-policy");
assert.equal(slugify(" Auth / Login!! "), "auth-login");
assert.equal(slugify("already-kebab"), "already-kebab");
});
test("slugify: collapses runs and trims edge dashes", () => {
assert.equal(slugify("a---b c"), "a-b-c");
assert.equal(slugify("!!!only!!!"), "only");
assert.equal(slugify("***"), "");
});
// ── type/scope guards ─────────────────────────────────────────────────────
test("isMemoryType / isMemoryScope: accept vocabulary, reject others", () => {
for (const t of ["user", "feedback", "project", "reference"]) {
assert.equal(isMemoryType(t), true);
}
assert.equal(isMemoryType("note"), false);
assert.equal(isMemoryType(undefined), false);
assert.equal(isMemoryScope("project"), true);
assert.equal(isMemoryScope("user"), true);
assert.equal(isMemoryScope("global"), false);
});
// ── normalizeEnumArg ──────────────────────────────────────────────────────
test("normalizeEnumArg: strips model double-quoting, whitespace, and casing", () => {
assert.equal(normalizeEnumArg('"feedback"'), "feedback");
assert.equal(normalizeEnumArg("'user'"), "user");
assert.equal(normalizeEnumArg(' "USER" '), "user");
assert.equal(normalizeEnumArg('""project""'), "project");
assert.equal(normalizeEnumArg("reference"), "reference");
assert.equal(normalizeEnumArg(""), "");
assert.equal(normalizeEnumArg(undefined), "");
assert.equal(normalizeEnumArg(123), "");
});
test("normalizeEnumArg: output satisfies the vocabulary guards", () => {
// The exact failure from the bug report: model sent type as '"feedback"'.
assert.equal(isMemoryType(normalizeEnumArg('"feedback"')), true);
assert.equal(isMemoryScope(normalizeEnumArg(' "user" ')), true);
assert.equal(isForeignSystem(normalizeEnumArg('"claude"')), true);
});
// ── store dirs ────────────────────────────────────────────────────────────
test("store dirs: project lives under cwd, user under home", () => {
assert.equal(projectStoreDir("/repo"), join("/repo", ".pi", "memory"));
assert.equal(userStoreDir("/home/u"), join("/home/u", ".pi", "memory"));
assert.equal(storeDirForScope("project", "/repo", "/home/u"), projectStoreDir("/repo"));
assert.equal(storeDirForScope("user", "/repo", "/home/u"), userStoreDir("/home/u"));
});
// ── parseFrontmatter ──────────────────────────────────────────────────────
test("parseFrontmatter: extracts scalar fields and body", () => {
const raw = ["---", "name: foo", "type: feedback", "---", "", "the body", "more"].join("\n");
const { meta, body } = parseFrontmatter(raw);
assert.equal(meta.name, "foo");
assert.equal(meta.type, "feedback");
assert.equal(body, "the body\nmore");
});
test("parseFrontmatter: unquotes quoted values and keeps interior colons", () => {
const raw = ["---", 'description: "see: details"', "name: 'x'", "---", "b"].join("\n");
const { meta } = parseFrontmatter(raw);
assert.equal(meta.description, "see: details");
assert.equal(meta.name, "x");
});
test("parseFrontmatter: no frontmatter returns empty meta and full body", () => {
const { meta, body } = parseFrontmatter("\n\njust text");
assert.deepEqual(meta, {});
assert.equal(body, "just text");
});
test("parseFrontmatter: normalizes CRLF", () => {
const raw = "---\r\nname: foo\r\n---\r\n\r\nbody";
const { meta, body } = parseFrontmatter(raw);
assert.equal(meta.name, "foo");
assert.equal(body, "body");
});
test("parseFrontmatter: unterminated fence treated as body", () => {
const raw = "---\nname: foo\nno close here";
const { meta, body } = parseFrontmatter(raw);
assert.deepEqual(meta, {});
assert.equal(body, raw);
});
// ── serializeMemory round-trip ──────────────────────────────────────────
test("serializeMemory: round-trips through parseFrontmatter", () => {
const serialized = serializeMemory({
name: "Testing Policy",
description: "Integration tests hit a real DB",
type: "feedback",
body: "## Why\nbecause prod migration broke once",
created: "2026-01-01T00:00:00.000Z",
updated: "2026-02-01T00:00:00.000Z",
});
const { meta, body } = parseFrontmatter(serialized);
assert.equal(meta.name, "Testing Policy");
assert.equal(meta.description, "Integration tests hit a real DB");
assert.equal(meta.type, "feedback");
assert.equal(meta.created, "2026-01-01T00:00:00.000Z");
assert.equal(meta.updated, "2026-02-01T00:00:00.000Z");
// parseFrontmatter is a low-level parser: it preserves the file's trailing
// newline. The store (readMemoryFile/writeMemory) trims body, so the
// canonical pipeline stays clean — assert on the trimmed value.
assert.equal(body.trim(), "## Why\nbecause prod migration broke once");
});
// ── writeMemory / readMemoryFile / listMemories / delete ──────────────────
test("writeMemory: creates a file readable back via listMemories", () => {
const root = tempRoot();
try {
const dir = projectStoreDir(root);
const res = writeMemory(
dir,
{ name: "Role", description: "user is a data scientist", type: "user", body: "details here" },
"2026-03-01T00:00:00.000Z",
);
assert.equal(res.created, true);
assert.equal(res.memory.name, "Role");
assert.ok(res.memory.path.endsWith(join(".pi", "memory", "role.md")));
const listed = listMemories(dir, "project");
assert.equal(listed.length, 1);
assert.equal(listed[0].name, "Role");
assert.equal(listed[0].type, "user");
assert.equal(listed[0].description, "user is a data scientist");
assert.equal(listed[0].body, "details here");
assert.equal(listed[0].scope, "project");
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("writeMemory: re-write with same name updates and preserves created", () => {
const root = tempRoot();
try {
const dir = projectStoreDir(root);
writeMemory(
dir,
{ name: "Role", description: "v1", type: "user", body: "first" },
"2026-03-01T00:00:00.000Z",
);
const second = writeMemory(
dir,
{ name: "Role", description: "v2", type: "user", body: "second" },
"2026-04-01T00:00:00.000Z",
);
assert.equal(second.created, false);
assert.equal(second.memory.created, "2026-03-01T00:00:00.000Z");
assert.equal(second.memory.updated, "2026-04-01T00:00:00.000Z");
const listed = listMemories(dir, "project");
assert.equal(listed.length, 1, "same name must not create a second file");
assert.equal(listed[0].body, "second");
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("writeMemory: rejects names with no alphanumerics", () => {
const root = tempRoot();
try {
assert.throws(() =>
writeMemory(projectStoreDir(root), {
name: "***",
description: "x",
type: "user",
body: "b",
}),
);
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("listMemories: missing dir returns empty, ignores non-markdown", () => {
const root = tempRoot();
try {
assert.deepEqual(listMemories(join(root, "nope"), "project"), []);
const dir = projectStoreDir(root);
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, "notes.txt"), "ignored", "utf8");
writeFileSync(join(dir, "real.md"), "---\nname: Real\ntype: project\n---\nbody", "utf8");
const listed = listMemories(dir, "project");
assert.equal(listed.length, 1);
assert.equal(listed[0].name, "Real");
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("deleteMemory: removes by name, returns false when absent", () => {
const root = tempRoot();
try {
const dir = projectStoreDir(root);
writeMemory(dir, { name: "Temp Note", description: "d", type: "project", body: "b" });
assert.equal(deleteMemory(dir, "Temp Note"), true);
assert.equal(listMemories(dir, "project").length, 0);
assert.equal(deleteMemory(dir, "Temp Note"), false);
assert.equal(deleteMemory(dir, "never-existed"), false);
} finally {
rmSync(root, { recursive: true, force: true });
}
});
// ── findByName ────────────────────────────────────────────────────────────
test("findByName: matches slug-insensitively", () => {
const memories = [mem({ name: "Testing Policy" }), mem({ name: "Other" })];
assert.equal(findByName(memories, "testing policy")?.name, "Testing Policy");
assert.equal(findByName(memories, "TESTING-POLICY")?.name, "Testing Policy");
assert.equal(findByName(memories, "missing"), undefined);
});
// ── buildIndex ────────────────────────────────────────────────────────────
test("buildIndex: empty store yields empty string", () => {
assert.equal(buildIndex([]), "");
});
test("buildIndex: one line per memory, sorted by scope/type/name", () => {
const out = buildIndex([
mem({ scope: "user", type: "user", name: "zeta", description: "" }),
mem({ scope: "project", type: "feedback", name: "beta", description: "b desc" }),
mem({ scope: "project", type: "feedback", name: "alpha", description: "a desc" }),
]);
assert.equal(
out,
[
"- [project/feedback] alpha — a desc",
"- [project/feedback] beta — b desc",
"- [user/user] zeta",
].join("\n"),
);
});
// ── searchMemories ────────────────────────────────────────────────────────
test("searchMemories: empty query returns nothing", () => {
assert.deepEqual(searchMemories([mem()], " "), []);
});
test("searchMemories: case-insensitive substring across fields", () => {
const hits = searchMemories([mem({ body: "the DATABASE migration" })], "database");
assert.equal(hits.length, 1);
assert.ok(hits[0].snippet.toLowerCase().includes("database"));
});
test("searchMemories: ranks by occurrence count then recency", () => {
const many = mem({ name: "many", body: "db db db", description: "db", updated: "2026-01-01" });
const few = mem({ name: "few", body: "db once", description: "", updated: "2026-09-01" });
const hits = searchMemories([few, many], "db");
assert.equal(hits[0].memory.name, "many");
assert.equal(hits[1].memory.name, "few");
});
test("searchMemories: ties on score break by most-recently-updated", () => {
const older = mem({ name: "older", body: "db", description: "", updated: "2026-01-01" });
const newer = mem({ name: "newer", body: "db", description: "", updated: "2026-09-01" });
const hits = searchMemories([older, newer], "db");
assert.equal(hits[0].memory.name, "newer");
});
// ── claude slug derivation ──────────────────────────────────────────────
test("claudeProjectSlugCandidates: slash form first, then non-alnum form, deduped", () => {
const cands = claudeProjectSlugCandidates("/home/u/My Proj.v2");
assert.equal(cands[0], "-home-u-My Proj.v2");
assert.equal(cands[1], "-home-u-My-Proj-v2");
assert.equal(new Set(cands).size, cands.length);
});
test("claudeProjectSlugCandidates: dedupes when both forms match", () => {
const cands = claudeProjectSlugCandidates("/home/u/proj");
assert.deepEqual(cands, ["-home-u-proj"]);
});
// ── foreign sources ───────────────────────────────────────────────────────
test("isForeignSystem: validates the adapter set", () => {
assert.equal(isForeignSystem("claude"), true);
assert.equal(isForeignSystem("aider"), true);
assert.equal(isForeignSystem("emacs"), false);
});
test("claudeMemoryDir: resolves the existing slug directory", () => {
const root = tempRoot();
try {
const cwd = join(root, "proj");
const home = join(root, "home");
mkdirSync(cwd, { recursive: true });
const slug = claudeProjectSlugCandidates(cwd)[0];
const memDir = join(home, ".claude", "projects", slug, "memory");
mkdirSync(memDir, { recursive: true });
assert.equal(claudeMemoryDir(cwd, home), memDir);
assert.equal(claudeMemoryDir(join(root, "absent"), home), null);
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("discoverForeignSources: reads and source-tags all systems", () => {
const root = tempRoot();
try {
const cwd = join(root, "proj");
const home = join(root, "home");
mkdirSync(join(cwd, ".github", "instructions"), { recursive: true });
mkdirSync(join(home, ".claude"), { recursive: true });
mkdirSync(join(home, ".gemini"), { recursive: true });
writeFileSync(join(cwd, "CLAUDE.md"), "project claude guidance", "utf8");
writeFileSync(join(cwd, "CLAUDE.local.md"), " ", "utf8"); // empty → skipped
writeFileSync(join(cwd, "AGENTS.md"), "agents content", "utf8");
writeFileSync(join(cwd, "GEMINI.md"), "project gemini", "utf8");
writeFileSync(join(cwd, "CONVENTIONS.md"), "aider conventions", "utf8");
writeFileSync(join(cwd, ".github", "copilot-instructions.md"), "copilot main", "utf8");
writeFileSync(
join(cwd, ".github", "instructions", "ts.instructions.md"),
"copilot ts rules",
"utf8",
);
writeFileSync(join(home, ".claude", "CLAUDE.md"), "user claude", "utf8");
writeFileSync(join(home, ".gemini", "GEMINI.md"), "user gemini", "utf8");
const slug = claudeProjectSlugCandidates(cwd)[0];
const memDir = join(home, ".claude", "projects", slug, "memory");
mkdirSync(memDir, { recursive: true });
writeFileSync(join(memDir, "MEMORY.md"), "- index entry", "utf8");
writeFileSync(join(memDir, "user_role.md"), "user is a dev", "utf8");
const sources = discoverForeignSources(cwd, home);
const bySystem = (sys: string) => sources.filter((s) => s.system === sys);
// claude: project CLAUDE.md + user CLAUDE.md + 2 auto-memory files = 4
assert.equal(bySystem("claude").length, 4);
assert.ok(bySystem("claude").some((s) => s.scope === "user"));
assert.equal(bySystem("copilot").length, 2);
assert.equal(bySystem("agents").length, 1);
assert.equal(bySystem("gemini").length, 2);
assert.equal(bySystem("aider").length, 1);
// Empty CLAUDE.local.md was skipped.
assert.equal(
sources.some((s) => s.path.endsWith("CLAUDE.local.md")),
false,
);
// systems filter narrows the result set.
const onlyAgents = discoverForeignSources(cwd, home, ["agents"]);
assert.equal(onlyAgents.length, 1);
assert.equal(onlyAgents[0].system, "agents");
assert.equal(onlyAgents[0].content, "agents content");
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test("findForeignByName: matches basename with/without .md, case-insensitive", () => {
const sources = [
{
system: "claude" as const,
scope: "project" as const,
path: "/home/u/.claude/projects/x/memory/MEMORY.md",
content: "the index",
},
{
system: "claude" as const,
scope: "project" as const,
path: "/home/u/.claude/projects/x/memory/project_nextcloud.md",
content: "nextcloud notes",
},
];
assert.equal(findForeignByName(sources, "MEMORY.md")?.content, "the index");
assert.equal(findForeignByName(sources, "memory")?.content, "the index");
assert.equal(findForeignByName(sources, "PROJECT_NEXTCLOUD.md")?.content, "nextcloud notes");
assert.equal(findForeignByName(sources, "nope"), null);
assert.equal(findForeignByName(sources, ""), null);
});
test("formatForeignDigest / formatForeignFull: shape output", () => {
const sources = [
{ system: "claude" as const, scope: "project" as const, path: "/p/CLAUDE.md", content: "abc" },
];
assert.equal(formatForeignDigest(sources), "- [claude/project] /p/CLAUDE.md (3 chars)");
assert.equal(formatForeignDigest([]), "");
assert.equal(formatForeignFull(sources), "### claude · project · /p/CLAUDE.md\n\nabc");
});