feat: implement semantic cache (#32) #121

shahondin1624 · 2026-03-09T23:46:10+01:00

shahondin1624 commented

2026-03-09 23:46:10 +01:00

Summary

Implements an in-memory semantic cache for the Memory Service's QueryMemory pipeline. Cache entries are keyed by query embedding cosine similarity (not exact match), enabling deduplication of semantically similar queries.

Key changes

New cache/ module: SemanticCache with RwLock-protected entries, CachedResult type, CacheMetrics with atomic counters
cache/similarity.rs: Pure-Rust cosine similarity with f64 intermediate precision
CacheConfig in config.rs with enabled, max_entries, ttl_secs, similarity_threshold
Service integration: Cache lookup before pipeline execution, cache population after successful query+extraction
TTL expiration: Entries expire after configurable TTL, evicted on insert
Write invalidation: invalidate_by_memory_id() ready for wiring into WriteMemory (currently unimplemented)
Metrics: Hit/miss/eviction/invalidation counters with periodic logging in main.rs
Proto: is_cached and cached_extracted_segment fields populated on cache hits

Quality gates

Build: PASS
Clippy: PASS (zero warnings)
Tests: PASS (201 tests, 0 failures)
Coverage: PASS (cache/mod.rs: 99.1%, similarity.rs: 100%)

Files

Created: cache/mod.rs, cache/similarity.rs, extraction/mod.rs, extraction/prompt.rs, issue-031.md, issue-032.md
Modified: memory.proto, config.rs, service.rs, main.rs, lib.rs, _index.md

Note: This PR includes #31 (extraction step) changes as a dependency since #31's PR (#119) has not been merged yet.

Closes #32 | Plan: implementation-plans/issue-032.md

## Summary Implements an in-memory semantic cache for the Memory Service's `QueryMemory` pipeline. Cache entries are keyed by query embedding cosine similarity (not exact match), enabling deduplication of semantically similar queries. ### Key changes - **New `cache/` module:** `SemanticCache` with `RwLock`-protected entries, `CachedResult` type, `CacheMetrics` with atomic counters - **`cache/similarity.rs`:** Pure-Rust cosine similarity with f64 intermediate precision - **`CacheConfig`** in `config.rs` with `enabled`, `max_entries`, `ttl_secs`, `similarity_threshold` - **Service integration:** Cache lookup before pipeline execution, cache population after successful query+extraction - **TTL expiration:** Entries expire after configurable TTL, evicted on insert - **Write invalidation:** `invalidate_by_memory_id()` ready for wiring into `WriteMemory` (currently unimplemented) - **Metrics:** Hit/miss/eviction/invalidation counters with periodic logging in main.rs - **Proto:** `is_cached` and `cached_extracted_segment` fields populated on cache hits ### Quality gates - Build: PASS - Clippy: PASS (zero warnings) - Tests: PASS (201 tests, 0 failures) - Coverage: PASS (cache/mod.rs: 99.1%, similarity.rs: 100%) ### Files - **Created:** `cache/mod.rs`, `cache/similarity.rs`, `extraction/mod.rs`, `extraction/prompt.rs`, `issue-031.md`, `issue-032.md` - **Modified:** `memory.proto`, `config.rs`, `service.rs`, `main.rs`, `lib.rs`, `_index.md` Note: This PR includes #31 (extraction step) changes as a dependency since #31's PR (#119) has not been merged yet. Closes #32 | Plan: `implementation-plans/issue-032.md`

shahondin1624 added 12 commits 2026-03-09 23:46:10 +01:00

feat: semantic cache (#32 ): issue-031.md 13cebc225f

feat: semantic cache (#32 ): issue-032.md d6e5902a59

feat: semantic cache (#32 ): mod.rs 35eb00b3aa

feat: semantic cache (#32 ): similarity.rs 2da5e97bf9

feat: semantic cache (#32 ): mod.rs c9677d7d5b

feat: semantic cache (#32 ): prompt.rs 02e27a6a60

feat: semantic cache (#32 ): _index.md 3d0addcd0a

feat: semantic cache (#32 ): memory.proto 9f49cab2da

feat: semantic cache (#32 ): config.rs fe284e2fdc

feat: semantic cache (#32 ): lib.rs 395f44fde3

feat: semantic cache (#32 ): main.rs 6c38044904