llm-multiverse/llm-multiverse
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Set up DuckDB schema and VSS extension #28

New Issue
Closed
opened 2026-03-08 10:48:42 +01:00 by shahondin1624 · 1 comment
shahondin1624 commented 2026-03-08 10:48:42 +01:00
Owner
Copy Link

Description

Configure DuckDB with the VSS (Vector Similarity Search) extension. Define the memory storage schema with tables for memories, embeddings, provenance, and correlations.

Acceptance Criteria

  • DuckDB initialized with VSS extension loaded
  • Schema: memories table (id, content, timestamp, session_id, agent_id, provenance)
  • Schema: embeddings table linked to memories with vector column
  • Schema: correlations table for cross-reference lookups
  • HNSW index configured for vector similarity search
  • Schema migration support

Blocked by

## Description Configure DuckDB with the VSS (Vector Similarity Search) extension. Define the memory storage schema with tables for memories, embeddings, provenance, and correlations. ## Acceptance Criteria - [ ] DuckDB initialized with VSS extension loaded - [ ] Schema: memories table (id, content, timestamp, session_id, agent_id, provenance) - [ ] Schema: embeddings table linked to memories with vector column - [ ] Schema: correlations table for cross-reference lookups - [ ] HNSW index configured for vector similarity search - [ ] Schema migration support ## Blocked by - #27
shahondin1624 added this to the Phase 4: Memory Service milestone 2026-03-08 10:48:42 +01:00
shahondin1624 commented 2026-03-09 18:59:39 +01:00
Author
Owner
Copy Link

Implementation Complete

PR #113 implements the DuckDB schema and VSS extension for the Memory Service.

What was implemented:

  • DuckDbManager with VSS extension loading, connection management (Mutex<Connection>), and with_connection() API
  • Schema: memories, embeddings (FLOAT[768] vectors), correlations, memory_tags, schema_version tables
  • HNSW index for vector similarity search (deferred creation pattern for empty tables)
  • Versioned migration system with idempotency

Quality Gates: All PASS

  • Build, Clippy, Tests (25 memory-service, 87 total), Coverage (97%+ on new files)

Notable deviations:

  • DuckDB doesn't support ON DELETE CASCADE — manual deletion order required
  • DuckDB VSS doesn't support filtered HNSW indexes — single unfiltered index with query-time filtering

🤖 Auto-implemented by Claude Code

## Implementation Complete PR #113 implements the DuckDB schema and VSS extension for the Memory Service. ### What was implemented: - `DuckDbManager` with VSS extension loading, connection management (`Mutex<Connection>`), and `with_connection()` API - Schema: `memories`, `embeddings` (FLOAT[768] vectors), `correlations`, `memory_tags`, `schema_version` tables - HNSW index for vector similarity search (deferred creation pattern for empty tables) - Versioned migration system with idempotency ### Quality Gates: All PASS - Build, Clippy, Tests (25 memory-service, 87 total), Coverage (97%+ on new files) ### Notable deviations: - DuckDB doesn't support `ON DELETE CASCADE` — manual deletion order required - DuckDB VSS doesn't support filtered HNSW indexes — single unfiltered index with query-time filtering 🤖 Auto-implemented by Claude Code
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
cat:agent
Agent related
 cat:cross-cutting
Cross-cutting concern
 cat:deployment
Deployment related
 cat:devops
DevOps related
 cat:performance
Performance related
 cat:security
Security related
 cat:tech-debt
Technical debt from code reviews
 lang:protobuf
Protocol Buffers
 lang:python
Python language
 lang:rust
Rust language
 priority:critical
Critical priority
 priority:high
High priority
 priority:low
Low priority
 priority:medium
Medium priority
 service:audit
Audit service
 service:memory
Memory service
 service:model-gateway
Model Gateway service
 service:orchestrator
Orchestrator service
 service:search
Search service
 service:secrets
Secrets service
 service:tool-broker
Tool Broker service
 type:bug
Bug fix
 type:documentation
Docs
 type:feature
New functionality
 type:infrastructure
Build tooling, project setup
 type:refactor
Restructuring
 type:research
Investigation, benchmarking
 type:testing
Tests
 workflow:blocked
Blocked after max retries
 workflow:manual
Requires manual intervention — blocked from auto-dev
No Label
Milestone
No items
No Milestone Phase 4: Memory Service
Projects
Clear projects
No project
Assignees
Clear assignees
shahondin1624
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: llm-multiverse/llm-multiverse#28
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?