52eaf661c4
Add InferenceStats proto message and InferenceStatsPanel component that displays token counts, throughput, and context window utilization as a collapsible panel below assistant messages after orchestration completes. Closes #43 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
61 lines
2.1 KiB
Markdown
61 lines
2.1 KiB
Markdown
# Issue #43: Display Inference Statistics in Chat UI
|
|
|
|
**Status:** IMPLEMENTING
|
|
**Issue:** [#43](https://git.shahondin1624.de/llm-multiverse/llm-multiverse-ui/issues/43)
|
|
**Branch:** `feature/issue-43-inference-stats`
|
|
|
|
## Overview
|
|
|
|
Add a collapsible UI panel that displays LLM inference statistics (token counts, context window utilization, throughput) below assistant messages after orchestration completes.
|
|
|
|
## Phases
|
|
|
|
### Phase 1: Proto Types
|
|
|
|
**Files:**
|
|
- `proto/upstream/proto/llm_multiverse/v1/common.proto` — Add `InferenceStats` message
|
|
- `proto/upstream/proto/llm_multiverse/v1/orchestrator.proto` — Add optional `inference_stats` field to `ProcessRequestResponse`
|
|
|
|
**InferenceStats message fields:**
|
|
- `prompt_tokens` (uint32) — tokens in the prompt
|
|
- `completion_tokens` (uint32) — tokens generated
|
|
- `total_tokens` (uint32) — sum of prompt + completion
|
|
- `context_window_size` (uint32) — model's maximum context length
|
|
- `tokens_per_second` (float) — generation throughput
|
|
|
|
**Then regenerate types:** `npm run generate`
|
|
|
|
### Phase 2: Orchestration State
|
|
|
|
**Files:**
|
|
- `src/lib/composables/useOrchestration.svelte.ts` — Extract `inferenceStats` from response, expose via store getter
|
|
|
|
### Phase 3: InferenceStatsPanel Component
|
|
|
|
**Files:**
|
|
- `src/lib/components/InferenceStatsPanel.svelte` — New component
|
|
|
|
**Design:**
|
|
- Follow `<details>` pattern from FinalResult.svelte
|
|
- Collapsed by default
|
|
- Summary line shows key stat (e.g., total tokens + tokens/sec)
|
|
- Expanded content shows all stats in a grid layout
|
|
- Context utilization shown as a progress bar
|
|
- Blue/indigo color scheme (neutral, info-like)
|
|
- Full dark mode support
|
|
|
|
### Phase 4: Chat Page Integration
|
|
|
|
**Files:**
|
|
- `src/routes/chat/+page.svelte` — Render `InferenceStatsPanel` after `FinalResult` when stats available
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [x] InferenceStats proto message defined and TypeScript types generated
|
|
- [x] InferenceStatsPanel displays all required metrics
|
|
- [x] Panel is collapsible, collapsed by default
|
|
- [x] Context utilization shows visual progress bar
|
|
- [x] Integrates cleanly into chat page below assistant message
|
|
- [x] Dark mode support
|
|
- [x] Build, lint, typecheck pass
|