test: add session memory summary replay framework - #249
Conversation
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #249 +/- ##
==========================================
Coverage ? 87.87129%
==========================================
Files ? 482
Lines ? 45248
Branches ? 0
==========================================
Hits ? 39760
Misses ? 5488
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR introduces a cross-backend “replay consistency” framework for Session / Memory / Summary semantics, and updates the session services and summarization pipeline so persisted summary events become the recovery source of truth and backend replays converge across InMemory/SQLite (with opt-in Redis/SQL integration runs).
Changes:
- Add a reusable replay harness + 10 standard JSONL traces that execute operations, reread from storage, normalize volatile fields, inject faults, and produce a JSON diff report.
- Make event appends idempotent (Event ID as session-scoped key) and add “failed write retry” repair behavior across InMemory / Redis / SQL backends.
- Persist structured, versioned Summary metadata as a summary anchor event (with ordering gap for SQL timestamp rounding) and recover summary cache from persisted summary events.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| trpc_agent_sdk/sessions/_summarizer_manager.py | Rebuild summary cache from persisted summary events; enforce session ownership checks. |
| trpc_agent_sdk/sessions/_sql_session_service.py | Idempotent append_event with SQL-side existence checks; align active/historical windows and scoped state writes. |
| trpc_agent_sdk/sessions/_session_summarizer.py | Add structured, versioned summary event metadata + recovery helper; ordering timestamp gap logic for SQL. |
| trpc_agent_sdk/sessions/_redis_session_service.py | Idempotent append_event + retry repair; fix multi-field HSET by using redis-py mapping; decode bytes hash state. |
| trpc_agent_sdk/sessions/_in_memory_session_service.py | Idempotent append_event + retry repair; store deep-isolated session snapshots on update_session. |
| trpc_agent_sdk/sessions/_base_session_service.py | Centralize session-scoped idempotency via _append_event_to_session returning appended status + filtered events. |
| trpc_agent_sdk/sessions/init.py | Export public replay-facing summary metadata key + recovery helper. |
| trpc_agent_sdk/memory/_sql_memory_service.py | Treat store_session as complete snapshot replacement (delete stale rows in-scope). |
| tests/sessions/test_summarizer_manager.py | Add tests for cache recovery from persisted summary event + ownership rejection. |
| tests/sessions/test_session_summarizer.py | Test that ordering gap survives SQL float→datetime rounding. |
| tests/sessions/test_replay_unit.py | Verify normalizer keeps Summary semantics while normalizing only volatile metadata fields. |
| tests/sessions/test_replay_harness_consistency.py | New tests for harness invariants, lightweight mode, summary fault localization, optional Redis/SQL integration. |
| tests/sessions/test_replay_consistency.py | Remove known drift list (now expected consistent). |
| tests/sessions/test_redis_session_service.py | Tests for HSET mapping writes and bytes→str hash decoding behavior. |
| tests/sessions/test_in_memory_session_service.py | Test ensuring update_session does not share event list with storage (no duplicate append). |
| tests/sessions/test_base_session_service.py | Test Event-ID idempotency at BaseSessionService level. |
| tests/sessions/session_memory_summary_diff_report.json | Update expected sample report to reflect fixed consistency (no mismatches). |
| tests/sessions/replay/normalizer.py | Normalize only volatile summary metadata fields using exported key. |
| tests/sessions/replay_harness.py | New replay harness implementation: run cases, reread storage, normalize/compare, inject faults, emit report. |
| tests/sessions/replay_cases/standard_cases.jsonl | New catalog of 10 standard replay traces. |
| tests/memory/test_sql_memory_service.py | Test SQL memory snapshot replacement deletes stale rows and avoids duplicates. |
| session_memory_summary_diff_report.json | New generated root-level report output for the replay suite. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # This is either the normal first write or recovery of a write that | ||
| # stopped after local mutation; both paths append exactly once here. | ||
| # 此处既处理首次写入，也补偿“本地已改、存储未写”的失败，且只追加一次。 | ||
| storage_session.events.append(event) | ||
|
|
| # Prefer replay metadata while retaining safe defaults for legacy events. | ||
| # 优先使用回放元数据，同时为旧事件保留安全的字段回退值。 | ||
| version = metadata.get("version", event.version or 1) | ||
| original_event_count = metadata.get("original_event_count", 0) | ||
| compressed_event_count = metadata.get("compressed_event_count", 0) | ||
| summary_timestamp = metadata.get("summary_timestamp", event.timestamp) | ||
| replaces_summary_id = metadata.get("replaces_summary_id") | ||
|
|
||
| return SessionSummary( | ||
| summary_id=event.id, | ||
| session_id=session_id, | ||
| summary_text=summary_text, | ||
| version=int(version), | ||
| replaces_summary_id=replaces_summary_id if isinstance(replaces_summary_id, str) else None, | ||
| original_event_count=int(original_event_count), | ||
| compressed_event_count=int(compressed_event_count), | ||
| summary_timestamp=float(summary_timestamp), | ||
| metadata=metadata, | ||
| ) |
AI Code Review我已完整审查 发现的问题🚨 Critical无。
|
|
I have read the CLA Document and I hereby sign the CLA |
AI Code ReviewRedis 会立即进行序列化，因此别名（aliasing）没有危害。很好。 我已经完成了审查。该 diff 非常稳健，测试覆盖率极佳，未发现严重或警告级别的问题。让我来写最终结论。 发现的问题未发现 Critical 或 Warning 级别的阻塞问题。本 PR 的生产代码改动（SQL Memory 快照替换语义、InMemory/Redis/SQL 的 💡 Suggestion
总结整体风险低，未发现必须修复的问题；生产代码的关键正确性改动均有配套测试覆盖，回放框架与报告产物迁移（ 测试建议暂无额外测试建议。现有测试已覆盖幂等重试、跨后端窗口一致性、摘要版本/归属/覆盖链恢复、Redis 可达性探测与凭据不泄露等关键风险路径。 |
Summary
新增 Session / Memory / Summary 跨后端回放一致性框架。10 条标准轨迹覆盖对话、工具调用、State、Memory、Summary、事件截断和失败重试，默认比较 InMemory 与 SQLite，并支持可选 Redis/SQL 集成测试。
Changes
Testing
在项目根目录激活虚拟环境：
source .venv/bin/activate运行完整 Replay 测试：
运行 Redis 测试：
redis-server --port 6389 --bind 127.0.0.1 --save "" --appendonly no --daemonize yes redis-cli -u redis://127.0.0.1:6389/15 ping TRPC_REPLAY_REDIS_URL=redis://127.0.0.1:6389/15 \ python -m pytest \ tests/sessions/test_replay_harness_consistency.py::test_optional_redis_integration \ tests/sessions/test_replay_injections.py::TestEndToEndRedisInjection \ -v redis-cli -p 6389 shutdown nosave