Skip to content

examples: add skills-based automatic code review agent - #256

Open
zhangfan-x wants to merge 1 commit into
trpc-group:mainfrom
zhangfan-x:feat/skills-code-review-agent
Open

examples: add skills-based automatic code review agent#256
zhangfan-x wants to merge 1 commit into
trpc-group:mainfrom
zhangfan-x:feat/skills-code-review-agent

Conversation

@zhangfan-x

Copy link
Copy Markdown

实现 issue #92「基于 Skills + 沙箱 + 数据库存储构建自动代码评审 Agent」,新增示例 examples/skills_code_review_agent/

Fixes #92

方案设计说明

总体链路:CLI 将输入(unified diff / git 工作区 / 文件列表)解析为文件-hunk-候选行号结构并落库;LlmAgent 仅暴露 skill_load / skill_run 两个工具(刻意不用完整 SkillToolSet,避免暴露 workspace_exec 等第二执行面);skill_load 注入规则文档,skill_run 单次调用在沙箱内执行 driver 脚本串行跑 6 类静态检查,findings 经 $OUTPUT_DIR 文件通道回传(绕开 stdout 截断);有 API key 时 LLM 单次复核并可补充发现。dry-run 用脚本化 FakeModel 发起同样的工具调用,无 key 也真实覆盖 Skills / Filter / 沙箱 / 落库全链路。

Skill 设计:SKILL.md + docs/rules-*.md(6 类规则)+ scripts/。diff 解析器单一实现,宿主与沙箱两侧复用,永不分歧。repo 模式重建完整 post-image 跑 AST;diff-only 模式保持行号对齐,AST 失败降级正则并降置信度。

沙箱隔离:默认 Container runtime——禁网(network_mode=none,附容器内出网探针断言)、宿主环境变量不进容器、skills 目录只读挂载;超时钳制 + 输出截断,超时/失败返回结构化结果不崩溃,任务降级为 partial 并落库。local 仅 --unsafe-local 显式回退。

Filter 策略:三层不重叠——工具级 allowed_cmds 白名单;ReviewToolFilter 前置拦截脚本越权(未知脚本 → needs_human_review)、路径逃逸、env 注入、超预算,并钳制超时参数,全部决策(含 allow)写 filter_event 表;agent 级 after_tool_callback 统一脱敏。deny / needs_human_review 均不进入沙箱执行。

数据库:SqlStorage 自定义 metadata 建 7 表(review_task / diff_file / sandbox_run / filter_event / finding / report / metrics),SQLite 默认,DSN 可切换其他 SQL 后端。

去重降噪:exact 键 sha256(rule_id+file+行号+归一化证据) 配 UNIQUE 约束;同 (file, line, category) 合并保留最高 severity,其余以 suppressed 状态留档。确定性决策表分流:高精度静态直接进 findings,低精度按类别进 warnings / needs_human_review;LLM 补充必须逐字引用 diff 否则丢弃。

监控:metrics 表记录总耗时、沙箱耗时、工具调用数、拦截数、finding 数、severity / 异常分布、token 用量、阶段耗时。

安全边界:容器禁网、env 白名单、超时与输出上限、Redactor(20+ 密钥格式 + allowlist)在工具输出 / 落库 / 报告三处兜底;diff 内容在 prompt 中框定为不可信数据(附提示注入 fixture 断言结论不变)。

交付物对照

  • skills/code-review/:SKILL.md、6 类规则文档、沙箱检查脚本
  • review_agent/:Agent 入口、diff 解析、Filter、存储、脱敏、指标
  • fixtures/:14 个 diff 测试样例(覆盖题目要求的 8 类场景及边界情况)
  • sample_output/review_report.json / .md 示例输出
  • README.md 使用说明、DESIGN.md 方案设计说明
  • tests/ 单元与端到端测试,tests/evaluation/test_skills_code_review_example.py 接入仓库评测目录

测试

$ pytest examples/skills_code_review_agent/tests tests/evaluation/test_skills_code_review_example.py
40 passed in 48s

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@f2a34ff). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             main        #256   +/-   ##
==========================================
  Coverage        ?   88.19009%           
==========================================
  Files           ?         482           
  Lines           ?       45157           
  Branches        ?           0           
==========================================
  Hits            ?       39824           
  Misses          ?        5333           
  Partials        ?           0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zhangfan-x

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Rook1ex added a commit to trpc-group/cla-database that referenced this pull request Jul 29, 2026
@zhangfan-x

Copy link
Copy Markdown
Author

recheck

基于 Skill 体系、沙箱执行、Filter 治理和 SQL 存储实现自动代码评审
Agent 示例(issue trpc-group#92)。支持解析 unified diff / git 工作区变更,加载
覆盖 6 类规则的 code-review Skill,经 Filter 放行后在禁网容器沙箱中
执行静态检查,对 findings 去重降噪、敏感信息脱敏,并将审查任务、
沙箱执行、拦截记录、findings、报告和监控指标写入 SQLite(DSN 可
切换其他 SQL 后端)。提供脚本化 fake model 的 dry-run 模式,无
API key 也可完整覆盖解析、沙箱和落库链路。

Fixes trpc-group#92
@zhangfan-x
zhangfan-x force-pushed the feat/skills-code-review-agent branch from 5b99606 to 4afed31 Compare July 29, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

基于 Skills + 沙箱 + 数据库存储构建自动代码评审 Agent

1 participant