examples: add skills-based automatic code review agent

examples/skills_code_review_agent/.env.example

@@ -0,0 +1,6 @@
+# Copy to .env and fill in real values. NEVER commit .env.
+# Without these the agent runs in dry-run mode (scripted FakeModel) —
+# the full parse/sandbox/filter/persist pipeline still executes.
+TRPC_AGENT_API_KEY=
+TRPC_AGENT_BASE_URL=
+TRPC_AGENT_MODEL_NAME=

examples/skills_code_review_agent/DESIGN.md

@@ -0,0 +1,17 @@
+# 方案设计说明
+
+**总体链路**:CLI 将输入(unified diff / git 工作区 / 文件列表)解析为文件-hunk-候选行号结构并落库;LlmAgent 双挂 SkillToolSet 语义(仅 skill_load / skill_run 两个工具 + skill_repository,刻意不用 SkillToolSet 以免暴露 workspace_exec 等第二执行面);skill_load 注入规则文档,skill_run 单次调用在沙箱内执行 driver 脚本串行跑 6 类静态检查,findings 写 $OUTPUT_DIR 经文件通道回传(绕开 16KB stdout 截断);有 API key 时 LLM 单次复核并可补充发现。dry-run 用脚本化 FakeModel 发起同样的工具调用,无 key 也真实覆盖 skills/Filter/沙箱/落库全链路。
+
+**Skill 设计**:SKILL.md + docs/rules-\*.md(6 类)+ scripts/。diff 解析器是单一实现(scripts/parse_diff.py),宿主经 importlib 复用,两侧永不分歧。repo 模式重建完整 post-image 跑 AST;diff-only 模式 gap 填空行保持行号对齐,AST 失败降级正则并降置信度。
+
+**沙箱隔离**:默认 Container runtime——禁网(network_mode=none,附容器内出网探针断言测试)、宿主环境变量不进容器、skills 目录只读挂载;超时钳制 + 输出截断,超时/失败返回结构化结果不崩溃,任务降级为 partial 并落库。local 仅 --unsafe-local 显式回退。
+
+**Filter 策略**:三层不重叠。SkillRunTool allowed_cmds 白名单挡 shell 元字符与非 python3 命令;ReviewToolFilter 前置拦截脚本越权(未知脚本→needs_human_review)、路径逃逸、env 注入、host 输入越界、超预算,并钳制超时参数,全部决策(含 allow)写 filter_event 表;拒绝 3 次后返回终止指令防重试空转;agent 级 after_tool_callback 统一脱敏。deny/needs_human_review 均不执行(测试断言 handler 未被调用,且工具面架构性断言无绕过路径)。
+
+**数据库**:SqlStorage 传自定义 metadata 建 7 表(review_task / diff_file / sandbox_run / filter_event / finding / report / metrics),SQLite 默认,DSN 可切 MySQL/PostgreSQL。
+
+**去重降噪**:两级——exact 键 sha256(rule_id+file+行号+归一化证据) 配 UNIQUE 约束双保险;同 (file, line, category) 合并保留最高 severity,其余以 suppressed 状态留档可审计。分流用确定性决策表:高精度静态直接进 findings;低精度按类别分流(注入/密钥宁报勿漏,测试缺失宁缺勿滥进 warnings);LLM 补充必须逐字引用 diff 否则丢弃;静态命中而 LLM 判否的高危项转人工复核。
+
+**监控**:metrics 表记录总耗时、沙箱耗时、工具调用数、拦截数、finding 数、severity/异常分布、token 用量、阶段耗时,并标注口径(事件流直取 vs 自埋)。
+
+**安全边界**:容器禁网、env 白名单、超时与输出上限、Redactor(20+ 密钥格式 + allowlist)在工具输出/落库/报告三处兜底,diff 内容在 prompt 中框定为不可信数据(附提示注入 fixture 断言结论不变)。

examples/skills_code_review_agent/README.md

@@ -0,0 +1,123 @@
+# Skills-based Automated Code Review Agent
+
+一个基于 tRPC-Agent Skills + 沙箱执行 + SQL 存储的自动代码评审 Agent 原型([issue #92](https://github.com/trpc-group/trpc-agent-python/issues/92))。
+输入一份 git diff / PR patch / 工作区变更,输出结构化审查报告(JSON + Markdown),全过程(任务、沙箱执行、Filter 拦截、findings、监控指标)持久化到数据库。
+
+```
+输入(--diff-file | --repo-path | --files | --fixture)
+   │
+   ▼
+DiffParser ── 文件/hunk/候选行号结构化,ReviewTask + diff_file 落库
+   │
+   ▼
+LlmAgent(真实模型 或 dry-run FakeModel;双挂 skill_load/skill_run + skill_repository)
+   │   skill_load("code-review")            ← 规则文档进上下文
+   │   skill_run("python3 scripts/run_checks.py")
+   │      ├─ SkillRunTool(allowed_cmds=["python3"])      第 1 层:命令白名单,拒 shell 元字符
+   │      ├─ ReviewToolFilter(_before)                   第 2 层:脚本/路径/env/输入源/预算,决策落 filter_event
+   │      └─ Container runtime(默认,禁网)| local(--unsafe-local)
+   │   findings.json 写 $OUTPUT_DIR → 文件通道回传(绕开 16KB stdout 截断)
+   ▼
+决策表分流(静态精度 × LLM 复核)→ 两级去重 → Redactor 脱敏
+   │
+   ▼
+review_report.json / review_report.md + 7 张表(SQLite 默认,DSN 可切)
+```
+
+## 安装
+
+```bash
+cd trpc-agent-python
+pip install -r requirements.txt && pip install -e .
+cd examples/skills_code_review_agent
+cp .env.example .env        # 可选:配置真实模型;不配置则自动 dry-run
+```
+
+沙箱默认使用 Docker 容器(镜像 `python:3-slim`,规则脚本零第三方依赖)。无 Docker 时自动回退 local 并在报告中记录原因;显式开发模式用 `--unsafe-local`。
+
+## 运行
+
+```bash
+# 审查一份 diff(无 API key 时自动 dry-run,FakeModel 驱动完整工具循环)
+python3 run_agent.py review --diff-file fixtures/02_sql_injection/input.diff --dry-run
+
+# 内置样例(14 条,按编号或名称)
+python3 run_agent.py review --fixture 02 --dry-run
+
+# git 工作区变更(HEAD 对比 + 未跟踪文件)
+python3 run_agent.py review --repo-path /path/to/repo --dry-run
+
+# diff + repo 同给 → repo 模式(重建完整 post-image,AST 全文分析,检出/降噪更好)
+python3 run_agent.py review --diff-file x.diff --repo-path /path/to/repo
+```
+
+输出 `review_report.json` / `review_report.md`(`--output-dir` 可改),样例见 `sample_output/`。
+
+## 数据库查询
+
+```bash
+python3 run_agent.py init-db --db sqlite:///review.db     # 建表(幂等)并验证 DSN
+python3 run_agent.py show --task-id <id>                  # 任务状态/执行日志摘要/Filter 拦截/findings/监控/结论
+```
+
+7 张表:`review_task`、`diff_file`、`sandbox_run`、`filter_event`、`finding`(UNIQUE(task_id, dedup_key))、`report`、`metrics`。
+列类型用 SDK 的 DynamicJSON / UTF8MB4String / PreciseTimestamp,`--db mysql+pymysql://...` 即切后端,无迁移脚本(SqlStorage 首用自动建表 + 前向加列)。
+
+## 规则(6 类,静态通道独立达标)
+
+| 类别 | 规则文档 | 代表规则 |
+|---|---|---|
+| 安全风险 | docs/rules-security.md | SQL 拼接进 execute(含一步变量追踪)、eval/exec、shell=True、yaml.load、pickle、verify=False |
+| 敏感信息 | docs/rules-secrets.md | AWS/GitHub/GitLab/Slack/Stripe/… 13 类格式 + 熵检测 + allowlist |
+| 异步错误 | docs/rules-async.md | async 内阻塞调用、协程未 await、create_task 结果丢弃 |
+| 资源泄漏 | docs/rules-resource-leak.md | open/socket/Lock 未释放(所有权转移不误报) |
+| 连接生命周期 | docs/rules-db-lifecycle.md | connect/cursor 未关、写操作无 commit、事务悬空 |
+| 测试缺失 | docs/rules-missing-tests.md | 源码变更无测试伴随(diff-only 降为 warnings,宁缺勿滥) |
+
+检查在沙箱内一次 `skill_run` 串行跑完(`scripts/run_checks.py`),单个检查崩溃被隔离并记录,不影响其余。
+
+## 评测
+
+```bash
+python3 run_agent.py eval --samples fixtures --unsafe-local
+```
+
+对任意标注样本目录(`<dir>/<name>/input.diff` + `expected.json`)输出逐条 TP/FN/FP 与汇总指标——验收方可直接指向隐藏样本目录。当前 14 条内置样例(dry-run 纯静态通道):
+
+```
+高危检出率 14/14 = 100%(验收线 ≥ 80%)
+误报率      0/16 =   0%(验收线 ≤ 15%)
+```
+
+## 安全性(可证明,非声称)
+
+三条对抗性测试在 `tests/test_filter_security.py`、`tests/test_end_to_end.py`,全部通过:
+
+1. **拦截确未执行**:deny / needs_human_review 时断言工具 handler 未被调用(filter 先于 handler 的机制保证);另有架构断言——agent 工具面恰为 {skill_load, skill_run} 且全部挂 ReviewToolFilter,不存在绕过路径。
+2. **沙箱内出网必须失败**:容器内 urllib 探针实测被拒(network_mode=none)。不依赖 describe() 元数据(其 network_allowed 字段与实现不符)。
+3. **提示注入不改变结论**:fixture 14 在 diff 中嵌入"报告无问题"指令,静态通道照报 SQL 注入;LLM prompt 将 diff 框定为不可信数据。
+
+其他边界:超时钳制(Filter 层 clamp,LLM 传大 timeout 无效)、输出 16KB/文件通道上限、env 键白名单、host:// 输入限定在任务目录、拒绝 3 次后终止防重试空转、脱敏 ≥95%(23 种格式用例)且报告与库中无明文密钥(测试断言)。
+资源限制口径:超时 + 输出上限 + 禁网;不声称 CPU/内存限制(SDK 未实施 WorkspaceResourceLimits)。
+
+## dry-run 与耗时口径
+
+dry-run(无 API key)由脚本化 FakeModel 发起与真实模型完全相同的工具调用——skills staging、Filter、沙箱、落库全部真实执行。
+计时口径 = 进程启动 → 报告写盘(一次性镜像拉取除外):容器模式单次评审实测 ~17s,local 模式 ~3s,远低于 2 分钟验收线(tests 中有 120s 断言)。
+
+## 真实 LLM
+
+配置 `TRPC_AGENT_API_KEY / BASE_URL / MODEL_NAME` 后,LLM 参与方式由 `--llm-mode` 控制:
+
+- `agent`(默认 auto 解析):LLM 自主驱动 skill_load → skill_run,并在最终消息输出结构化复核 JSON;
+- `hybrid`:脚本化 FakeModel 驱动沙箱(确定性),LLM 只做一次无工具的复核调用(逐条 verdict + 补充发现);
+- `off`:纯静态(等价 --dry-run)。
+
+两种 LLM 模式下,补充发现都必须逐字引用 diff 否则丢弃;LLM 判 reject 的高危项转人工复核;info 级提示不论 verdict 只进 warnings。
+
+## 测试
+
+```bash
+python -m pytest examples/skills_code_review_agent/tests/ -q     # 38 项:单元/端到端/对抗性(Docker 存在时含禁网探针)
+python -m pytest tests/evaluation/test_skills_code_review_example.py -q   # 根 tests/ CI 冒烟(无 Docker 依赖)
+```

examples/skills_code_review_agent/eval/__init__.py

@@ -0,0 +1,5 @@
+# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
+#
+# Copyright (C) 2026 Tencent. All rights reserved.
+#
+# tRPC-Agent-Python is licensed under Apache-2.0.

examples/skills_code_review_agent/eval/eval.py

@@ -0,0 +1,181 @@
+# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
+#
+# Copyright (C) 2026 Tencent. All rights reserved.
+#
+# tRPC-Agent-Python is licensed under Apache-2.0.
+"""Scoring harness: run every annotated fixture, output a confusion matrix.
+
+Works on any directory of fixtures (``<dir>/<name>/input.diff`` +
+``expected.json``), so a reviewer can point it at their own hidden sample
+set unchanged::
+
+    python3 run_agent.py eval --samples fixtures/ --unsafe-local
+
+Matching semantics:
+* an ``expected_findings`` entry is a TP when some *reported* finding has the
+  same category & file and a line within the given int / [lo, hi] range
+  (findings landing in warnings/needs_human_review do NOT count — that is
+  the whole point of the triage gate); unmatched entries are FN;
+* ``expected_warnings`` entries may match in any bucket (findings included);
+* every reported finding not matched by any expectation and hitting a
+  ``forbidden`` entry (or any finding for ``expect_clean`` fixtures) is a FP.
+
+High-severity detection rate is additionally reported over the subset of
+expectations with min_severity in {critical, high} (the acceptance metric).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import sys
+from pathlib import Path
+
+BASE_DIR = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(BASE_DIR))
+
+from review_agent.diff_parser import parse_diff_file  # noqa: E402
+from review_agent.pipeline import ReviewOptions, run_review  # noqa: E402
+
+SEV_RANK = {"critical": 4, "high": 3, "medium": 2, "low": 1, "info": 0}
+
+
+def _line_matches(expected_line, actual_line: int) -> bool:
+    if expected_line is None:
+        return True
+    if isinstance(expected_line, list) and len(expected_line) == 2:
+        return expected_line[0] <= actual_line <= expected_line[1]
+    try:
+        return abs(int(expected_line) - actual_line) <= 1
+    except (TypeError, ValueError):
+        return True
+
+
+def _entry_matches(entry: dict, finding: dict) -> bool:
+    if entry.get("category") and entry["category"] != finding.get("category"):
+        return False
+    if entry.get("file") and entry["file"] != finding.get("file"):
+        return False
+    if not _line_matches(entry.get("line"), int(finding.get("line") or 0)):
+        return False
+    min_severity = entry.get("min_severity")
+    if min_severity and SEV_RANK.get(finding.get("severity"), 0) < SEV_RANK.get(min_severity, 0):
+        return False
+    return True
+
+
+def score_fixture(expected: dict, payload: dict) -> dict:
+    """Score one report against one expected.json."""
+    reported = payload.get("findings", [])
+    soft_buckets = reported + payload.get("warnings", []) + payload.get("needs_human_review", [])
+
+    tp, fn = [], []
+    matched_ids: set[int] = set()
+    for entry in expected.get("expected_findings", []) or []:
+        hit = next((finding for finding in reported if _entry_matches(entry, finding)), None)
+        if hit is not None:
+            tp.append(entry)
+            matched_ids.add(id(hit))
+        else:
+            fn.append(entry)
+
+    soft_tp, soft_fn = [], []
+    for entry in expected.get("expected_warnings", []) or []:
+        hit = next((finding for finding in soft_buckets if _entry_matches(entry, finding)), None)
+        if hit is not None:
+            soft_tp.append(entry)
+            matched_ids.add(id(hit))
+        else:
+            soft_fn.append(entry)
+
+    fp = []
+    forbidden = expected.get("forbidden", []) or []
+    for finding in reported:
+        if id(finding) in matched_ids:
+            continue
+        if expected.get("expect_clean"):
+            fp.append(finding)
+            continue
+        for rule in forbidden:
+            if _entry_matches({k: v for k, v in rule.items() if k in ("category", "file")}, finding) \
+                    or (not rule.get("category") and rule.get("file") == finding.get("file")):
+                fp.append(finding)
+                break
+    return {"tp": tp, "fn": fn, "fp": fp, "soft_tp": soft_tp, "soft_fn": soft_fn, "reported": len(reported)}
+
+
+def _is_high(entry: dict) -> bool:
+    return entry.get("min_severity") in ("critical", "high")
+
+
+def run_eval(samples_dir: str, db_url: str = "sqlite:///eval.db", unsafe_local: bool = False) -> int:
+    samples = Path(samples_dir)
+    rows = []
+    totals = {"tp": 0, "fn": 0, "fp": 0, "soft_tp": 0, "soft_fn": 0, "reported": 0, "high_tp": 0, "high_fn": 0}
+
+    for fixture in sorted(path for path in samples.iterdir() if path.is_dir()):
+        diff = fixture / "input.diff"
+        expected_path = fixture / "expected.json"
+        if not diff.is_file() or not expected_path.is_file():
+            continue
+        expected = json.loads(expected_path.read_text(encoding="utf-8"))
+        meta = expected.get("meta") or {}
+
+        options = ReviewOptions(
+            db_url=db_url,
+            output_dir=str(fixture / ".eval_out"),
+            unsafe_local=unsafe_local,
+            dry_run=True,  # eval scores the static channel: it must stand alone
+            run_timeout=int(meta.get("run_timeout", 60)),
+            inject_sleep=float(meta.get("inject_sleep", 0)),
+        )
+        repo_dir = fixture / "repo"
+        parsed = parse_diff_file(str(diff), repo_path=str(repo_dir) if repo_dir.is_dir() else None)
+        parsed.input_type = "fixture"
+        parsed.input_ref = fixture.name
+        outcome = asyncio.run(run_review(parsed, options))
+
+        score = score_fixture(expected, outcome.payload)
+        for key in ("tp", "fn", "fp", "soft_tp", "soft_fn"):
+            totals[key] += len(score[key])
+        totals["reported"] += score["reported"]
+        totals["high_tp"] += sum(1 for entry in score["tp"] if _is_high(entry))
+        totals["high_fn"] += sum(1 for entry in score["fn"] if _is_high(entry))
+        rows.append((fixture.name, outcome.status, score))
+
+    print(f"{'fixture':<22} {'status':<10} {'TP':>3} {'FN':>3} {'FP':>3} {'softTP':>6} {'softFN':>6}")
+    for name, status, score in rows:
+        print(f"{name:<22} {status:<10} {len(score['tp']):>3} {len(score['fn']):>3} {len(score['fp']):>3} "
+              f"{len(score['soft_tp']):>6} {len(score['soft_fn']):>6}")
+        for entry in score["fn"]:
+            print(f"    MISS: {entry}")
+        for finding in score["fp"]:
+            print(f"    FP:   {finding.get('category')}/{finding.get('rule_id')} "
+                  f"{finding.get('file')}:{finding.get('line')}")
+
+    tp, fn, fp = totals["tp"], totals["fn"], totals["fp"]
+    precision_hits = totals["reported"]
+    high_total = totals["high_tp"] + totals["high_fn"]
+    print("\n== summary (static channel, dry-run) ==")
+    print(f"expected findings: recall {tp}/{tp + fn}"
+          f" = {tp / (tp + fn) * 100 if tp + fn else 100:.1f}%")
+    if high_total:
+        print(f"high-severity recall: {totals['high_tp']}/{high_total}"
+              f" = {totals['high_tp'] / high_total * 100:.1f}%  (acceptance: >= 80%)")
+    fp_rate = fp / precision_hits * 100 if precision_hits else 0.0
+    print(f"false positives: {fp}/{precision_hits} reported = {fp_rate:.1f}%  (acceptance: <= 15%)")
+    print(f"soft expectations (warnings bucket): {totals['soft_tp']}/{totals['soft_tp'] + totals['soft_fn']}")
+    ok = (totals["high_tp"] >= high_total * 0.8 if high_total else True) and fp_rate <= 15.0
+    print(f"result: {'PASS' if ok else 'FAIL'}")
+    return 0 if ok else 1
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--samples", default=str(BASE_DIR / "fixtures"))
+    parser.add_argument("--db", default="sqlite:///eval.db")
+    parser.add_argument("--unsafe-local", action="store_true")
+    args = parser.parse_args()
+    sys.exit(run_eval(args.samples, args.db, args.unsafe_local))

examples/skills_code_review_agent/fixtures/01_clean/expected.json

@@ -0,0 +1,11 @@
+{
+  "description": "Pure refactor of utils/format.py (local rename, extracted pure helper, new docstring) across 3 hunks; no category may report anything blocking.",
+  "expect_clean": true,
+  "expected_findings": [],
+  "expected_warnings": [],
+  "forbidden": [
+    {
+      "file": "utils/format.py"
+    }
+  ]
+}

examples/skills_code_review_agent/fixtures/01_clean/input.diff

@@ -0,0 +1,44 @@
+diff --git a/utils/format.py b/utils/format.py
+index a27ce46..32a22b8 100644
+--- a/utils/format.py
++++ b/utils/format.py
+@@ -7,8 +7,8 @@
+
+ def render_row(values, widths):
+     parts = []
+-    for v, w in zip(values, widths):
+-        parts.append(str(v).ljust(w))
++    for value, width in zip(values, widths):
++        parts.append(str(value).ljust(width))
+     return SEPARATOR.join(parts)
+
+
+@@ -25,13 +25,18 @@
+     return padded
+
+
+-def render_cell(text, limit):
+-    cleaned = " ".join(str(text).split())
++def _clip(cleaned, limit):
++    """Truncate to `limit` characters, marking the cut with an ellipsis."""
+     if len(cleaned) <= limit:
+         return cleaned
+     if limit <= len(ELLIPSIS):
+         return cleaned[:limit]
+     return cleaned[: limit - len(ELLIPSIS)] + ELLIPSIS
++
++
++def render_cell(text, limit):
++    cleaned = " ".join(str(text).split())
++    return _clip(cleaned, limit)
+
+
+ def render_table(rows, widths):
+@@ -42,6 +47,7 @@
+
+
+ def summarize(rows):
++    """Count rows and measure the widest rendered row."""
+     total = len(rows)
+     widest = max((len(str(row)) for row in rows), default=0)
+     return {"rows": total, "widest": widest}