[Pelis Agent Factory Advisor] Pelis Agent Factory Advisor — April 4, 2026

[github-actions[bot]]

📊 Executive Summary

gh-aw-firewall has reached Level 4 agentic maturity — a rare and impressive position. With 27 active agentic workflows spanning security red-teaming, CI investigation, token cost analytics, documentation maintenance, and smoke testing across three engines, this repository is already a showcase of the Pelis Agent Factory philosophy. However, six high-value workflow patterns from the factory are still missing, and several existing workflows have actionable enhancement opportunities — particularly around issue triage, workflow health meta-monitoring, and domain-specific security automation.

---

🎓 Patterns Learned from Pelis Agent Factory

Key Patterns from the Documentation Site

1. Heterogeneous specialization — 100+ workflows, each doing one thing well. This repo has embraced this: separate Claude/Copilot/Codex smoke tests, separate token analyzers per engine, and separate secret diggers per engine.

2. Meta-agents that watch other agents — The Workflow Health Manager and Audit Workflows track other agent runs. The repo has CI Doctor but lacks a general health manager.

3. Read-only analyst + proposer separation — Some agents create reports; others propose PRs. The repo mixes both well but could better leverage the PR-proposing pattern for code quality.

4. Continuous improvement cadence — Daily/weekly code simplifier, refactoring, style consistency agents. The repo has weekly CLI flag checks but lacks daily code quality agents.

5. Causal chain automation — Issue-creating agents whose issues then trigger other agents (Issue Monster → Copilot Coding Agent). The repo uses Issue Monster well.

6. Cache-memory for persistent state — The Issue Duplication Detector already uses this pattern correctly.

7. skip-if-match idempotency guard — Prevents duplicate open PRs/issues. Already used in doc-maintainer.md and test-coverage-improver.md.

8. Strict network + safe-outputs isolation — The repo enforces this throughout, exemplary practice.

From the agentics Repository

Interesting workflows available from githubnext/agentics not yet adopted:

• daily-test-improver.md — incremental test coverage improvement
• issue-triage.md — auto-label incoming issues
• issue-arborist.md — link related issues as sub-issues
• sub-issue-closer.md — auto-close completed sub-issues
• daily-malicious-code-scan.md — scan commits for suspicious patterns
• grumpy-reviewer.md / pr-nitpick-reviewer.md — automated PR quality reviewers
• vex-generator.md — vulnerability exchange format (very relevant for a security tool)
• weekly-issue-summary.md — community summary of issue activity
• ci-coach.md — CI optimization suggestions

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test	Build/test suite across 5 language runtimes	PR, dispatch	✅ Comprehensive multi-language
ci-cd-gaps-assessment	Daily CI/CD coverage gap analysis	Daily, dispatch	✅ Good observability
ci-doctor	Investigate failed CI runs → diagnostic issues	workflow_run failure	✅ High value — missing some workflows in its watch list
claude-token-optimizer	Optimize Claude workflow prompts for token efficiency	Manual/schedule	✅ Cost-conscious
claude-token-usage-analyzer	Daily Claude token cost analysis	Daily	✅ Strong cost tracking
cli-flag-consistency-checker	Weekly CLI flag vs docs consistency	Weekly	✅ Good hygiene
copilot-token-optimizer	Optimize Copilot workflow prompts	Manual/schedule	✅ Cost-conscious
copilot-token-usage-analyzer	Daily Copilot token cost analysis	Daily	✅ Strong cost tracking
dependency-security-monitor	Daily CVE scanning + PR proposals	Daily	⚠️ Currently failing
doc-maintainer	Sync docs with code changes	Daily	✅ Active (PR #1641 open)
firewall-issue-dispatcher	Port awf-labeled issues from gh-aw → here	Every 6h	✅ Cross-repo coordination
issue-duplication-detector	Detect duplicate issues with cache-memory	Issue opened	✅ Cache-memory pattern done right
issue-monster	Dispatch issues to Copilot coding agent	Hourly, issue opened	✅ Task dispatcher pattern
pelis-agent-factory-advisor	This workflow	Scheduled	✅ Meta-advisory
plan	/plan slash command → sub-issue breakdown	Slash command	✅ ChatOps pattern
secret-digger-claude	Red-team: find secrets in agent container	Hourly	✅ Unique adversarial testing
secret-digger-codex	Red-team: find secrets (Codex engine)	Hourly	✅ Multi-engine red team
secret-digger-copilot	Red-team: find secrets (Copilot engine)	Hourly	✅ Multi-engine red team
security-guard	PR reviewer for security boundary changes	PR	⚠️ Currently failing (token issue)
security-review	Daily comprehensive security review	Daily	✅ Deep evidence-based
smoke-chroot	Chroot functionality smoke test	Every 12h, PR	⚠️ Intermittent failures
smoke-claude	Claude engine smoke test	Every 12h, PR	⚠️ Intermittent failures
smoke-codex	Codex engine smoke test	Every 12h, PR	⚠️ Intermittent failures
smoke-copilot	Copilot engine smoke test	Every 12h	✅ Stable
smoke-services	Service-level smoke test	Every 12h	✅ Stable
test-coverage-improver	Weekly security-focused test coverage PRs	Weekly	✅ Targeted security testing
update-release-notes	Enrich release notes on publish	Release	✅ Good automation

---

🚀 Actionable Recommendations

P0 — Implement Immediately

[P0] Issue Triage Agent — Auto-label Incoming Issues

What: An agent that automatically labels newly-opened issues based on content analysis.

Why: This repo receives issues from cross-repo dispatch, red team findings, CI failures, and end users. Currently issues pile up unlabeled. The factory pattern shows this is the "hello world" of agentic workflows with immediate, observable value. Given the issue tracker is already active (1600+ issues), this would improve maintainer efficiency significantly.

How: Add issue-triage-agent.md triggered on issues: [opened, reopened] with categories appropriate for this repo:

• bug, security, enhancement, documentation, question, chroot, networking, api-proxy, smoke-test, good-first-issue

Effort: Low — ~20 lines of Markdown

Example:

---
timeout-minutes: 5
on:
  issues:
    types: [opened, reopened]
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
    min-integrity: none
safe-outputs:
  add-labels:
    allowed: [bug, security, enhancement, documentation, question, chroot, networking, api-proxy, smoke-test, good-first-issue]
  add-comment: {}
---
Analyze issue #$\{\{ github.event.issue.number }} in $\{\{ github.repository }} and apply one appropriate label.

---

[P0] Add CI Doctor to Its Own Watch List + Missing Workflows

What: ci-doctor.md monitors workflow failures but its own name (Pelis Agent Factory Advisor) is not in the monitored workflow list. Also missing: Smoke Services, Dependency Security Monitor, and Firewall Issue Dispatcher.

Why: CI Doctor failing silently means it can't report its own failures. This is a simple 4-line fix.

How: Add the missing workflow names to the workflows: list in ci-doctor.md.

Effort: Trivial — 4 lines added to existing file.

---

P1 — Plan for Near-Term

[P1] Daily Malicious Code Scan — Scan Commits for Suspicious Patterns

What: A daily agent that reviews recent code changes (last 24h) for suspicious patterns: hidden backdoors, hardcoded credentials, suspicious network calls, obfuscated code.

Why: This repo IS the security firewall. It's deployed by users to protect their agents. A supply-chain attack here would be catastrophic. The Pelis Agent Factory runs this daily and it's one of the most security-relevant workflows for a project of this nature. The security-review.md workflow does comprehensive reviews but is broad; this agent focuses specifically on recent commit-level changes.

Effort: Medium — adapt from githubnext/agentics/workflows/daily-malicious-code-scan.md

---

[P1] Static Analysis Report — Daily zizmor/poutine/actionlint

What: A daily agent running zizmor, poutine, and actionlint across all workflow files and reporting findings as a discussion.

Why: The factory has created 57 analysis discussions + 12 Zizmor reports from this pattern. Given that gh-aw-firewall generates .lock.yml files from Markdown sources, and those lock files are what GitHub Actions executes, systematic static analysis of the generated workflows is critical. poutine detects supply chain risks; zizmor finds GitHub Actions security issues; actionlint catches correctness issues. The existing security-review.md uses agentic-workflows tool but doesn't systematically run these CLI tools.

Effort: Low — the agenticworkflows-compile tool with zizmor: true, poutine: true, actionlint: true provides this capability.

---

[P1] Workflow Health Manager — Meta-Agent Monitoring Workflow Health

What: A meta-agent that regularly reviews the health of all 27 agentic workflows — checking recent run success rates, identifying patterns of failure, detecting cost outliers, and creating remediation issues.

Why: Currently CI Doctor reacts to individual failures, but there's no agent that looks at trends. A workflow that was working last week but failing 60% of the time this week should get a health issue even if no single run triggered CI Doctor. The factory's Workflow Health Manager created 40 issues and 5 direct PRs + 14 causal chain PRs. This repo is complex enough to need this.

Effort: Medium — uses agentic-workflows tool + agenticworkflows-logs for trend analysis.

---

[P1] Breaking Change Checker — PR Backward Compatibility Guard

What: A PR-triggered agent that detects changes to public CLI interfaces, Docker compose schemas, environment variable contracts, and iptables rule changes that could break existing users.

Why: AWF is used as a GitHub Action (action.yml) and CLI tool by downstream users. Breaking changes to --allow-domains, container images, or env var names are high-impact. The existing security-guard.md focuses on security posture; this would focus on user-facing compatibility. The factory pattern created alert issues like #14113 catching CLI version updates.

Effort: Medium — PR-triggered, reads action.yml, src/cli.ts, src/docker-manager.ts for contract changes.

---

P2 — Consider for Roadmap

[P2] Container Security Scan — Daily Docker Image CVE Scanning

What: A daily workflow that scans the three AWF Docker images (squid, agent, api-proxy) for known CVEs using tools like trivy or grype, creating security issues for HIGH/CRITICAL findings.

Why: The dependency-security-monitor.md scans npm dependencies but not the Docker base images. The agent container is based on ubuntu:22.04 and squid on ubuntu/squid. These base images receive security updates. Users deploying AWF trust that the containers don't contain known vulnerabilities.

Effort: Medium — requires --build-local or pulling images + running scanner tool.

---

[P2] VEX Generator — Vulnerability Exchange Format Reports

What: An agent that generates VEX (Vulnerability Exploitability eXchange) documents for known vulnerabilities in AWF dependencies, documenting whether CVEs are actually exploitable in AWF's specific usage context.

Why: The dependency-security-monitor already tracks CVEs. VEX documents would formalize the security stance (e.g., "handlebars GHSA-2w6w-674q-4c4q is not exploitable because AWF doesn't process user-controlled templates in handlebars"). This is directly available from the agentics repo (vex-generator.md) and is exceptionally relevant for a security tool with open CVE issues like #1489.

Effort: Low-Medium — available from githubnext/agentics/workflows/vex-generator.md

---

[P2] Issue Arborist — Link Related Issues as Sub-Issues

What: A periodic agent that groups related issues (e.g., all chroot-related issues, all networking issues) into parent-child relationships using GitHub's sub-issues feature.

Why: With 1600+ issues, the tracker is becoming unwieldy. The factory's Issue Arborist created 77 discussion reports and 18 parent issues. For AWF, grouping [aw]-labeled CI failure issues, chroot issues, and networking issues would make the backlog navigable.

Effort: Low — available from githubnext/agentics/workflows/issue-arborist.md

---

[P2] PR Reviewer / Grumpy Reviewer

What: An automated PR reviewer that provides detailed technical feedback on code changes, focusing on TypeScript correctness, Docker security, iptables rule safety, and container architecture concerns.

Why: The security-guard.md reviews for security posture changes. A general code quality reviewer would catch TypeScript anti-patterns, Docker best practices violations, and iptables rule errors before human reviewers spend time on them.

Effort: Medium — can be adapted from githubnext/agentics/workflows/grumpy-reviewer.md

---

[P2] Changeset Generator — Automated Version Bumps and Changelogs

What: An agent that auto-generates version bump PRs and CHANGELOG entries for every merge to main, determining whether the change is patch, minor, or major based on the diff.

Why: update-release-notes.md enriches notes after a release. A changeset agent would automate the decision of when to cut a release and what version to use. The factory's Changeset workflow has a 78% merge rate across 28 proposed PRs. AWF has an active release cadence.

Effort: Medium — available from github/gh-aw pattern.

---

P3 — Future Ideas

[P3] Weekly Issue Summary — Community Activity Digest

What: A weekly discussion post summarizing issue activity: new issues opened, issues closed, top contributors, trending topics.

Why: Builds community engagement and gives maintainers a weekly health check of the issue tracker. Available from githubnext/agentics/workflows/weekly-issue-summary.md.

Effort: Low

---

[P3] Mergefest — Auto-Merge Main into PR Branches

What: Auto-merge main into long-running PR branches to keep them up to date.

Why: With multiple AI-generated PRs in flight simultaneously (docs, test coverage, token optimization), merge conflicts accumulate. Available from the factory pattern.

Effort: Low

---

[P3] Sub Issue Closer — Auto-Close Completed Sub-Issues

What: When a parent issue is closed, automatically close completed sub-issues.

Why: With Issue Monster dispatching work and Issue Arborist grouping issues, sub-issues can become stale after parent resolution.

Effort: Low

---

[P3] Daily Repo Chronicle

What: A daily summary of notable repository activity — commits, PRs merged, issues closed — posted as a discussion.

Why: At 27 workflows running continuously, it's hard to keep track of what automated agents have done. A chronicle creates a readable log of agentic activity. Available from githubnext/agentics/workflows/daily-repo-chronicle.md.

Effort: Low

---

📈 Maturity Assessment

Dimension	Level	Notes
Current Level	4 / 5	27 active workflows, multi-engine, strong security focus
Target Level	5 / 5	Full factory with meta-monitoring and issue lifecycle automation

Level Scale:

• Level 1: Manual everything, no agentic workflows
• Level 2: Basic CI/CD + 1-3 simple agentic workflows
• Level 3: 5-15 workflows covering code quality and documentation
• Level 4: 15+ specialized workflows, multi-engine, observability + security automation ← Here
• Level 5: Full factory with meta-agents, issue lifecycle automation, continuous improvement agents

Gap to Level 5:

1. Add issue triage (gap: unlabeled issues)
2. Add workflow health manager (gap: trend-based failure detection)
3. Add malicious code scan + static analysis report (gap: daily commit-level security)
4. Add breaking change checker (gap: user-facing contract monitoring)

---

🔄 Comparison with Pelis Agent Factory Best Practices

What This Repo Does Exceptionally Well

• Adversarial testing: Running secret-digger agents (Claude, Codex, Copilot) hourly is an extraordinary commitment to security — the factory's "Firewall" workflow is its equivalent
• Multi-engine smoke testing: Testing across Claude/Codex/Copilot/chroot simultaneously is more comprehensive than most factory repos
• Token cost tracking: Separate analyzers and optimizers per engine is the right specialization
• Security depth: security-review.md with bash: true and evidence-based investigation goes beyond factory norms
• Cross-repo coordination: firewall-issue-dispatcher.md bridging gh-aw → gh-aw-firewall is a sophisticated pattern

What Could Improve

• Issue lifecycle: No auto-labeling, no arborist, no sub-issue closer — 1600+ issues with manual triage
• Daily commit hygiene: No malicious code scan or static analysis report running daily
• Meta-monitoring: CI Doctor reacts to individual failures but no agent tracks health trends across the 27-workflow ecosystem
• PR quality gate: Security Guard is the only PR agent; a general code quality reviewer is missing
• User-facing contracts: No breaking change checker for the CLI/Action API surface

Unique Opportunities Given This Repo's Domain

This repo is a firewall. That creates unique opportunities other repos don't have:

1. Self-referential firewall testing: Agents that test whether the firewall correctly blocks/allows specific domains — already done via smoke tests, but could be more systematic
2. Compliance reporting: Generate NIST SP 800-53, SOC2, or CIS Benchmark alignment reports automatically
3. Domain allowlist health: Agent that checks if allowed domains in smoke tests are still reachable and their certificates are valid
4. CVE → firewall rule: Agent that automatically proposes firewall rule changes based on CVE disclosures (e.g., "CVE-X in package Y from cdn.example.com → add that domain to security review")

---

📝 Notes for Future Runs

Stored in /tmp/gh-aw/cache-memory/advisor-notes-2026-04-04.json

• 2026-04-04: 27 workflows active. Key gaps: issue-triage, workflow-health-manager, breaking-change-checker, malicious-code-scan, static-analysis-report. CI Doctor missing itself from watch list. Multiple smoke test failures observed in open issues.
• Issues to monitor: [Security] [GHSA-2w6w-674q-4c4q] CRITICAL: JavaScript Injection in handlebars (CVSS 9.8) #1489 (critical handlebars CVE unresolved), [awf] cli/docker-manager: SIGTERM from GH Actions step timeout does not kill the agent container fast enough #1590 (SIGTERM timeout), feat: deprecate pkg binary, require Node.js >= 20 as prerequisite #1578 (deprecate pkg binary)
• PRs to watch: Docs sync PRs from doc-maintainer, test coverage PRs from test-coverage-improver

AI generated by Pelis Agent Factory Advisor

• expires on Apr 11, 2026, 3:30 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-04-11T03:30:49.485Z.

Closed by Workflow