Skip to content

feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain)#1248

Open
imran-siddique wants to merge 2 commits intogithub:stagedfrom
imran-siddique:skills/agt-security-governance
Open

feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain)#1248
imran-siddique wants to merge 2 commits intogithub:stagedfrom
imran-siddique:skills/agt-security-governance

Conversation

@imran-siddique
Copy link
Copy Markdown
Contributor

Adds 3 new skills for AI agent security and governance — an area not currently covered in the awesome-copilot collection.

New Skills

1. mcp-security-audit

Audit MCP server configurations for security issues:

  • Hardcoded secrets detection (API keys, tokens, credentials)
  • Shell injection pattern detection in server args
  • Unpinned dependency flagging (@latest)
  • Dangerous command patterns (\�val, \�ash -c, pipe-to-shell)

2. agent-owasp-compliance

Check agent systems against the OWASP Agentic Security Initiative (ASI) Top 10:

  • Maps all 10 ASI risks with detection signals and passing/failing examples
  • Generates X/10 compliance report
  • Includes quick assessment questions for rapid evaluation
  • Covers risks specific to agents (not LLMs): tool governance, trust boundaries, delegation

3. agent-supply-chain

Supply chain integrity for agent plugins and tools:

  • SHA-256 integrity manifest generation and verification
  • Tamper detection (modified, deleted, untracked files)
  • Dependency version pinning audit
  • Promotion gate pattern (dev to production readiness check)
  • CI integration example

Why These Skills

The existing \�gent-governance\ skill covers governance patterns. These 3 skills extend into specific operational areas:

  • MCP audit: MCP servers are everywhere but ungoverned
  • OWASP compliance: Industry standard that most agent systems don't measure against
  • Supply chain: No agent ecosystem has npm-provenance-equivalent integrity verification yet

Related: Agent Governance Toolkit

Copilot AI review requested due to automatic review settings April 1, 2026 04:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds three new security/governance-focused Agent Skills to expand the repository’s coverage of agent security operations: MCP configuration auditing, OWASP ASI-oriented compliance checking, and agent/plugin supply-chain integrity verification.

Changes:

  • Added a new mcp-security-audit skill describing checks for secrets, shell-injection patterns, and unpinned MCP server dependencies.
  • Added a new agent-owasp-compliance skill outlining an OWASP ASI Top 10-oriented assessment workflow and report format.
  • Added a new agent-supply-chain skill with integrity manifest generation/verification patterns and CI gating examples.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
skills/mcp-security-audit/SKILL.md New skill documentation for auditing .mcp.json server configs for secrets, injection patterns, and version pinning issues.
skills/agent-owasp-compliance/SKILL.md New skill documentation for assessing agent systems against OWASP ASI Top 10 risks and producing a compliance report.
skills/agent-supply-chain/SKILL.md New skill documentation for hashing-based integrity manifests, tamper detection, dependency pinning audits, and CI verification patterns.

Comment on lines +3 to +11
description: |
Audit MCP (Model Context Protocol) server configurations for security issues. Use this skill when:
- Reviewing .mcp.json files for security risks
- Checking MCP server args for hardcoded secrets or shell injection patterns
- Validating that MCP servers use pinned versions (not @latest)
- Detecting unpinned dependencies in MCP server configurations
- Auditing which MCP servers a project registers and whether they're on an approved list
- Checking for environment variable usage vs. hardcoded credentials in MCP configs
- Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json"
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new skills should be added to the skills index in docs/README.skills.md so they show up in the repo’s documented skill catalog/discovery list (add entries for mcp-security-audit, agent-owasp-compliance, and agent-supply-chain).

Suggested change
description: |
Audit MCP (Model Context Protocol) server configurations for security issues. Use this skill when:
- Reviewing .mcp.json files for security risks
- Checking MCP server args for hardcoded secrets or shell injection patterns
- Validating that MCP servers use pinned versions (not @latest)
- Detecting unpinned dependencies in MCP server configurations
- Auditing which MCP servers a project registers and whether they're on an approved list
- Checking for environment variable usage vs. hardcoded credentials in MCP configs
- Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json"
description: 'audit mcp server configurations in .mcp.json files for security issues including secrets exposure, shell injection, unpinned dependencies, and unapproved servers'

Copilot uses AI. Check for mistakes.
Comment on lines +131 to +135
findings = []
args_text = json.dumps(server_config.get("args", []))
for pattern, description in DANGEROUS_PATTERNS:
if re.search(pattern, args_text):
findings.append({
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This snippet calls json.dumps(...) and re.search(...) but this code block doesn’t include imports for json/re (they’re only present in a previous snippet). Add the imports here or explicitly note that this block depends on earlier imports so it’s copy/pasteable.

Copilot uses AI. Check for mistakes.
Comment on lines +147 to +151

Flag MCP servers using `@latest` or unversioned packages.

```python
def check_pinned_versions(server_config: dict) -> list[dict]:
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text says this check flags "unversioned packages", but the shown implementation only flags @latest (and otherwise doesn’t emit findings). Either implement an unversioned/unpinned package detection or adjust the wording to match what the code actually checks.

Copilot uses AI. Check for mistakes.
Comment on lines +209 to +213
continue
findings = []
findings.extend(check_secrets(config))
findings.extend(check_shell_injection(server_config))
findings.extend(check_pinned_versions(server_config))
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check_secrets(config) is executed inside the per-server loop, which will duplicate the same secret findings for every server and can misattribute them. Run the secret scan once outside the loop and/or scope it to the specific server config being evaluated.

Copilot uses AI. Check for mistakes.
Comment on lines +123 to +127
def verify_manifest(plugin_dir: str) -> tuple[bool, list[str]]:
"""Verify plugin files against INTEGRITY.json."""
root = Path(plugin_dir)
manifest_path = root / "INTEGRITY.json"

Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verify_manifest snippet relies on Path, json, hash_file, and generate_manifest, but this code block doesn’t show the needed imports/definitions. Consider adding minimal imports here or adding a brief note that it depends on the previous snippet so readers don’t copy an incomplete block.

Copilot uses AI. Check for mistakes.
Comment on lines +232 to +233
required = ["README.md", ".claude-plugin/plugin.json"]
missing = [f for f in required if not (root / f).exists()]
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

promotion_check hard-codes .claude-plugin/plugin.json as a required file. If this is meant to apply to this repo’s plugin layout, the canonical path is .github/plugin/plugin.json (see e.g. plugins/awesome-copilot/.github/plugin/plugin.json). Otherwise, please clarify that this required-files list is specific to Claude-style plugins so it’s not misleading.

Suggested change
required = ["README.md", ".claude-plugin/plugin.json"]
missing = [f for f in required if not (root / f).exists()]
# README is always required; plugin manifest can be in either canonical or legacy location
required = ["README.md"]
missing = [f for f in required if not (root / f).exists()]
# Require at least one plugin manifest in the repo's canonical layout or the legacy Claude layout
plugin_manifest_paths = [
root / ".github/plugin/plugin.json",
root / ".claude-plugin/plugin.json",
]
if not any(p.exists() for p in plugin_manifest_paths):
missing.append(".github/plugin/plugin.json (or .claude-plugin/plugin.json)")

Copilot uses AI. Check for mistakes.
Comment on lines +284 to +288
for c in iter(lambda: f.read(8192), b''):
h.update(c)
return h.hexdigest()

manifest = json.loads(Path('INTEGRITY.json').read_text())
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI example reads INTEGRITY.json from the current working directory. Unless the workflow cds into the plugin directory first, this will fail for manifests stored under a plugin subfolder (as shown earlier). Consider updating the snippet to cd into the plugin dir or to reference the manifest path explicitly.

Copilot uses AI. Check for mistakes.
Comment on lines +23 to +27
Codebase → Scan for each ASI control:
ASI-01: Prompt Injection Protection
ASI-02: Tool Use Governance
ASI-03: Agency Boundaries
ASI-04: Escalation Controls
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overview implies there are concrete checks for each ASI-01..ASI-10 control, but the document only defines detailed check sections for a subset (01, 02, 05, 07, 09). Either add guidance/check sections for the remaining risks or make it explicit that only partial checks are included so expectations match.

Copilot uses AI. Check for mistakes.
Comment on lines +76 to +80
"risk": "ASI-01",
"name": "Prompt Injection",
"status": "pass" if positive_found and not negative_found else "fail",
"controls_found": positive_matches,
"vulnerabilities": negative_matches,
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this code snippet, positive_found, negative_found, positive_matches, and negative_matches are undefined, so the example as written isn’t runnable. If this is meant as pseudocode, call that out explicitly; otherwise, include a minimal implementation sketch for computing these values (e.g., via grep/ripgrep or AST scanning).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

@aaronpowell aaronpowell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you've incorrectly branched from the main branch not staged, and as a result all the materialised plugins are included in this PR.

You can attempt to fix this with a rebase:

git fetch origin staged
git rebase --onto origin/staged origin/main <branch name>
git push --force-with-lease

If that does not resolve it, you can run npm run plugin:clean which will delete the materialised plugins and you can commit that change.

@github-actions github-actions bot requested a review from dvelton as a code owner April 1, 2026 22:52
@aaronpowell aaronpowell changed the base branch from main to staged April 1, 2026 23:00
@imran-siddique imran-siddique force-pushed the skills/agt-security-governance branch from 37d2a45 to eb1b932 Compare April 4, 2026 16:45
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

🔍 Skill Validator Results

4 resource(s) checked | ✅ All checks passed

Full output
Found 3 skill(s)
[agent-owasp-compliance] 📊 agent-owasp-compliance: 2,754 BPE tokens [chars/4: 2,970] (standard ~), 22 sections, 7 code blocks
[agent-owasp-compliance]    ⚠  Skill is 2,754 BPE tokens (chars/4 estimate: 2,970) — approaching "comprehensive" range where gains diminish.
[agent-supply-chain] 📊 agent-supply-chain: 2,515 BPE tokens [chars/4: 2,677] (standard ~), 13 sections, 8 code blocks
[agent-supply-chain]    ⚠  Skill is 2,515 BPE tokens (chars/4 estimate: 2,677) — approaching "comprehensive" range where gains diminish.
[agent-supply-chain]    ⚠  No numbered workflow steps — agents follow sequenced procedures more reliably.
[mcp-security-audit] 📊 mcp-security-audit: 2,264 BPE tokens [chars/4: 2,211] (detailed ✓), 9 sections, 11 code blocks
[mcp-security-audit]    ⚠  No numbered workflow steps — agents follow sequenced procedures more reliably.
�[32m✅ All checks passed (3 skill(s))�[0m

imran-siddique and others added 2 commits April 4, 2026 11:27
…y chain)

- mcp-security-audit: Audit .mcp.json files for hardcoded secrets,
  shell injection, unpinned versions, dangerous command patterns
- agent-owasp-compliance: Check agent systems against OWASP ASI 2026
  Top 10 risks with compliance report generation
- agent-supply-chain: SHA-256 integrity manifests, tamper detection,
  version pinning audit, promotion gates for agent plugins

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. Added 3 new skills to docs/README.skills.md index
2. Added imports (json, re) to shell injection check snippet
3. Updated unpinned deps wording to match code behavior (@latest only)
4. Moved check_secrets() outside per-server loop to avoid duplicates
5. Added imports note to verify_manifest snippet
6. Updated promotion_check to support both .github/plugin and .claude-plugin layouts
7. Updated CI example to cd into plugin directory before verifying
8. Added check sections for all 10 ASI controls (was missing 03, 04, 06, 08, 10)
9. Made ASI-01 code snippet runnable with actual file scanning implementation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@imran-siddique imran-siddique force-pushed the skills/agt-security-governance branch from eb1b932 to d5052e1 Compare April 4, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants