Skip to content

feat(tier-check): support configurable label taxonomy and aliases for Tier 1 repo checks #482

Description

@anubhav756

Is your feature request related to a problem? Please describe.

The MCP SDK Tiering specification (SEP-1730 / docs/community/sdk-tiers.mdx) requires specific labels for issue triage (bug, enhancement, question), status (needs confirmation, needs repro, ready for work, good first issue, help wanted), and priority (P0P3).

Currently, tier-check enforces these through exact hardcoded string matches:

  • Matches lowercase names strictly against TYPE_LABELS, STATUS_LABELS, and PRIORITY_LABELS.
  • Hardcodes labels: 'P0' when querying GitHub's API to calculate critical issue resolution times.

Many organizations and SDK ecosystems (e.g. Google/DevRel, cloud providers, and established OSS projects) use namespaced labels like priority: p0, type: bug, and status: waiting for response to support organization-wide issue taxonomies, automation bots, and issue-mirroring bridges (e.g., Jira, Google Issue Tracker).

Even when an SDK repository fully adheres to the required semantic taxonomy and respects the triage/P0 SLAs, tier-check fails due to exact string mismatch. Forcing these repos to adopt bare strings creates unnecessary migration friction for maintainers seeking Tier 1 / Tier 2 certification.

Describe the solution you'd like.

Allow tier-check to recognize configurable label aliases or mappings so repositories can pass tier checks while preserving their established label conventions.

  1. Configuration Mechanism:
    Support an optional label alias mapping, configured either via:

    • A repository-level config file (e.g., .github/mcp-conformance.json or .mcp/tier-check.json).
    • The built-in SDK config in src/sdk-runner/known-sdks.ts (or via a CLI flag like --labels-config).

    Example configuration:

    {
      "labels": {
        "aliases": {
          "bug": ["type: bug"],
          "enhancement": ["type: feature request"],
          "question": ["type: question"],
          "needs confirmation": ["status: feedback wanted", "status: investigating"],
          "needs repro": ["status: waiting for response", "needs more info"],
          "ready for work": ["ready for work", "needs work"],
          "help wanted": ["status: help wanted", "help wanted"],
          "P0": ["priority: p0"],
          "P1": ["priority: p1"],
          "P2": ["priority: p2"],
          "P3": ["priority: p3"]
        }
      }
    }
  2. Evaluation in checkLabels:
    When evaluating whether a required label is present, check if the repo contains the standard label name or any of its configured aliases.

  3. Evaluation in checkP0Resolution:
    Update checkP0Resolution to query the configured P0 label (e.g., priority: p0 instead of the hardcoded 'P0'), ensuring that P0 resolution metrics reflect the repository's actual critical issues.

  4. Default Behavior:
    If no mapping is provided, retain the exact current behavior with zero breaking changes.

Describe alternatives you've considered

  1. Renaming labels in each SDK repository:
    Breaks existing workflows, bot automation, and internal issue-sync tooling across repos that rely on structured label prefixes (priority:*, type:*, status:*).

  2. Duplicate labels (e.g., both P0 and priority: p0):
    Creates contributor confusion, label clutter, and requires maintainers to double-tag every issue to satisfy both external checks and internal automations.

  3. Relying solely on GitHub Issue Types:
    checkLabels already supports GitHub Issue Types for bug/enhancement/question, but GitHub Issue Types do not cover status or priority (P0P3), leaving the main gap unaddressed.

  4. Simple prefix-stripping:
    While some differences are purely prefix/case conventions (e.g. priority: p0 vs P0, type: bug vs bug), many existing repos use established semantic synonyms for the same triage stages:

    • enhancementtype: feature request
    • needs confirmationstatus: feedback wanted or status: investigating
    • needs reprostatus: waiting for response or needs more info
    • ready for workneeds work

    Supporting a 1-to-many alias dictionary ensures existing repositories with mature workflows don't have to break internal automations or rename established labels.

Additional Context

  • We are seeing this while preparing the MCP Toolbox repositories for Tier 1 conformance. All required triage and priority categories are tracked, but naming conventions follow standard namespaced prefixes (e.g., priority: p0, status: feedback wanted).
  • We are happy to submit a PR to implement this alias support in src/tier-check/ if this approach aligns with the team.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions