Skip to content

research(agent): validate direction-aware model handoffs #559

Description

@ss-o

Objective

Validate whether direction-aware model handoff interfaces improve the existing
generator-verifier workflow before changing organization guidance.

The pilot replays 12 historical tasks across z-shell/.github, z-shell/zi,
and z-shell/zsh-lint: six low-capability-to-high-capability escalations and
six high-capability-to-low-capability downshifts. It compares raw trajectory
transfer, sender-authored compaction, and trajectory removal while preserving
the task working tree.

Reference: The Handoff Tax: Continuing Non-Native Trajectories in LLM Agents

Acceptance criteria

  • Run no more than 84 Codex prompts using GPT-5.6 Luna and Sol with medium
    reasoning and standard speed.
  • Use included Codex allowance only, with no API-key billing, credit purchase,
    or automatic overage.
  • Keep every replay in a temporary isolated repository copy and preserve all
    current workspace worktrees.
  • Record deterministic validation, task outcome, model direction, interface,
    and duplicated work for every condition.
  • Treat the directional result as locally supported only when the preferred
    interface wins at least four of six tasks in each direction without a
    critical regression.
  • Change canonical guidance only if the threshold is met.

Instruction impact review

  1. Classification: scoped guidance for the generator-verifier workflow,
    informed by a bounded runtime evaluation. It is not shared baseline policy
    or deterministic enforcement.
  2. Consumers: Codex, Claude Code, Copilot, Gemini CLI, and humans performing
    generator-verifier work in organization repositories.
  3. Canonical owner:
    .github/instructions/generator-verifier-workflow.instructions.md remains
    the correct owner.
  4. Duplication and conflicts: .github/AGENT_MEMORY.md remains the durable
    GitHub handoff protocol and must not duplicate live trajectory-transfer
    guidance. Context-engineering guidance remains complementary.
  5. Manifest impact: no new surface or route is expected because the public
    and private manifests already route generator-verifier tasks to the
    canonical file. Recheck after drafting the change.
  6. Runtime delivery: the existing manifest-declared scoped instruction is
    available to all supported runtimes without relying on an optional skill or
    hook.
  7. Generated output and limits: the target is not part of the generated
    root AGENTS.md composite. Run public policy validation and the applicable
    private checks to confirm no routing or generation drift.

Execution boundaries

  • No commit, push, pull request, merge, release, or deployment is authorized.
  • No API key or paid credit purchase is authorized.
  • Stop if included Codex usage is exhausted or a replay cannot remain isolated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:metaOrganization-wide policy, templates, or meta-repo work.type:maintenanceNon-feature maintenance, cleanup, or org work.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions