[workshop-sim] Workshop Simulation Report — 2026-08-08 (Run #30, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-08-08
• Students simulated: 46 × 1,000 Monte Carlo runs
• Workshop steps available: 29/29 main steps
• Overall success rate: 20.0% (95% Monte Carlo interval: 19.6%–20.3%)
• Highest-dropout step: 07-first-workflow (35.1% conditional dropout among 22,355 at-risk runs; 95% Monte Carlo interval: 34.5%–35.7%)
• Lowest curriculum quality step: 07d-confirm-model-access.md (overall score 5.29/10)
• Learning KPI index: 2.89/10 (active_learning 4.20 · checkpoint_quality 0.00 · scaffolding 5.00)
• Model: None / 2026-07-assumption-model-v2 (parameter hash None)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	15	6.70 / 10.0	±1.75
Part 2 — advanced (lessons 15+)	14	6.10 / 10.0	±0.33
Overall corpus	29	6.41 / 10.0	±1.50

No pages are classified as other.

Critical Findings

1. Step 07 is the primary completion bottleneck (Part 1): After agent-insight adjustments, 07-first-workflow sees 35.1% conditional dropout—nearly double the next step. The top failure mode is workflow-authoring-friction, driven by Copilot auth complexity and the mandatory Codespace terminal transition that ui_preferred: true learners (22 of 46 students, mean success rate 9.6%) struggle to navigate.
2. Concept overload blocks beginners before they reach Step 07: Steps 04-actions-intro and 05-agentic-intro together shed ~13,000–7,700 runs in the Part 1 funnel. Beginner-level students (11 of 46) show a mean success rate of 0.3%—they exit almost entirely before reaching the first hands-on workflow step. 05-agentic-intro's Activity 1 asks learners to open a .lock.yml file that does not yet exist at that point.
3. Learning quality health is constrained by a corpus-wide checkpoint gap: The cohort-mean checkpoint_quality score is 0.00/10 across all 29 steps, depressing the Learning KPI index to 2.89/10. Students who persist through the workshop are practicing without embedded knowledge-check loops. The active_learning mean of 4.20 and scaffolding mean of 5.00 are adequate, but without checkpoints the learning signal is weak.
4. The most critical repair belongs to Part 1 (Step 07 / 07d-confirm-model-access.md, score 5.29): Fixing auth recovery UX and making the Codespace-terminal onboarding explicit for UI-preference learners would address the largest single contributor to overall dropout without requiring structural curriculum changes.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add inline Copilot auth recovery guidance to 07d-confirm-model-access.md — embed a minimal decision tree for the two most common auth errors (missing Copilot seat, missing copilot-requests: write permission) as a collapsible block. (completion impact: ↑ · learning KPI impact: ↔)
2. Fix the Activity 1 forward-reference in 05-agentic-workflows-intro.md — replace "open any .lock.yml in your repo" with a sample inline lock snippet so learners can complete the activity before Step 06. Add a second self-check prompt before the checkpoint. (completion impact: ↑ · learning KPI impact: ↑)
3. Add knowledge-check callouts to 04-github-actions-intro.md and 07d-confirm-model-access.md — the corpus-wide checkpoint_quality score is 0.00; adding even one verifiable self-check to these two high-dropout pages would raise both pages' overall_score and the cohort Learning KPI index. (completion impact: ↑ · learning KPI impact: ↑)

Dropout by step

Step	At-risk runs	Dropouts	Conditional rate	95% CI	Failure mode	Top reason
07-first-workflow	22,355	7,847	35.1%	34.5%–35.7%	Access barrier	Workflow authoring friction (auth/billing config confusion)
05-agentic-intro	38,276	7,684	20.1%	19.7%–20.5%	Learning barrier	Agentic concept gap (new paradigm, insufficient anchoring)
04-actions-intro	44,160	5,884	13.3%	13.0%–13.6%	Learning barrier	Concept overload (five new primitives in one pass)
05c-agentic-practice	30,592	4,019	13.1%	12.8%–13.5%	Learning barrier	Agentic classification gap (can't distinguish agentic from standard)
05b-agentic-security	26,573	2,586	9.7%	9.4%–10.1%	Learning barrier	Agentic security gap (sandbox/permission concepts not internalized)
06-install-gh-aw	23,987	1,632	6.8%	6.5%–7.1%	Access barrier	Extension install friction (CLI/Codespace setup issues)
17-add-mcp-tools	12,466	585	4.7%	4.3%–5.1%	Learning barrier	MCP tooling friction (tool configuration complexity)
02-setup	46,000	1,840	4.0%	3.8%–4.2%	Access barrier	Setup friction (Codespace or repo provisioning issues)
19-research-driven-training-node	11,434	438	3.8%	3.5%–4.2%	Learning barrier	Research node friction (advanced multi-step prompting)
18-share-and-reuse	11,881	447	3.8%	3.4%–4.1%	Learning barrier	Workflow reuse friction (import/export concepts)
15-conditional-logic	13,193	494	3.7%	3.4%–4.1%	Learning barrier	Conditional logic friction (templating syntax complexity)
24-self-hosted-runners	10,127	306	3.0%	2.7%–3.4%	Access barrier	Self-hosted runner friction (enterprise environment setup)
09-agentic-editing	14,020	421	3.0%	2.7%–3.3%	Learning barrier	Workflow editing friction (prompt-driven iteration unfamiliar)
14b-pr-reviewer-workflow	13,599	406	3.0%	2.7%–3.3%	Learning barrier	Event trigger friction (pull_request trigger concepts)
22-error-handling-and-resilience	10,539	257	2.4%	2.2%–2.8%	Learning barrier	Resilience friction (retry/fallback patterns)

Curriculum quality and learning KPIs

Step file	Overall	active_learning	checkpoint_quality	scaffolding	KPI	Lowest dim	Repair priority
07d-confirm-model-access.md	5.29	3.7	0.0	5.0	2.71	checkpoint_quality	🔴 High
04-github-actions-intro.md	5.42	4.6	0.0	5.0	3.04	checkpoint_quality	🔴 High
05-agentic-workflows-intro.md	5.43	2.4	0.0	5.0	2.24	checkpoint_quality	🔴 High
15-conditional-logic.md	5.53	3.8	0.0	5.0	2.75	checkpoint_quality	🟡 Medium
08-run-your-workflow.md	5.67	3.0	0.0	5.0	2.45	checkpoint_quality	🟡 Medium
14b-pr-reviewer-workflow.md	5.67	4.8	0.0	5.0	3.11	checkpoint_quality	🟡 Medium
16-connect-data-source.md	5.71	3.8	0.0	5.0	2.75	checkpoint_quality	🟡 Medium
05b-agentic-workflows-security.md	5.75	2.5	0.0	5.0	2.27	checkpoint_quality	🟡 Medium
17-add-mcp-tools.md	5.75	3.4	0.0	5.0	2.60	checkpoint_quality	🟡 Medium
20-persistent-memory.md	5.81	3.6	0.0	5.0	2.67	checkpoint_quality	🟡 Medium
26-manage-costs-and-budgets.md	5.85	4.0	0.0	5.0	2.82	checkpoint_quality	🟢 Low
14-next-steps.md	5.91	3.3	0.0	5.0	2.56	checkpoint_quality	🟢 Low
21-inline-sub-agents.md	5.99	4.1	0.0	5.0	2.85	checkpoint_quality	🟢 Low
09-agentic-editing.md	6.03	4.8	0.0	5.0	3.11	checkpoint_quality	🟢 Low
08b-interpret-your-run.md	6.07	4.1	0.0	5.0	2.85	checkpoint_quality	🟢 Low
02a-setup-codespace.md	6.09	5.0	0.0	5.0	3.18	checkpoint_quality	🟢 Low
18-share-and-reuse.md	6.15	4.5	0.0	5.0	3.00	checkpoint_quality	🟢 Low
25-audit-and-observability.md	6.23	4.9	0.0	5.0	3.15	checkpoint_quality	🟢 Low
28-orchestrate-workflows.md	6.23	4.9	0.0	5.0	3.15	checkpoint_quality	🟢 Low
22-error-handling-and-resilience.md	6.27	5.1	0.0	5.0	3.22	checkpoint_quality	🟢 Low
19-research-driven-training-node.md	6.37	5.6	0.0	5.0	3.40	checkpoint_quality	🟢 Low
27-evaluate-workflow-quality.md	6.37	5.6	0.0	5.0	3.40	checkpoint_quality	🟢 Low
23-ab-experiments.md	6.53	6.4	0.0	5.0	3.69	checkpoint_quality	🟢 Low
05c-agentic-workflows-practice.md	6.59	6.7	0.0	5.0	3.80	checkpoint_quality	🟢 Low
07-your-first-workflow.md	6.59	6.7	0.0	5.0	3.80	checkpoint_quality	🟡 Medium
24-self-hosted-runners.md	6.63	6.9	0.0	5.0	3.87	checkpoint_quality	🟢 Low
00-welcome.md	10.00	0.0	0.0	5.0	1.36	active_learning	—
01-prerequisites.md	10.00	0.0	0.0	5.0	1.36	active_learning	—
06-install-gh-aw.md	10.00	3.5	0.0	5.0	2.64	checkpoint_quality	—
Cohort mean	6.41	4.20	0.00	5.00	2.89	checkpoint_quality	—

Segment breakdowns

By technical level:

Level	Students	Mean success rate
beginner	11	0.3%
github-basic	19	13.3%
actions-user	11	41.1%
advanced	5	41.9%

By personality:

Personality	Students	Mean success rate
curious	15	17.3%
methodical	12	21.6%
impatient	6	23.5%
confused	6	19.5%
skeptical	7	20.2%

By UI preference:

UI preference	Students	Mean success rate
UI-preferred (true)	22	9.6%
Terminal-preferred (false)	24	29.5%

Terminal-preferred students complete at 3× the rate of UI-preferred students, reflecting the mandatory Codespace terminal path at Step 07.

Notable student journeys (3)

Surprising success — Learner 037 (beginner · methodical · enterprise-dev · cli): With a beginner level and no prior coding background in an enterprise environment, this student achieved a 2.6% success rate — far above the beginner mean of 0.3%. The methodical personality and personal-learning goal predict careful reading of each step; the CLI tool preference eliminates the Codespace-terminal transition penalty. The step 05-agentic-intro concept gap remains the primary blocker (340 failures), but the student occasionally pushes through, a pattern consistent with prior accumulated runs showing 757 historical successes. This suggests the workshop's Activities do provide a viable scaffolding path for highly persistent beginners who use the CLI.

Unexpected dropout — Learner 028 (actions-user · confused · enterprise-dev · team-evaluation · cli): Despite actions-user level (should easily clear Steps 04–05), this student achieves only 16.5% success — well below the actions-user mean of 41.1%. The confused personality combined with team-evaluation goal (pressure to exit if unclear ROI) produces repeated failures at 05-agentic-intro (300 failures) and 06-install-gh-aw (43 failures). The student's enterprise background introduces additional friction from proxy/network assumptions at the extension install step. The confused personality fails to leverage the skip-ahead link on step 04 and gets stuck on the agentic concept explanation before reaching any hands-on activity.

Content-gap case — Learner 010 (actions-user · curious · enterprise-dev · personal-learning · cli): This student knows GitHub Actions well yet fails most often at 05-agentic-intro (228 failures) — the page specifically about what makes agentic workflows different from Actions. The activity_before_prerequisite_state risk tag from the agent insight explains this: Activity 1 asks the learner to open a .lock.yml file, which doesn't exist until after Step 06. A curious actions-user will try to execute this instruction literally, fail, and get disoriented. The step's active_learning score of 2.4/10 reflects this instructional gap — the activities don't match the learner's actual state.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 134.7 AIC · ⌖ 8.22 AIC · ⊞ 10.4K · ◷

• expires on Aug 9, 2026, 3:11 AM UTC