Inspect
Which slice drives the release decision? Which aggregate movement is real but insufficient?
Paired lab · 20 minutes
Write the gate before reading the recommendation.
Version B looks better overall. Your job is to decide whether the evidence supports a release—and make your product assumptions executable.
Fictional teaching data: BugBoard and all metrics below were created for this class. Work in pairs: one product owner and one release engineer; switch roles halfway through.
Candidate dashboard
Loading…paired results
| Scope | n | Success A / B | Visual comparison | Schema A / B | p95 A / B | Cost A / B |
|---|---|---|---|---|---|---|
| Loading the paired results… | ||||||
Phase 1 · Interrogate the evidence
Challenge
Name one measurement you distrust and the assumption that could invalidate it.
Specify
Write thresholds for critical risk, overall quality, operations, monitoring, and rollback.
Your release report
Commit before the reveal
Instructor reveal · open only after your team commits
Sample decision: no-go. B improves overall success by four points but reduces security containment by twenty-five points, misses a 99% schema gate, and exceeds illustrative latency and cost limits.
A defensible next experiment would isolate the security regression—starting with prompt/tool-policy differences and permission boundaries—then rerun the complete gate, including the slices that improved.