ml-stack-experiment
Plan or review an isolated ML experiment when a user asks to test a hypothesis, compare runs, tune one variable, reproduce a result, or lock validation; prevent self-promotion and preserve reproducibl
SKILL.mdskills/ml-stack-experiment/SKILL.md
GitHubIsolated experiments
Require a hypothesis, declared baseline, one main variable by default, immutable validation revision, seeds/folds/slices, resource and environment constraints, budget, stop rules, and artifact destination.
Workflow
- Capture or verify requirements with
ml_stack.capture_requirements; reject a run whose requirements are unlocked. - Define baseline, variable, controls, randomization, folds/slices, metrics, and eligibility before execution.
- Create a run directory and record code/data/model/environment revisions, resource use, seeds, logs, and artifact hashes.
- Compare only runs with compatible validation and provenance; investigate NaN, divergence, stalls, and budget overruns.
- Hand results to
ml-stack-evaluationand independentml-stack-audit; never promote the experiment's own result.
Boundary and outputs
Do not change validation data, tune on held-out results, delete evidence, or claim a remote job ran. If execution is unavailable, emit a bounded run specification and comparison schema. Return hypothesis, baseline, immutable inputs, measured metrics, uncertainty, artifacts, and eligibility decision. Next owner: ml-stack-evaluation for quality gates or ml-stack-audit for independent release review.
See hypothesis and baseline, validation protocol, reproducibility, and result comparison.