kpi-scorecard-contract
Use this skill whenever writing KPI calculation logic, dashboard metric queries, reporting cadence automation, baseline-freeze logic, or any code that produces institutional or platform-level performa
SKILL.mdprimary_build_source/docs/skills/4/mnt/user-data/outputs/kpi-scorecard-contract/SKILL.md
GitHubThis skill defines the KPI scorecard contract for the Myelin platform. KPIs are the platform's accountability mechanism toward institutions, regulators, and commercial buyers. Every formula here is the authoritative calculation. Do not approximate, abbreviate, or substitute alternate formulas.
KPI Families — Overview
Seven families. Each family has one or more named metrics.
| # | Family | Audience | Cadence |
|---|---|---|---|
| 1 | Learner Completion | Admin, Ministry Viewer | Monthly |
| 2 | Core Curriculum Coverage | Admin, Regulator Auditor | Monthly |
| 3 | Competency Attainment | Admin, Employer | Monthly |
| 4 | Time-to-Competency | Admin, Procurement Reviewer | Monthly |
| 5 | Instructor Capacity | Admin | Monthly |
| 6 | Pilot Conversion Readiness | Admin, Procurement Reviewer | Weekly during pilot |
| 7 | Demo-to-Pilot Conversion | Super Admin | Weekly |
Family 1 — Learner Completion
1.1 Module Completion Rate
completionRate = completedEnrollments / totalEnrollments
completedEnrollments: enrollments where all required modules have status COMPLETED within the cohort windowtotalEnrollments: all active enrollments in the cohort as of the measurement date- Scope: per cohort, per programme, per tenant
- Exclude: enrollments with status WITHDRAWN or SUSPENDED
async function computeModuleCompletionRate(cohortId, programmeId, tenantId, asOfDate) {
const result = await db.query(`
SELECT
COUNT(*) FILTER (WHERE e.status = 'COMPLETED') AS completed,
COUNT(*) FILTER (WHERE e.status NOT IN ('WITHDRAWN','SUSPENDED')) AS total
FROM enrollments e
WHERE e.cohort_id = $1
AND e.programme_id = $2
AND e.tenant_id = $3
AND e.enrolled_at <= $4
`, [cohortId, programmeId, tenantId, asOfDate]);
const { completed, total } = result.rows[0];
return total > 0 ? parseFloat(completed) / parseFloat(total) : null;
}
1.2 On-Time Completion Rate
onTimeRate = completedOnTime / totalEnrollments
completedOnTime: enrollments wherecompleted_at <= cohort.end_date
Family 2 — Core Curriculum Coverage
2.1 Mandatory Module Delivery Rate
deliveryRate = modulesDelivered / mandatoryModulesInProgramme
modulesDelivered: modules with at least one learner completion in the tenantmandatoryModulesInProgramme: modules flaggedrequired: truein the programme definition- This metric flags tenants that have activated the platform but not deployed the full curriculum
2.2 Simulation Scenario Activation Rate
activationRate = activatedScenarios / totalScenariosLicensed
- A scenario is "activated" when at least one learner has completed it in the measurement period
- LOTO must always be 100% activated for v1 — it is release-blocking
Family 3 — Competency Attainment
3.1 Simulation First-Pass Rate
firstPassRate = learnersPassingOnFirstAttempt / learnersAttemptingScenario
learnersPassingOnFirstAttempt: learners who completed all 7 LOTO steps without a session reset- Scope: per scenario, per cohort
3.2 Simulation Mastery Rate
masteryRate = learnersCompletingScenario / learnersAttemptingScenario
- Completion regardless of attempt count
- Target threshold for programme endorsement: >= 0.80
3.3 Average Attempts to Mastery
avgAttempts = SUM(sessionCount per learner) / learnersCompletingScenario
sessionCount: number of distinct simulation sessions per learner before first SESSION_COMPLETE- Lower is better; elevated values surface in the instructor view as cohort-level coaching signals
Family 4 — Time-to-Competency
4.1 Median Time to First Simulation Pass
medianTime = MEDIAN(firstPassTimestamp - enrollmentTimestamp) for all completers
- Measured in calendar days from enrollment to first SESSION_COMPLETE
- Median (not mean) — resistant to outliers from long-inactive enrollments
- Report alongside the 25th and 75th percentile for distribution visibility
4.2 Average Active Learning Hours
avgLearningHours = SUM(activeSessionMinutes per learner) / learnerCount / 60
activeSessionMinutes: sum of minutes where tab visibility was active during a learning session- Excludes time where the tab was hidden (enforced by the Page Visibility API in the simulator)
Family 5 — Instructor Capacity
5.1 Learner-to-Instructor Ratio
ratio = activeLearnersInPeriod / activeInstructorsInPeriod
activeLearnersInPeriod: learners with at least one learning event in the measurement monthactiveInstructorsInPeriod: lecturers who reviewed at least one submission or logged a cohort activity
5.2 Submission Review Turnaround Time
avgTurnaround = AVG(reviewedAt - submittedAt) across all submissions in period
- Measured in hours
- SLA target: <= 72 hours from submission to first review outcome
- Surface in the admin dashboard with a count of submissions exceeding 72 hours
Family 6 — Pilot Conversion Readiness
Used during pilot period only. Replaces the standard monthly cadence with weekly reporting.
6.1 Pilot Learner Activation Rate
activationRate = learnersWithAtLeastOneSimAttempt / totalPilotLearnerEnrollments
- Measures whether enrolled learners are actually using the platform
- Target: >= 0.70 by week 4 of pilot
6.2 Pilot NPS Proxy (Engagement Signal)
Not a formal NPS survey — a session engagement proxy:
engagementScore = (completingSessions + assistModeUsedSessions) / totalSessions
- Sessions where the learner reached at least step 4 count as "engaging"
- Reported as a 0–1 ratio with weekly delta
6.3 Demo-to-Pilot Conversion Rate
conversionRate = pilotsStartedFromDemo / totalDemoSessions
pilotsStartedFromDemo: tenants who completed a demo session and signed a pilot agreement within 30 days- Measured at the Super Admin level only — not visible to individual tenants
Family 7 — Demo-to-Pilot (Super Admin Only)
7.1 Demo Session Completion Rate
demoCompletionRate = demosReachingWaypoint5 / totalDemoSessionsStarted
- The 5-waypoint demo must reach all 5 waypoints to count as "completed"
- Incomplete demos (abandoned at waypoint 1–4) are tracked separately for drop-off analysis
7.2 Time from Demo to Pilot Agreement
avgConversionDays = AVG(pilotAgreementDate - demoSessionDate) for converted tenants
Baseline Freeze Rule
KPI baselines are frozen at cohort start and at the end of the pilot's first month. Retroactive changes to baseline data are prohibited. If the programme definition changes after baseline freeze, the delta must be calculated against the original baseline — not recalculated from scratch.
async function freezeKPIBaseline(cohortId, programmeId, tenantId) {
const exists = await getKPIBaseline(cohortId, programmeId, tenantId);
if (exists) throw new Error('BASELINE_ALREADY_FROZEN');
const baseline = {
cohortId,
programmeId,
tenantId,
frozenAt: new Date().toISOString(),
totalEnrollments: await countEnrollments(cohortId, tenantId),
mandatoryModules: await countMandatoryModules(programmeId, tenantId),
scenariosLicensed: await countLicensedScenarios(tenantId)
};
await db.query(`
INSERT INTO kpi_baselines
(baseline_id, cohort_id, programme_id, tenant_id, frozen_at,
total_enrollments, mandatory_modules, scenarios_licensed)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8)
`, [generateUUID(), cohortId, programmeId, tenantId, baseline.frozenAt,
baseline.totalEnrollments, baseline.mandatoryModules, baseline.scenariosLicensed]);
return baseline;
}
Reporting Cadence Automation
KPI snapshots are computed and stored nightly. They are not computed on demand from live data — dashboards read from the snapshot table.
async function runKPISnapshotJob(measurementDate) {
const tenants = await getActiveTenants();
for (const tenant of tenants) {
const cohorts = await getActiveCohorts(tenant.tenantId);
for (const cohort of cohorts) {
const snapshot = await computeAllKPIs(cohort, tenant, measurementDate);
await insertKPISnapshot({
snapshotId: generateUUID(),
tenantId: tenant.tenantId,
cohortId: cohort.cohortId,
measurementDate,
snapshot: JSON.stringify(snapshot),
computedAt: new Date().toISOString()
});
}
}
await writeAuditEntry('KPI_SNAPSHOT_JOB_COMPLETE', {
metadata: { measurementDate, tenantCount: tenants.length }
});
}
Weekly KPIs (Family 6, 7) run on the nightly job but are only surfaced to the dashboard in weekly aggregates.
Role-Scoped KPI Access
| Role | Families visible |
|---|---|
| Admin | 1, 2, 3, 4, 5, 6 (own tenant only) |
| Ministry Viewer | 1, 2, 3 (read-only, own tenant) |
| Procurement Reviewer | 4, 6 (anonymised cohort view) |
| Regulator Auditor | 2, 3 (read-only, own tenant) |
| Super Admin | All families, all tenants |
What Not To Do
- Do not compute KPIs from live data on dashboard load — always read from snapshot table
- Do not recalculate baselines after the baseline freeze date
- Do not expose Family 7 (Demo-to-Pilot) metrics to tenant-level Admin roles
- Do not use mean for time-based distribution metrics — use median with percentile band
- Do not count WITHDRAWN or SUSPENDED enrollments in completion rate denominators
- Do not include hidden-tab time in active learning hours calculations