eks-upgrade-check

Assess EKS cluster upgrade readiness by running automated checks across 8 areas (version validation, breaking changes, deprecated APIs, add-on compatibility, node readiness, workload risks, AWS Upgrad

SKILL.md

GitHub

EKS Upgrade Readiness Skill

Overview

This skill assesses a live EKS cluster's readiness for a Kubernetes version upgrade. It connects to the cluster via AWS APIs, runs automated checks across 8 assessment areas, calculates a readiness score (0-100%), and produces a detailed report with prioritized remediation steps and pre-filled AWS CLI commands.

This skill is laser-focused on upgrade safety — answering the question: "Is it safe to upgrade this cluster to the next version?"

All operations are read-only — this skill does not modify your cluster.

What Gets Assessed

# Section Key Checks
01 Version Validation Upgrade path validity, version skew policy, support status & cost
02 Breaking Changes Version-specific API removals, behavioral changes, resource impact
03 Deprecated API Detection Live scan of cluster resources for deprecated/removed APIs
04 Add-on Compatibility Core add-on versions, OSS add-on matrix, Karpenter compatibility
05 Node Readiness Node version skew, AL2→AL2023 migration, AMI compatibility
06 Workload Risks Single replicas, missing PDBs, health probes, resource requests
07 AWS Upgrade Insights Official EKS pre-upgrade checks and recommendations
08 Upgrade Plan Pre-filled CLI commands, step-by-step upgrade sequence

Readiness Score

The skill calculates a weighted readiness score:

Category Max Deduction Rationale
Breaking Changes 25 pts Highest risk — can break apps
Deprecated APIs 20 pts Actionable, fixable pre-upgrade
Node Readiness (skew + subnet IPs) 20 pts Can block upgrade entirely
Unsupported Version 15 pts No security patches, urgent upgrade needed
Add-on Compatibility 15 pts Critical > optional add-ons
Karpenter 10 pts Only if installed
Workload Risks 10 pts Best-practice, not blockers
AWS Upgrade Insights 10 pts Official AWS checks
AL2 Nodes / Behavioral 10 pts Informational

Hard Blocker Override: If any hard blocker is detected (e.g., incompatible Karpenter, critical add-on DEGRADED, the conservative subnet-capacity review guard, cluster not ACTIVE), the score is capped at ≤ 59% (NOT READY) regardless of other findings. See references/report-generation.md for the full list.

Score Interpretation:

Score Level Meaning
90-100 READY Safe to proceed
80-89 GOOD Minor issues, can proceed with caution
70-79 FAIR Several issues need attention first
60-69 RISKY Significant issues, not recommended yet
0-59 NOT READY Critical blockers, must resolve first

Assessment Workflow

Step 0: Pre-flight

Execution model — autonomous with hard stops. This skill runs autonomously and does NOT pause for interactive input. It proceeds only when the target cluster and version are unambiguous. If any gating criterion below is not met, it performs a HARD STOP: it does NOT guess, auto-select, or partially assess an ambiguous cluster/target — it emits a structured stop message and ends. Never assess a cluster or target the user did not unambiguously specify or that cannot be uniquely determined. (Identity and cluster/target gates must pass first. Denied reads on the verified cluster are handled via ## Unassessed, not a hard stop — see Action 3 below.)

HARD STOP output format — whenever a criterion below triggers a hard stop, output exactly this and end the run (produce no readiness score):

## Assessment Halted — <one-line reason>

**Criterion not met:** <which check failed>
**What was found:** <observed state>
**To proceed:** <the specific input needed, e.g., re-run naming the cluster + region + target>

Action 1 — Discover clusters

Use EKS ListClusters to discover available clusters, then apply this decision table.

Region caveat. EKS ListClusters is region-scoped and returns names only, not regions. A zero-cluster result means "none in this region," NOT "none in the account" — confirm the intended region before treating zero clusters as terminal. The "name + region" shown below pairs each returned name with the region actually queried, not a region returned by the API.

Condition Action
User named a cluster in the request Confirm it exists in the list, then proceed. If it does not exist → HARD STOP.
Exactly one cluster found, none named Proceed. State which cluster is being assessed.
More than one cluster found, none named HARD STOP. List all clusters (name + region). Do NOT auto-select by first/newest/any heuristic.
Zero clusters found HARD STOP. Report that no clusters were found in the queried region (state the region), and note that other regions were not checked.

Action 2 — Describe the selected cluster

Use EKS DescribeCluster and record: cluster name, Kubernetes version, platform version, region, status, account ID.

Account ID hygiene: the account ID is sensitive. If the generated report will be shared outside the account, mask or omit the account ID before sharing.

Action 2b — Validate cluster status

If status is NOT ACTIVE → HARD STOP:

  • CREATING/UPDATING/DELETING — cluster is in transition; the EKS API will reject an upgrade.
  • FAILED — cluster must be recovered before an upgrade can be attempted.

Cluster status gates the whole assessment; node group status gates node readiness. If a node group's lifecycle status == UPDATING (mid-rotation), the assessment can still run but node readings may be a transient old/new mix — flag it as potentially unstable and recommend re-running after rotation (see references/node-readiness.md §5.1).

Action 2c — Bind Kubernetes reads to the selected cluster

Before any Kubernetes call (including permission probes), verify the Agent Space connection's cluster identity against the selected DescribeCluster ARN/account/region or endpoint. Record the verified binding and explicitly scope EVERY Kubernetes read to it. A display name alone, or a local kubectl context, does not prove the tool's target. Keep AWS reads bound to the same account, region, and cluster. Recheck identity if the connection changes.

If the connection points elsewhere or its target cannot be verified → HARD STOP, produce no readiness score, and name the required connection setup. Do not auto-select an ambiguous connection or combine unverified Kubernetes data with AWS data. This is an identity gate, distinct from a denied read on a verified cluster.

The assessing IAM principal needs an EKS access entry (or the cluster's existing aws-auth mapping) and Kubernetes read permissions; see README.md for access-entry and RBAC setup. An access entry alone does not prove connection identity. Record the verified connection in report metadata, applying the account-ID masking guidance.

Action 3 — Validate permissions (AWS + Kubernetes)

3a — AWS API preflight. Verify access to: ListNodegroups, ListAddons, DescribeAddonVersions (add-on compatibility — references/addon-compatibility.md marks this a MUST-run read), ListInsights, and EC2 DescribeSubnets (node-readiness subnet-IP hard-blocker input). DescribeCluster / DescribeNodegroup / DescribeAddon / DescribeInsight are exercised implicitly by the assessment steps.

3b — Kubernetes RBAC preflight. The high-weight categories read Kubernetes objects, not just AWS APIs. Confirm cluster read access to: Deployments/DaemonSets/StatefulSets (workloads + deprecated-apis), Validating/MutatingWebhookConfigurations (breaking-changes), HorizontalPodAutoscalers (deprecated-apis), and nodepools.karpenter.sh (node-readiness + add-on compatibility) via a can-i-style list check. If kubectl auth can-i itself errors (not a clean yes/no), treat the read as denied.

If any required AWS permission or Kubernetes read is denied on the verified cluster → this is NOT an identity-gate failure. Instead, proceed as a partial assessment:

  1. Record exactly which IAM action or RBAC verb/resource is denied.
  2. Continue the assessment. Every category whose backing read was denied is reported UNKNOWN / not-scored (NOT a clean 0-deduction pass) and listed in ## Unassessed, per references/report-generation.md.
  3. The headline verdict carries the (partial — N category/categories unassessed) marker and is capped below READY — a partial assessment can NEVER yield an uncaveated READY.

The guarantee this preflight gives extends only to the reads it actually probes.

Action 4 — Determine target version

  • User specified a target version → use it. Validate the upgrade path per references/version-validation.md; if the path is invalid (downgrade, same version, or the target does not exist on EKS) → HARD STOP.
  • No target specified → default to the immediate next minor version (current + 1). This is deterministic (EKS upgrades one minor at a time), so it is a safe default, not a guess. State the assumed target explicitly in the report metadata.
  • User requested a multi-hop upgrade (target > current + 1) → HARD STOP. Show the required one-hop-at-a-time path (e.g., 1.30 → 1.31 → 1.32) and state that this run assesses only a single hop; re-run naming the immediate next version.

Action 5 — Proceed

If all criteria above pass, proceed through Steps 1-8 without further pauses.

Steps 1-8: Run Assessment

For each assessment area, read the corresponding reference file from references/ and execute the checks described.

Assessment Area Reference File
Version / upgrade path references/version-validation.md
Breaking changes / API removals references/breaking-changes.md
Deprecated APIs references/deprecated-apis.md
Add-on compatibility / Karpenter references/addon-compatibility.md
Node readiness / AL2 / AMI references/node-readiness.md
Workload risks / PDB / probes references/workload-risks.md
AWS Insights references/upgrade-insights.md
Generate report references/report-generation.md

Step 9: Calculate Score & Generate Report

Follow the scoring algorithm in references/report-generation.md to calculate the final score and produce the report.

Tool Usage Guidelines

  1. Do NOT hardcode or guess cluster names. Always discover by listing first.
  2. Do NOT retry a failed API call more than once.
  3. Always read the relevant reference file before executing checks for that section.
  4. Use EKS APIs for cluster queries — DescribeCluster, ListNodegroups, DescribeNodegroup, ListAddons, DescribeAddon, DescribeAddonVersions, ListInsights, DescribeInsight.
  5. Use Kubernetes APIs for resource scanning — list Deployments, DaemonSets, StatefulSets, Pods, Nodes, PDBs, etc.
  6. Use EC2 DescribeSubnets for subnet IP capacity checks.

Data Files

  • OSS Add-on Registry: assets/oss_addon_registry.json — identifiers and authoritative upstream URLs for common OSS add-ons. This file does NOT contain compatibility data. Compatibility is always verified live via the registry's compatibility_url and releases_url fields.

Report Output Format

The report is rendered as Markdown inline in your response — the DevOps Agent runtime cannot write files. Use the report-title pattern from references/report-generation.md as the report's title/heading, not as a saved filename. There is only one delivery path: render the full report inline.

Each report includes:

  • Readiness score with breakdown
  • Blockers (hard blockers only — these cap the score at ≤59)
  • Critical Actions (other HIGH-severity findings, not score-capping)
  • Recommended actions
  • Informational findings
  • Evidence tables (add-ons, nodes, workloads)
  • Step-by-step upgrade plan with pre-filled commands
  • AWS reference links