Industry Data Model Alignment (Active + Advisory)

Overview

This worker gives the Gold design an active voice for aligning to a Databricks Industry Vibe Data Model (or any canonical industry reference), instead of only grading alignment after the fact. It works in two touchpoints:

1. Active (Phases 0 & 2): overlay the parsed source schema with industry entities, then steer domain assignment, conformed-dimension identification, grain/measure completeness, and Gold naming toward industry standards.
2. Advisory (Phase 8): the crosswalk this worker writes is re-scored by 07-design-validation Validation 6 into INDUSTRY_ALIGNMENT.md.

REQUIRED BACKGROUND: Read references/industry-data-model-alignment.md for the Silver-vs-Gold nuance, the manifest schema, the crosswalk states, and the scoring rubric. This SKILL.md is the behavioral companion to that reference.

The Non-Negotiable Guardrail: Extract, Don't Generate

This worker is a lens over the source schema — never a source of new tables. It upholds databricks-expert-agent's "Extract, Don't Generate" principle: the industry model informs how you model what the source contains; it must never inject entities the source lacks.

The worker MAY	The worker MUST NOT
Map each parsed source table to an industry entity	Invent a Gold table for an industry entity with no source data
Flag industry-core source tables at risk of exclusion	Add columns not derivable from the source schema
Recommend industry-standard names for Gold objects (still built from source columns)	Rename to industry terms that misrepresent the source column's meaning
Recommend domain grouping, conformed dims, measures to consider	Silently auto-apply any recommendation
Surface a coverage gap for human Waived/Planned disposition	Fabricate the missing entity to close the gap

A surfaced gap is resolved as Waived/Planned via SOURCE_TABLE_MAPPING.csv with a rationale — never by inventing a table. All recommendations are recorded in DESIGN_DECISIONS.md and confirmed, mirroring the orchestrator's "never silently correct a classification" rule.

When to Use / When to Skip

• Use when Phase 1 captured industry_reference_source: vibe_generated | published and context/industry_reference.yaml exists (reduce a Vibe Silver model or a published vertical model to that manifest — see the reference doc, Mode A/B).
• Skip entirely when industry_reference_source: none. Record "industry alignment: N/A" in the intake report and DESIGN_DECISIONS.md. Skipping is never a failure — this worker is optional and vertical-specific.

Phase 0: Coverage Overlay

Run immediately after classify_tables() in orchestrator Phase 0, before any modeling, so coverage gaps are visible up front.

Use scripts/industry_overlay.py:

• build_industry_overlay(classified, reference_path) — annotates each source table with its industry entity (Covered by name/alias, Absorbed when attributes live in another table, or Gap), prints the overlay, and returns the crosswalk seed + provisional coverage %.
• Fold the printed overlay into the Schema Intake Report. Every core/extended Gap must get an explicit decision in Phase 1/2.

The manifest schema and matching rules live in references/industry-data-model-alignment.md.

Phase 2: Active Guidance Rules

Apply the industry reference as an active checklist during dimensional modeling. Each rule recommends and records in DESIGN_DECISIONS.md Section 8 — it never auto-applies.

#	Guidance	Uses reference for	Recorded as
1	Canonical domain assignment	Map source tables to the industry's domain names so Gold domains match industry language	Domain column in the table inventory
2	Conformed-dimension identification	Industry models expose shared dims (customer/product/date); promote matched entities referenced by 2+ facts	Bus-matrix note + 04-conformed-dimensions decision
3	Grain & measure completeness	Compare the entity's core_attributes/expected measures against your fact design; flag standard measures the source can support	Grain/measure table + rationale
4	Terminology steering	When a source table maps to an entity but is named off-standard, recommend the industry name (only if it faithfully represents the source)	Naming decision + old→new mapping
5	Gap surfacing (not filling)	For each core/extended Gap, force a Waived (out of scope) or Planned (later phase) decision	SOURCE_TABLE_MAPPING.csv EXCLUDED/PLANNED row + rationale
6	Sensitivity alignment	Where the entity is PII, ensure the covering Gold table carries the PII tag	table_properties.PII on the YAML (Phase 4)

At the end of Phase 2, after gaps are dispositioned, write the crosswalk with write_crosswalk().

The Crosswalk Artifact (hand-off contract)

gold_layer_design/INDUSTRY_CROSSWALK.csv — the single artifact Validation 6 consumes. Matching is owned here; scoring/gating is owned by 07-design-validation.

Column	Meaning
entity	Industry reference entity name
domain	Industry domain
importance	core | extended | optional
sensitivity	PII | none
state	Covered | Absorbed | Waived | Planned | Gap
gold_tables	Pipe-delimited Gold table(s), or empty
rationale	Required for Waived/Planned/Gap

Hand-off to Validation 6

07-design-validation Validation 6 loads INDUSTRY_CROSSWALK.csv (it does not re-match): it verifies each Covered/Absorbed row's gold_tables still exist in the current YAML, re-checks PII sensitivity against table_properties.PII, scores coverage/terminology/PII per the rubric, emits INDUSTRY_ALIGNMENT.md, and stays advisory (excluded from all_valid). The _norm/IMPORTANCE_WEIGHT constants live in scripts/industry_overlay.py.

Common Mistakes

Mistake	Why it's wrong
Inventing dim_supplier to hit 100%	Violates Extract-Don't-Generate; creates a table with no source data
Auto-renaming source tables to industry terms silently	Recommendations must be recorded and confirmed, not applied invisibly
Making coverage a hard gate	Reference is external, volatile, and a different layer; false failures result
Re-implementing matching in Validation 6	Two matchers drift; the crosswalk is the single source of truth
Forcing normalized industry structure into Gold	Defeats the dimensional model's purpose

Inputs

• From Phase 0: classified source table inventory (classify_tables() output)
• From Phase 1: industry_vertical, industry_reference_source, industry_reference_path
• From customer: context/industry_reference.yaml (normalized manifest; template in assets/templates/)

Outputs

• Printed industry coverage overlay (Phase 0)
• Industry Alignment section appended to DESIGN_DECISIONS.md (Phase 2)
• gold_layer_design/INDUSTRY_CROSSWALK.csv (Phase 2, consumed by Validation 6)
• Terminology/domain/conformed-dim recommendations feeding Phases 2–4

Design Notes to Carry Forward

• Provisional coverage % from the Phase 0 overlay
• Every core/extended gap has a Waived/Planned disposition with rationale
• Conformed-dim promotions and terminology renames recorded in DESIGN_DECISIONS.md Section 8
• INDUSTRY_CROSSWALK.csv written and ready for Validation 6

Next Step

Continue Phase 2 dimensional modeling with the industry-informed decisions, then proceed through Phases 3–7. At Phase 8, 07-design-validation Validation 6 consumes the crosswalk and emits INDUSTRY_ALIGNMENT.md.

Reference Files

• Industry Data Model Alignment — manifest schema, Silver-vs-Gold nuance, crosswalk states, scoring rubric, report template
• Overlay script: scripts/industry_overlay.py — matching, overlay, crosswalk writer
• Manifest template: assets/templates/industry_reference.template.yaml

Related Skills

• design-workers/07-design-validation — Validation 6 (consumes the crosswalk)
• design-workers/04-conformed-dimensions — conformed-dimension patterns (guidance rule 2)
• common/naming-tagging-standards — snake_case, tags (layer/domain/PII), dual-purpose descriptions

References

• Vibe Data Modeling (Databricks blog)
• lakehouse-industry-data-models
• AgentSkills.io Specification