00-uc-resources-foundation

Use when any agent or downstream skill needs Unity Catalog schemas and managed volumes (knowledge_sources, agent_outputs, memory tables, benchmark tables, etc.) provisioned idempotently. Foundation St

SKILL.md

GitHub

Foundation Step 0: UC Resources (Schemas + Volumes)

Cross-cutting. This skill is consumed by every Foundation step and agent track. It exists so that no downstream skill (KA, agent memory, tool exports, benchmark persistence, prompt registry) has to repeat schema or volume provisioning. If you find yourself writing CREATE SCHEMA or CREATE VOLUME inside another skill, delegate to F0 instead.

What This Skill Owns

Resource Purpose Default name
Agent UC schema Holds agent-owned assets — KA source volume, tool output volume, benchmark tables, prompt registry pointers, memory tables ${user_schema_prefix}_agent
Ops UC schema Holds operational assets — MLflow OTeL trace tables (F2), monitoring tables (SDLC 07), eval-run rollups (SDLC 04) ${user_schema_prefix}_ops
knowledge_sources MANAGED volume (in agent schema) Source markdown / PDFs for Knowledge Assistants (F5) and any retrieval-style tool /Volumes/${catalog}/${agent_schema}/knowledge_sources
agent_outputs MANAGED volume (in agent schema) CSV exports, charts, generated artifacts produced by @function_tool calls /Volumes/${catalog}/${agent_schema}/agent_outputs
signoffs MANAGED volume (in ops schema) Stakeholder + engineering sign-off decision.md artifacts (SDLC 04b) — the durable, UC-governed location read by SDLC 06 promotion gates /Volumes/${catalog}/${ops_schema}/signoffs
Custom volumes (in agent or ops schema) Anything declared in state://AgentSpec.required_volumes[] /Volumes/${catalog}/${schema}/${name}

signoffs is generic. F0 pre-creates the volume under a workshop-neutral name (signoffs) so SDLC 04b never has to provision its own UC location and SDLC 06 can resolve /Volumes/${catalog}/${ops_schema}/signoffs/v<N>/decision.md deterministically. The volume name does not carry any use-case-specific prefix (no skyloyalty_signoffs, no ${use_case_slug}_signoffs); the per-version directory v<N>/ carries the disambiguation.

Every operation is idempotent (CREATE SCHEMA IF NOT EXISTS, CREATE VOLUME IF NOT EXISTS, both as DDL — an existing object is a no-op). It is safe to call this skill from any prompt that needs to be sure these resources exist; the cost on a warm workspace is one round-trip per resource.

What This Skill Does NOT Own

  • Catalog creation. A UC catalog must already exist; F0 reads state://Resources.uc.catalog and validates the caller has USE CATALOG + CREATE SCHEMA on it.
  • UC table DDL. Tables for OTeL traces, benchmarks, memory, or monitoring are owned by their respective consuming skills (F2, SDLC 02/04/07).
  • Source content. F0 creates an empty knowledge_sources volume; F5 (or any other consumer) is responsible for populating it with files.
  • Bronze data. Bronze schema + Delta tables are owned by the bronze layer setup skill (data_product_accelerator/skills/bronze/00-bronze-layer-setup).

When to Use

  • Always, near the top of the prompt sequence — before F1 (MLflow env) so that anything downstream can assume schemas + volumes exist.
  • Whenever you add a new consuming skill that needs a new volume — extend state://AgentSpec.required_volumes[] rather than putting CREATE VOLUME inside the consuming skill.
  • After bootstrap when the resolved spec is available, but before any other Foundation step.

Prerequisites

Requirement How to Check
state://Resources.uc.catalog is resolved Set by vibecoding-state bootstrap op
state://Resources.uc.user_schema_prefix is resolved Derived in bootstrap from the user identity + use_case_slug
Workshop SQL warehouse is RUNNING state://Resources.warehouse_id (set by Prompt 1 preflight) — needed for CREATE SCHEMA execution via WorkspaceClient.statement_execution
databricks-sdk >= 0.30 installed python -c "from databricks.sdk import WorkspaceClient; from databricks.sdk.service.sql import StatementState"
Caller has USE CATALOG + CREATE SCHEMA on the catalog databricks unity-catalog catalogs get $CATALOG --output json | jq '.privileges'

Inputs

# Resolved by the caller from state — F0 does not read state directly.
uc_catalog:           "string (required)"        # e.g. "main"
user_schema_prefix:   "string (required)"        # e.g. "jane_d_booking_app"
agent_schema:         "string (required)"        # default: "${user_schema_prefix}_agent"
ops_schema:           "string (required)"        # default: "${user_schema_prefix}_ops"
warehouse_id:         "string (required)"        # SQL warehouse for DDL execution
required_volumes:                                 # optional — defaults below if not provided
  - { name: "knowledge_sources", schema: "agent", comment: "KA + retrieval source files" }
  - { name: "agent_outputs",     schema: "agent", comment: "Tool-generated artifacts (CSV, charts)" }
  - { name: "signoffs",          schema: "ops",   comment: "SDLC 04b stakeholder + engineering signoff decision.md artifacts" }
  # AgentSpec.required_volumes[] is appended verbatim

agent_schema / ops_schema may be overridden to any name that starts with ${user_schema_prefix}_; a non-prefixed override raises ValueError, because RULE_10 sanctions in-session provisioning of the participant's own prefixed schemas only.

Operations

flowchart TD
  start["F0 invoked"] --> grant["Verify USE CATALOG + CREATE SCHEMA"]
  grant --> agentSch["CREATE SCHEMA IF NOT EXISTS<br/>${catalog}.${agent_schema}"]
  grant --> opsSch["CREATE SCHEMA IF NOT EXISTS<br/>${catalog}.${ops_schema}"]
  agentSch --> volLoop["For each required_volume:<br/>create MANAGED volume idempotently"]
  opsSch --> volLoop
  volLoop --> capture["Capture { agent_schema, ops_schema, uc_volumes{name → path} }"]

Reference implementation

import time

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.sql import StatementState

def provision_uc_resources(
    *,
    uc_catalog: str,
    user_schema_prefix: str,
    agent_schema: str,
    ops_schema: str,
    warehouse_id: str,
    required_volumes: list[dict] | None = None,
    ddl_timeout_s: int = 300,
) -> dict:
    w = WorkspaceClient()

    # F0 provisions only the participant's own prefixed schemas (RULE_10 foundation carve-out).
    suffixes = {}
    for role, schema in (("agent", agent_schema), ("ops", ops_schema)):
        if not schema.startswith(f"{user_schema_prefix}_") or schema == f"{user_schema_prefix}_":
            raise ValueError(
                f"{role}_schema {schema!r} must start with '{user_schema_prefix}_': RULE_10 "
                "sanctions in-session provisioning of the participant's own prefixed schemas only"
            )
        suffixes[role] = schema.removeprefix(f"{user_schema_prefix}_")

    def run_ddl(statement: str) -> None:
        """Run one DDL statement to a terminal state; raise unless it SUCCEEDED (fail closed)."""
        resp = w.statement_execution.execute_statement(
            warehouse_id=warehouse_id, statement=statement, wait_timeout="30s",
        )
        deadline = time.monotonic() + ddl_timeout_s
        while resp.status.state in (StatementState.PENDING, StatementState.RUNNING):
            if time.monotonic() > deadline:
                w.statement_execution.cancel_execution(resp.statement_id)
                raise TimeoutError(f"DDL still {resp.status.state.value} after {ddl_timeout_s}s: {statement}")
            time.sleep(2)
            resp = w.statement_execution.get_statement(resp.statement_id)
        if resp.status.state != StatementState.SUCCEEDED:  # FAILED / CANCELED / CLOSED
            error = resp.status.error.message if resp.status.error else "no error message"
            raise RuntimeError(f"DDL {resp.status.state.value}: {error} ({statement})")

    # 1. Schemas (DDL via statement execution against the workshop warehouse)
    for suffix in suffixes.values():
        run_ddl(
            f"CREATE SCHEMA IF NOT EXISTS {uc_catalog}.{user_schema_prefix}_{suffix} "
            f"COMMENT 'F0-managed: agent or ops assets'"
        )

    # 2. MANAGED volumes (DDL IF NOT EXISTS through the same warehouse)
    defaults = [
        {"name": "knowledge_sources", "schema": "agent",
         "comment": "KA + retrieval source files"},
        {"name": "agent_outputs", "schema": "agent",
         "comment": "Tool-generated artifacts (CSV, charts)"},
        {"name": "signoffs", "schema": "ops",
         "comment": "SDLC 04b stakeholder + engineering signoff decision.md artifacts"},
    ]
    volumes_spec = (required_volumes or []) + defaults
    seen, paths = set(), {}
    for v in volumes_spec:
        suffix = suffixes["agent" if v["schema"] == "agent" else "ops"]
        target_schema, name = f"{user_schema_prefix}_{suffix}", v["name"]
        key = (target_schema, name)
        if key in seen:
            continue
        seen.add(key)
        comment = v.get("comment", "F0-managed volume").replace("'", "\\'")
        run_ddl(
            f"CREATE VOLUME IF NOT EXISTS {uc_catalog}.{user_schema_prefix}_{suffix}.{name} "
            f"COMMENT '{comment}'"
        )  # raises unless SUCCEEDED, so only confirmed volumes are captured
        paths[name] = f"/Volumes/{uc_catalog}/{target_schema}/{name}"

    return {
        "agent_schema": agent_schema,
        "ops_schema": ops_schema,
        "uc_volumes": paths,
    }

The skill caller passes the result back to vibecoding-state exit so downstream skills can read state://Resources.uc_volumes.knowledge_sources etc.


Outputs (handoff)

Key Consumer Value
agent_schema F2, F5, SDLC 02/04, Track A 05 (memory) ${user_schema_prefix}_agent
ops_schema F2 (OTeL tables), SDLC 07 (monitoring), SDLC 04b (signoffs) ${user_schema_prefix}_ops
uc_volumes.knowledge_sources F5 (Step 5_0 — branch B/C upload), F3 (Vector Search MCP source dir) /Volumes/${catalog}/${agent_schema}/knowledge_sources
uc_volumes.agent_outputs Track A 03 csv_export @function_tool, any code-interpreter output /Volumes/${catalog}/${agent_schema}/agent_outputs
uc_volumes.signoffs (alias signoff_volume_path) SDLC 04b (writes decision.md), SDLC 06 (reads it as the promotion gate) /Volumes/${catalog}/${ops_schema}/signoffs
uc_volumes.<custom> Whichever consuming skill declared the volume /Volumes/${catalog}/${schema}/${name}

Validation Gate

  • ${uc_catalog}.${agent_schema} exists (SHOW SCHEMAS IN ${uc_catalog} LIKE '${agent_schema}')
  • ${uc_catalog}.${ops_schema} exists
  • Every volume in required_volumes is listable via databricks volumes list ${uc_catalog} ${schema} --output json | jq '.volumes[] | select(.name=="<name>")'
  • The generic signoffs volume in ${ops_schema} is present (consumed by SDLC 04b / SDLC 06 — name is workshop-neutral)
  • agent_schema, ops_schema, the full uc_volumes map, and signoff_volume_path are captured in state

Delegation Map

Need Delegate To
Bronze schema + Delta tables bronze/00-bronze-layer-setup
OTeL trace tables in ${ops_schema} F2
KA source markdown into knowledge_sources F5 Step 5_0 — branch (A) skip if non-empty / (B) upload ka_source / (C) auto-generate from PRD
Lakebase memory tables Track A 05
Benchmark tables in ${agent_schema} SDLC 02
Monitoring tables in ${ops_schema} SDLC 07

Limitations to Plan For

  • Catalog must exist. F0 does not create catalogs (org-level admin action).
  • Workspace warehouse required. Schema DDL goes through the SQL warehouse, not the Python serverless API.
  • No drop semantics. F0 is create-only — to remove a schema or volume, delete out of band; do not put DROP in any agent skill.
  • Volume size budgeting. Managed volumes inherit the catalog's storage location; if your agent_outputs may grow large, document a retention policy in the consuming skill.

Notes to Carry Forward

Key Produced By Value
agent_schema F0 ${user_schema_prefix}_agent
ops_schema F0 ${user_schema_prefix}_ops
uc_volumes F0 Map of name → /Volumes/${catalog}/${schema}/${name}
knowledge_source_path F0 (alias of uc_volumes.knowledge_sources) Path that F5 attaches as a files knowledge source
agent_outputs_path F0 (alias of uc_volumes.agent_outputs) Path that Track A 03's csv_export writes to
signoff_volume_path F0 (alias of uc_volumes.signoffs) Generic, use-case-neutral signoff volume path SDLC 04b writes decision.md into and SDLC 06 reads as the promotion gate. Always /Volumes/${catalog}/${ops_schema}/signoffs — never workshop-specific.

Next Step

After F0's gate passes, continue with F1: MLflow GenAI Foundation. Every subsequent skill can now assume schemas + volumes exist.

References


Version History

Version Date Changes
1.1.0 2026-04-26 Added generic, workshop-neutral signoffs MANAGED volume in ${ops_schema} as a default required_volume. SDLC 04b writes decision.md artifacts here and SDLC 06 reads them as the promotion gate. Captures signoff_volume_path (alias of uc_volumes.signoffs) so downstream skills resolve /Volumes/${catalog}/${ops_schema}/signoffs/v<N>/decision.md deterministically without per-use-case naming.
1.0.0 2026-04-24 Initial skill — extracted from F5 (05-knowledge-assistant) Step 5_0 so schema + volume provisioning is owned in one cross-cutting place. Consumers (F5, F2, Track A 05, SDLC 02/04/07) now delegate.