NiFi E2E Deployment Skill

Manages the Docker-based NiFi + Keycloak test environment for E2E testing.

Parameters

Optional argument selects the workflow:

/deploy or /deploy status — Show container status
/deploy start — First-time start (build NAR + start containers)
/deploy redeploy — Redeploy after code changes (rebuild NAR + restart NiFi only)
/deploy test — Run E2E Playwright tests (containers must be running)
/deploy test <file> — Run specific E2E test file
/deploy integration-test (alias /deploy it) — Run the integration-testing module's Java Failsafe ITs (Maven manages container lifecycle)
/deploy stop — Stop all containers
/deploy full — Full CI workflow: stop → build → start → test → stop
/deploy troubleshoot — Diagnose common issues

Workflow

Step 1: Determine Action

Parse the argument to determine which workflow to execute. Default (no args) = show status. integration-test and it both route to the integration-test [it] subsection.

Step 2: Execute Action

status (default)

./integration-testing/src/main/docker/check-status.sh --quiet

start

./integration-testing/src/main/docker/run-and-deploy.sh

Builds NAR, copies to target/nifi-deploy/, starts Keycloak + NiFi. NiFi takes ~60-80s to become healthy. URLs after start:

NiFi: https://localhost:9095/nifi (testUser / drowssap)
Keycloak: https://localhost:9085 (admin / admin)

After containers are healthy, run the NiFi Browser Login procedure, then display the Sandbox Credentials.

redeploy

./integration-testing/src/main/docker/redeploy-nifi.sh

Rebuilds NAR, copies to deploy dir, runs docker compose restart nifi. Wait ~60-80s for NiFi to become healthy. Verify: ./integration-testing/src/main/docker/check-status.sh

After NiFi is healthy, run the NiFi Browser Login procedure, then display the Sandbox Credentials.

test [file]

cd e-2-e-playwright && npm run playwright:test [-- tests/<file>.spec.js]

Containers MUST be running. Use /deploy start first if needed.

integration-test [it]

python3 .plan/execute-script.py plan-marshall:build-maven:maven run --command-args "verify -Pintegration-tests -pl integration-testing -am"

Runs the integration-testing module's Java Failsafe ITs (e.g. RestApiGatewayIT) via the canonical plan-marshall Maven executor. Use a 10-minute (600000 ms) Bash timeout and analyze the build's TOON result (status, errors[], log_file) rather than scanning raw stdout.

Keep the invocation -pl integration-testing -am. The -am (also-make) adds only the root POM to the reactor; the two NAR files come from the nested build in copy-deployment.sh, which deploy-and-start.sh runs whenever one of them is missing.
The integration-tests profile manages its own container lifecycle: deploy-and-start.sh auto-stops the same-project docker stack (docker compose down -v) before building and starting fresh NiFi+Keycloak containers, and cleanup-containers tears them down on completion. No explicit pre-flight stop is needed here — the profile already reuses the existing Docker primitives.

stop

cd integration-testing/src/main/docker && docker compose down -v

full

cd integration-testing/src/main/docker && docker compose down -v
cd <project-root>
python3 .plan/execute-script.py plan-marshall:build-maven:maven run --command-args "clean install"
python3 .plan/execute-script.py plan-marshall:build-maven:maven run --command-args "verify -Pintegration-tests -pl e-2-e-playwright -am"

Run the two Maven steps via the canonical plan-marshall executor (10-minute / 600000 ms Bash timeout; analyze the TOON result rather than raw stdout). IMPORTANT: Maven manages its own container lifecycle. Do NOT have manually-started containers running.

troubleshoot

Check for common issues in order:

docker ps — Are containers running? Any orphaned containers on ports 7777/9095/9443?
docker compose -f integration-testing/src/main/docker/docker-compose.yml ps — Container states
Port conflicts: lsof -i :7777 -i :9095 -i :9443 2>/dev/null
NiFi logs: docker logs docker-nifi-1 --tail 50
NAR loaded: docker logs docker-nifi-1 2>&1 | grep -i cuioss

Always confirm NiFi actually STARTED — do not trust the deploy script's exit alone. The deploy script (or a backgrounded deploy job) can report "started" / exit while NiFi crashed during Jetty WAR load. After any start/redeploy, verify:

Health: docker inspect -f '{{.State.Health.Status}}' docker-nifi-1 must be healthy (starts as starting; can take ~60-80s).
Success marker: docker logs docker-nifi-1 2>&1 | grep "Started Server on" present.
No crash: docker logs docker-nifi-1 2>&1 | grep -E "Failed to start Server|Encountered duplicate UI" empty.
"Keycloak is ready ✅" in the deploy output is NOT "NiFi is ready" — keep waiting for NiFi health.

Encountered duplicate UI for <processor> crash at startup = stale-version NARs. After a project version bump (e.g. 0.5.0 → 0.6.0) old *-<oldversion>-SNAPSHOT.nar files linger in target/nifi-deploy/ AND in the module target/ dirs; docker compose down -v does NOT clean them (it only wipes volumes). NiFi then loads BOTH versions and dies with java.lang.IllegalStateException: Encountered duplicate UI .... Fix:

find . -name "*-<oldversion>-SNAPSHOT.nar" -not -path "*/node_modules/*"   # locate stale NARs

Delete every match (in target/nifi-deploy/ and the module */target/ dirs), then restart NiFi (docker compose ... restart nifi or a fresh up).

NiFi Browser Login

After every start or redeploy, open Chrome and log in to NiFi.

Prerequisites: The user must have accepted the self-signed certificate for https://localhost:9095 in Chrome beforehand. If the browser shows a certificate error page, ask the user to manually accept the certificate (Advanced → Proceed) and then retry.

CRITICAL — Chrome extension false negatives: The tabs_context_mcp call frequently returns "Browser extension is not connected" even when the extension IS connected and working. This is a transient MCP initialization issue, NOT an actual disconnection. You MUST retry up to 5 times before concluding the extension is truly disconnected. NEVER tell the user the extension is disconnected based on failed calls alone — just keep retrying silently. In practice, the extension is almost always connected; failures resolve after 1-5 retries.

Login steps:

Call tabs_context_mcp with createIfEmpty: true. If it returns "not connected", retry up to 5 times (no delay needed between retries). Only after 5 consecutive failures should you ask the user to check the extension.
Create a new tab with tabs_create_mcp
Navigate to https://localhost:9095/nifi/ — this redirects to NiFi's own login page (#/login)
Take a screenshot to see the current page state
Click the Username input field and type testUser
Click the Password input field and type drowssap
Click the "Log in" button
Wait until the URL contains #/process-groups/ and the page title is "NiFi Flow" to verify a successful login
Report login success to the user

Notes:

This is NiFi's built-in login form — not Keycloak. The login page URL is https://localhost:9095/nifi/#/login.
If the URL already contains #/process-groups/ after navigation, the user is already logged in — skip the login steps.
If NiFi is still starting and the page doesn't load, wait a few seconds and retry navigation.
Use computer tool with coordinate-based clicks for the login form — find and read_page can be unreliable inside NiFi's Angular UI.

Critical Rules

Maven E2E (python3 .plan/execute-script.py plan-marshall:build-maven:maven run --command-args "verify -Pintegration-tests") automatically stops existing containers before starting fresh ones (via fixed deploy-and-start.sh)
The integration-tests profile's deploy-and-start.sh already stops the same-project docker dev stack before starting, so /deploy integration-test (it) needs no competing explicit stop step
After a clean install build (python3 .plan/execute-script.py plan-marshall:build-maven:maven run --command-args "clean install"), the NAR in target/nifi-deploy/ is stale — must run /deploy redeploy to update running containers
NiFi loads NARs only at startup — code changes require container restart, not just file copy
The target/nifi-deploy/ directory is volume-mounted into the NiFi container
Deploy the checkout you intend. run-and-deploy.sh / copy-deployment.sh self-locate via their own path (dirname $0) and docker-compose.yml mounts target/nifi-deploy relative to the compose file's directory. So the checkout whose script you run is the code that deploys. When a plan-marshall worktree is active, running the main checkout's script deploys main's code (without the feature branch's changes) — run the worktree's copy (.plan/local/worktrees/<plan>/integration-testing/src/main/docker/run-and-deploy.sh) to deploy the feature-branch code you're testing. The two docker-compose projects share the container names docker-nifi-1 / docker-keycloak-1, so down the current stack before up-ing the other.
down -v resets flow state. Removing volumes reloads the baked flow.json.gz fresh: no runtime-added issuers/routes and no gateway request counters. E2E tests that assume seeded issuers or existing metric data behave differently on a freshly-reset stack than on a long-running one — reset before a definitive E2E run so local state matches CI's fresh stack.

Key Paths

Path	Purpose
`target/nifi-deploy/`	NAR deployment dir (Docker volume mount)
`integration-testing/src/main/docker/docker-compose.yml`	Docker Compose config
`integration-testing/src/main/docker/*.sh`	Local deployment scripts
`e-2-e-playwright/scripts/*.sh`	Maven-phase scripts
`e-2-e-playwright/target/test-results/`	Test artifacts

Sandbox Credentials

After a successful start or redeploy (and browser login), display these credentials to the user so they can use the Token Fetch form in the Endpunkt-Tester tab:

Sandbox Keycloak Credentials (for Token Fetch)
───────────────────────────────────────────────
Realm:         oauth_integration_tests
Client-ID:     test_client
Client-Secret: yTKslWLtf4giJcWCaoVJ20H8sy6STexM

ROPC (password grant):
  Username:    testUser
  Password:    drowssap

Token Endpoint (auto-discovered via issuer):
  https://keycloak:8443/realms/oauth_integration_tests/protocol/openid-connect/token

Other realm (for cross-issuer testing):
  Realm:         other_realm
  Client-ID:     other_client
  Client-Secret: otherClientSecretValue123456789
  Username:      otherUser / drowssap

Note: The issuer dropdown in the Endpunkt-Tester auto-populates from the processor's Controller Service config. The token endpoint is resolved via OIDC discovery when clicking "Ermitteln".

NiFi Custom UI Browser Verification

The Custom UI runs inside an iframe. To access it manually:

Navigate to https://localhost:9095/nifi/
Double-click process group to enter it
Right-click processor → look for "Advanced" menu item
If not visible: click "Configure" → click Custom UI icon (top-right of dialog, red sticky-note icon)
Custom UI loads in iframe with ?id={processorId} query parameter

deploy

SKILL.md.claude/skills/deploy/SKILL.md