craft-beat-cut
Cut footage on the beat with ArtCraft FilmCraft - analyse a track, cut it to length on its downbeat grid, pick camera-punch hits, place edits on downbeats and keyframe punch-ins on the hits, deliver a
SKILL.mdpackages/craft-apps/.pi/skills/craft-beat-cut/SKILL.md
GitHubcraft-beat-cut — music grid → footage cut on downbeats → punches on hits → mp4
Footage-timeline counterpart of hyperframes-showreel's beat-synced composition: here the picture is edited in
FilmCraft (filmcraft skill — read it first); for a composition-driven showreel use hyperframes-showreel
(see the split in craft-studio). The music side comes from the music-production skills; resumable-job rules are
in the craft-studio router, section Craft jobs.
Step table
| # | Step | Tool | Input → Output | Gate |
|---|---|---|---|---|
| 0 | Preflight + job | agent | request → job.json |
check: job |
| 1 | Analyse | music-analysis |
track → 01-music/track_map.json |
check: map |
| 2 | Cut music | music-edit-to-length |
map → 02-music/edit.wav, edit_edit.json |
pi-craft verify + check: edit |
| 3 | Pick hits | beat-sync-video |
edit → 03-hits/hits.json (music-hits/1) |
check: hits |
| 4 | Conform audio | SoundCraft | edit.wav → 04-audio/music.wav (PCM 24-bit 48 kHz) |
pi-craft verify --codec audio=pcm_s24le |
| 5 | Cut picture | FilmCraft | footage + grid + hits → 05-cut/p.fcproj, rough.mp4 |
pi-craft verify duration |
| 6 | Punch + deliver | FilmCraft | project + music → 06-delivery/final.mp4 |
pi-craft verify h264 + aac + --frame-diff |
Hand-off formats: wav PCM 24-bit 48 kHz, music-edit/1 and music-hits/1 json, mp4 h264 + aac. Write tolerances as
${D}±0.1 — in bash $D±0.1 swallows the ± into the variable name.
Step 0 — Preflight and job
Output: craft-jobs/<slug>/job.json (created or loaded).
Gate: check: job.
All three music skills are essential — never guess beat times.
| Sibling | Package | Check | If missing |
|---|---|---|---|
music-analysis |
music-production |
skill in the loaded skill list | stop: name music-production (pi install npm:@blackbelt-technology/pi-dashboard-music-production) |
music-edit-to-length |
music-production |
skill in the loaded skill list | stop, same package |
beat-sync-video |
music-production |
skill in the loaded skill list | stop, same package — do not invent hit times |
Runtime probe: the scripts need the quick-tier venv of music-analysis step 0 —
.venv-music/bin/python -c "import librosa" must succeed; a failed probe counts as absent (stop and point to
that skill's step 0). "Loaded skill list" is the available-skills list in your own system prompt; its entry location
also gives the package directory (PKG). Needs the user's local music file (no ripping from streaming services).
Then set up the job (Craft jobs in craft-studio: slug rule, job-helper) and run Resume.
# run from the directory that contains craft-jobs/ (SLUG set as in step 0)
jq -e '.name and .workflow and (.steps | type == "object")' "craft-jobs/$SLUG/job.json"
Step 1 — Analyse
Output: 01-music/track_map.json (music-map/1).
Gate: check: map.
Run music-analysis on the local track; read the sections and drop candidates and confirm the drop by ear.
# run inside the step directory: cd "$JOB/01-music" (the file is named as below)
jq -e '.schema=="music-map/1"' track_map.json
Step 2 — Cut the music
Output: 02-music/edit.wav and 02-music/edit_edit.json (music-edit/1).
Gate: pi-craft verify on the wav (duration = target) and check: edit.
music-edit-to-length renders the cut and a preview. Listen before lock: ask the user to approve the preview
(ask_user) before any picture is cut to its grid; keep rejected versions with a _vN suffix.
pi-craft verify "$JOB/02-music/edit.wav" --duration "${D}±0.1" --has-audio
# run inside the step directory: cd "$JOB/02-music" (the file is named as below)
jq -e '.schema=="music-edit/1" and .duration > 0 and (.downbeats_video | length) > 0' edit_edit.json
Step 3 — Pick the hits
Output: 03-hits/hits.json (music-hits/1).
Gate: check: hits — right schema and at least one hit.
Run beat-sync-video (pick_hits.py) on edit_edit.json. Hits are video time; add hand-placed hits with
"source":"manual". An empty or wrong-schema file stops the workflow — do not cut to guessed times.
# run inside the step directory: cd "$JOB/03-hits" (the file is named as below)
jq -e '.schema=="music-hits/1" and (.hits|length>0)' hits.json
Step 4 — Conform the audio
Output: 04-audio/music.wav (PCM 24-bit, 48 kHz).
Gate: pi-craft verify --codec audio=pcm_s24le, duration unchanged.
pi-craft soundcraft exec convert "$JOB/02-music/edit.wav" "$JOB/04-audio/music.wav" --bit-depth 24 --rate 48000
pi-craft verify "$JOB/04-audio/music.wav" --duration "${D}±0.05" --codec audio=pcm_s24le
Step 5 — Cut the picture
Output: 05-cut/p.fcproj, 05-cut/rough.mp4 (the retained reference for step 6).
Gate: pi-craft verify — duration equals the music, one clip more than there are cuts.
hits[].t and downbeats_video are video time, i.e. sequence time when the footage starts at 0 and the music
starts at 0 (add the lead-in to every time if the picture has one — probe the offset, never guess). FilmCraft: put
the footage on V1 and split all tracks at each cut time with sequence.addEditAllTracks (ticks, 254016000000 per
second; snapped to the frame). Cut times = every Nth downbeat (default N = 4; N = 2 for a busy edit) united with
every hit time, frame-rounded and de-duplicated by jq (no helper script needed). The footage must be at least
as long as the music; export with --start 0 --end <music seconds> so the picture ends with the music.
CUTS=$(jq -n --slurpfile e edit_edit.json --slurpfile h hits.json --argjson n 4 --argjson fps "$FPS" '
([$e[0].downbeats_video | to_entries[] | select(.key % $n == 0) | .value] + [$h[0].hits[].t])
| map(select(. > 0 and . < $e[0].duration)) | map(. * $fps | round) | unique | map(. / $fps)')
{ echo "{\"id\":\"file.newSequence\",\"params\":{\"name\":\"beat\",\"width\":$W,\"height\":$H,\"fps\":$FPS,\"fromItem\":$FOOT}}"
jq -c '.[] | {id:"sequence.addEditAllTracks",params:{time:(. * 254016000000 | round)}}' <<<"$CUTS"; } > "$JOB/05-cut/cuts.jsonl" # one line per cut
pi-craft filmcraft exec --project "$JOB/05-cut/p.fcproj" --save run "$JOB/05-cut/cuts.jsonl"
pi-craft filmcraft exec --project "$JOB/05-cut/p.fcproj" export "$JOB/05-cut/rough.mp4" --format h264 --start 0 --end "$D"
pi-craft verify "$JOB/05-cut/rough.mp4" --duration "${D}±0.1" --has-video
Count the clips with inspect sequence --compact ([.video[].items[]?]|length must be cuts + 1): two hits closer
than one frame produce one cut, not two, and the next step pairs clips with hits by start frame.
Step 6 — Punch and deliver
Output: 06-delivery/p.fcproj, 06-delivery/final.mp4 (h264 + aac).
Gate: pi-craft verify — --codec video=h264 --codec audio=aac, duration ±0.1 s, --frame-diff at a hit
(punched, > 5 %) and after the settle (identical to rough.mp4, < 1 %).
The punch is FilmCraft's own motion effect — no EffectCraft round trip. For the clip that starts at each hit,
keyframe the motion effect's scale: recipe.<tier>.scale × 100 at the hit and 100 after recipe.<tier>.settle
seconds (read both from hits.json; defaults: big 1.12 / 0.6 s, mid 1.07 / 0.45 s). Use
effects.setParam with keyframe:true and a time in ticks; inspect sequence --compact gives the clip ids and
start frames, paired with hits by round(t × fps). Then put music.wav on A2 with a source patch and mute
A1 (footage sound off — the music is the soundtrack), and export. A plain effects.setParam without
keyframe:true is a static scale on the whole clip, not a punch.
pi-craft filmcraft exec --project "$JOB/06-delivery/p.fcproj" --save run "$JOB/06-delivery/punch.jsonl"
pi-craft filmcraft exec --project "$JOB/06-delivery/p.fcproj" export "$JOB/06-delivery/final.mp4" --format h264 --start 0 --end "$D"
pi-craft verify "$JOB/06-delivery/final.mp4" --duration "${D}±0.1" --has-video --has-audio \
--codec video=h264 --codec audio=aac \
--frame-diff "$T_HIT_PLUS:$JOB/05-cut/rough.mp4>5" --frame-diff "$T_SETTLED:$JOB/05-cut/rough.mp4<1"
$T_HIT_PLUS ≈ hit time + 0.1 s; $T_SETTLED = hit time + settle + 0.1 s but before the next cut. Edge bleed (frame
edge showing background at peak scale) cannot occur at scale ≥ 100 %.
Resume
Follow the Craft jobs convention in craft-studio: read craft-jobs/<slug>/job.json as data, confirm the job
name, and start at the first step with no passing report or whose recorded inputs (path, size, mtime) changed —
a replaced footage file or a re-cut music edit. Re-run every step downstream of it and reuse the verified outputs
before it (a changed edit.wav invalidates hits, conformed audio, cut and delivery). Record each step with the
job-helper snippet after its gate passed.
When a gate fails
Stop at the failed gate; never continue past it. Re-discover the commands you used (pi-craft filmcraft exec describe <id> / commands <word>), repeat the step and re-run the gate. At most 2 retries per step; after the
second failure stop and report the verify output and what was tried.
Recipes
# The FilmCraft/SoundCraft steps end to end with stand-ins for the music skills' output (music-edit/1, music-hits/1).
FPS=25; D=8
ffmpeg -v error -y -f lavfi -i testsrc2=size=640x360:rate=$FPS:duration=12 -f lavfi -i sine=frequency=330:duration=12 \
-shortest -c:v libx264 -pix_fmt yuv420p -c:a aac footage.mp4
ffmpeg -v error -y -f lavfi -i sine=frequency=110:duration=$D -ar 44100 -c:a pcm_s16le edit.wav
echo '{"schema":"music-edit/1","duration":8,"bpm":120,"downbeats_video":[0,2,4,6]}' > edit.json
echo '{"schema":"music-hits/1","edit":"edit.json","hits":[{"t":2.0,"tier":"big","source":"structural"},{"t":5.0,"tier":"mid","source":"hook"}],"recipe":{"big":{"scale":1.12,"settle":0.6},"mid":{"scale":1.07,"settle":0.45}}}' > hits.json
mkdir -p 04-audio 05-cut 06-delivery
# 03 hits: the skill-level check
jq -e '.schema=="music-hits/1" and (.hits|length>0)' hits.json
# 04 conform audio: PCM 24-bit 48 kHz
pi-craft soundcraft exec convert edit.wav 04-audio/music.wav --bit-depth 24 --rate 48000
pi-craft verify 04-audio/music.wav --duration ${D}±0.05 --codec audio=pcm_s24le
# 05 cut picture: edits on every 2nd downbeat and on every hit
CUTS=$(jq -n --slurpfile e edit.json --slurpfile h hits.json --argjson n 2 --argjson fps $FPS '
([$e[0].downbeats_video | to_entries[] | select(.key % $n == 0) | .value] + [$h[0].hits[].t])
| map(select(. > 0 and . < $e[0].duration)) | map(. * $fps | round) | unique | map(. / $fps)')
echo "$CUTS" > 05-cut/cuts.json
pi-craft filmcraft exec --save-as 05-cut/p.fcproj import footage.mp4 > 05-cut/items.json
FOOT=$(jq -r '.items[0]' 05-cut/items.json)
{ echo "{\"id\":\"file.newSequence\",\"params\":{\"name\":\"beat\",\"width\":640,\"height\":360,\"fps\":$FPS,\"fromItem\":$FOOT}}"
jq -c '.[] | {id:"sequence.addEditAllTracks",params:{time:(. * 254016000000 | round)}}' 05-cut/cuts.json; } > 05-cut/cuts.jsonl
pi-craft filmcraft exec --project 05-cut/p.fcproj --save run 05-cut/cuts.jsonl
pi-craft filmcraft exec --project 05-cut/p.fcproj export 05-cut/rough.mp4 --format h264 --start 0 --end $D
pi-craft verify 05-cut/rough.mp4 --duration ${D}±0.1 --has-video
test "$(pi-craft filmcraft exec --project 05-cut/p.fcproj inspect sequence --compact | jq '[.video[].items[]?]|length')" -ge "$(( $(jq length 05-cut/cuts.json) + 1 ))"
# 06 punch + deliver: scale keyframes on the clip that starts at each hit, music on A2
pi-craft filmcraft exec --project 05-cut/p.fcproj --save-as 06-delivery/p.fcproj import 04-audio/music.wav > 06-delivery/items.json
MUSIC=$(jq -r '.items[0]' 06-delivery/items.json)
pi-craft filmcraft exec --project 06-delivery/p.fcproj inspect sequence --compact | jq -c '[.video[].items[]?|{clip,startFrame}]|sort_by(.startFrame)' > 06-delivery/clips.json
jq -n -c --slurpfile h hits.json --slurpfile c 06-delivery/clips.json --argjson fps $FPS '
$c[0][] as $clip
| ($h[0].hits[] | select((.t * $fps | round) == $clip.startFrame)) as $hit
| ($h[0].recipe[$hit.tier] // {scale:(if $hit.tier=="big" then 1.12 else 1.07 end),settle:0.5}) as $r
| ({id:"effects.setParam",params:{clip:$clip.clip,effect:"motion",param:"scale",value:($r.scale*100),time:($hit.t*254016000000|round),keyframe:true}},
{id:"effects.setParam",params:{clip:$clip.clip,effect:"motion",param:"scale",value:100,time:(($hit.t+$r.settle)*254016000000|round),keyframe:true}})' > 06-delivery/punch.jsonl
test -s 06-delivery/punch.jsonl
cat >> 06-delivery/punch.jsonl <<JSONL
{"id":"timeline.setTrack","params":{"track":"A1","muted":true}}
{"id":"source.open","params":{"item":$MUSIC}}
{"id":"timeline.setTargeting","params":{"track":"A2","sourcePatch":true,"targeted":true}}
{"id":"timeline.setTargeting","params":{"track":"A1","targeted":false}}
{"id":"playhead.set","params":{"seconds":0}}
{"id":"source.overwrite","params":{}}
JSONL
pi-craft filmcraft exec --project 06-delivery/p.fcproj --save run 06-delivery/punch.jsonl
pi-craft filmcraft exec --project 06-delivery/p.fcproj export 06-delivery/final.mp4 --format h264 --start 0 --end $D
pi-craft verify 06-delivery/final.mp4 --duration ${D}±0.1 --has-video --has-audio --codec video=h264 --codec audio=aac \
--frame-diff 2.1:05-cut/rough.mp4\>5 --frame-diff 3.5:05-cut/rough.mp4\<1