extract-clips
Extract and rank short clips from long-form video content for repurposing as shorts, reels, or TikToks.
SKILL.mdAutoAnimation/.claude/skills/extract-clips/SKILL.md
GitHubPurpose
Analyze a long-form video transcript and identify the 3-5 best standalone segments for repurposing as short-form content (30-90 seconds each). Uses Gemini AI to find viral-worthy moments, then ranks clips using a composite scoring system that combines virality, visual interest, audio energy, and hook potential.
Steps
Transcribe the source video. Upload the video/audio file using
transcribeAudio(file, options)fromwhisperTranscript.ts. Supported providers: 'whisper' (default), 'deepgram', 'assemblyai'. ReturnsWhisperResultwithsegments,words,text,language, andduration.Extract best clips. Call
extractBestClips(segments, totalDuration)fromclipExtractor.ts. This:- Formats the transcript with timestamps (e.g.,
[1:23] text...) - Sends to Gemini 2.0 Flash asking for 3-5 best standalone segments
- Requests JSON with: title (max 8 words), hookRewrite (attention-grabbing first sentence), startTime/endTime, viralityScore (0-100), reason
- Falls back from server proxy to direct API if proxy unavailable
- Returns
ExtractedClip[]sorted by virality score descending
- Formats the transcript with timestamps (e.g.,
Rank clips with composite scoring. Call
rankClips(clips)fromclipRanker.ts. Scoring weights:- Virality score (35%): From Gemini analysis (0-100)
- Visual interest (20%): Based on scene transition count (
sceneTransitions * 25 + 40, capped at 100) - Audio energy (25%): Based on emotional arc -- climax=90, rising=75, falling=40, flat=30
- Hook potential (20%): 80 if
hookRewriteexists, else 50 - Returns
ClipRank[]withcompositeScoreand per-signalbreakdown
Rewrite clip hooks. For any clip, call
rewriteClipHook(clip)fromclipExtractor.tsto get a Gemini-rewritten opening line that is more attention-grabbing. Uses temperature 0.7 for creative variation.Preview and select. Each
ExtractedClipincludes:id-- unique clip identifiertitle-- catchy title (max 8 words)hookRewrite-- rewritten opening sentencestartTime/endTime/duration-- exact time boundsviralityScore-- 0-100 score from AI analysissegments-- matchingWhisperSegment[]within the time rangereason-- explanation of why this segment works- Optional:
thumbnailDataUrl,videoPreviewUrl,sceneTransitions,emotionalArc
Export selected clips. Use the start/end timestamps to slice the original video. The clips can be imported as new ProAnimate projects or exported directly.
Key Files
src/services/clipExtractor.ts-- extractBestClips(), rewriteClipHook(), ExtractedClip typesrc/services/clipRanker.ts-- rankClips(), ClipRank type, composite scoring logicsrc/services/whisperTranscript.ts-- transcribeAudio(), transcribeFromUrl(), WhisperResult, WhisperSegment, WhisperWordsrc/components/panels/ClipExtractionPanel.tsx-- Clip extraction UIsrc/components/panels/ClipExtractorPanel.tsx-- Clip extractor UI
Common Issues
- Gemini API key required: Both extraction and hook rewriting need
VITE_GEMINI_API_KEY. The service tries a server proxy first, falls back to direct API. - Credit gating: Both
extractBestClips()andrewriteClipHook()are wrapped inwithCreditGate('gemini-script', ...). - Clip boundaries may be imprecise: Gemini estimates timestamps from transcript text, not audio analysis. Clips may need minor manual adjustment at boundaries.
- Short source content: Works best on 5+ minute source videos. Very short content (under 2 minutes) may not yield meaningful clips.
- Proxy fallback: The service first tries
/api/proxy/gemini/gemini-2.0-flash, then falls back to direct Gemini API. Ensure one path is available.
Examples
import { transcribeAudio } from '@/services/whisperTranscript'
import { extractBestClips, rewriteClipHook } from '@/services/clipExtractor'
import { rankClips } from '@/services/clipRanker'
// Transcribe
const transcript = await transcribeAudio(videoFile, { provider: 'whisper' })
// Extract clips
const clips = await extractBestClips(transcript.segments, transcript.duration)
// Rank with composite scoring
const ranked = rankClips(clips)
// ranked[0].compositeScore, ranked[0].breakdown.viralityScore, etc.
// Rewrite a hook for the best clip
const betterHook = await rewriteClipHook(clips[0])