Assembling a music video in code — how 딸깍 was built

An AI music video is not a song dropped into a video model. The models make the materials. The code makes the editing decisions. 딸깍 ("click") is a 186-second K-pop video built on that split.

Suno wrote the song. GPT Image painted Haeba, the lead. Seedance 2.5 shot the dance. But all 4,469 frames that reach the screen were drawn by JavaScript in a browser. Every AI output enters as one layer.

This post follows that structure in order. Numbers are from the v1 render on 2026-09-26.

Everything hangs on the song's clock

Every event in a music video belongs to the song's time — cuts, lights, mouth shapes, lyrics. So the first job was turning the song into an exact clock.

Suno ships a lyric alignment. It does not match what was sung. The sheet says "딸깍" twice; the singer sings it three times. Nine aligned lines are repeats nobody sings. We re-transcribed the lead vocal stem with whisper, section by section, and rebuilt the alignment from the 76 lines actually sung.

The tempo is not fixed either. Take A2 drifts from 126.2 to 127.3 BPM. A fixed-BPM grid misses beats by up to 164 ms. A measured beat map lands within 5 ms RMS.

With both in place, the code can ask questions like these:

BAR(26)                        // the second bar 26 starts
findLine('며칠 뒤 점수')          // start and end of that lyric line
sylOnsets(line, '깍')           // every moment "kkak" is sung in it

No second is typed anywhere in the video code. Swap the take, and every cut recomputes against the new song. The stage flashing on every "kkak" and the puppet tapping on every "kkak" both come from this syllable clock.

A frame is a function of time

The renderer is one web page running p5.js and p5.brush. A Node script drives headless Chrome, hands it a time t, and gets back that instant's frame.

One rule holds it together: the same t renders to the same pixels, every time. Randomness is re-seeded per frame and per layer. The spring that swings her hair keeps no state; it re-filters the past motion.

That rule lets frames render in any order. Two workers split the work, and a stopped render resumes. The first full render lost Chrome near frame 2,000. The restart filled in only the missing frames, identical to a clean run.

A frame is seven layers, stacked in order:

set → type → footage → chars → fx → inserts → grade
plate  big word  dancer  puppet  crowd  memes/lyrics  paper

Type is drawn before the dancer, so the giant "딸깍" sits behind Haeba. The final grade lays one paper tooth and grain over everything — different sources, one painter's hand.

Four materials reach the screen

Sets are one painting plus light

Five locations: a sunflower stage, a 2 a.m. rooftop room, a chalkboard lecture hall, a Yeouido rooftop, a persimmon hill. Each is one GPT Image painting.

The emissive parts were masked out of each painting — 18 LED petals, bulbs, neon, windows. At render time only the light is added back: petals chasing on the beat, swinging beams, a flash on every click. No set painting was regenerated once.

The singing face is one painting, deformed

Haeba's singing close-ups are not video. They are a puppet. One painting was cut into body, hair, face and jewellery; mouths, eyes, arm gestures and three-quarter heads are sprites from an edit model. Each piece sits on a grid mesh that bends every frame — the Live2D approach.

The mouth follows a vowel track from the lead stem; the syllable's own vowel picks the ㅏ, ㅗ, ㅜ or ㅣ shape. A 50 ms "딸깍" still opens the mouth for at least 2.5 frames. Head turns and blinks come from six face-mocap takes shot in Seedance, one per song part.

Gestures come from the lyric. A "딸깍" line gets the click pose, tapped on every "kkak"; "딸깍 한 번에" throws a finger gun on "한". Other lines pick a point, a thumbs-up, a wave or a heart.

The dance is Seedance footage, cut out

The full-body dance is eight Seedance 2.5 takes at 720p. Each starts from a crop of the stage or hill painting, so the style and background match the plates.

The dancer is cut out of every frame. BiRefNet runs on the Apple GPU and writes an alpha matte per frame, about 80 seconds a take. The model sometimes keeps an LED petal as part of her, so anything outside a pose-drawn body region is cut, and a three-frame median calms the edges.

The composite is a sandwich: the footage's own picture at the bottom, the big word in the middle, the cut-out dancer on top. The word ends up in front of the stage and behind her.

The crew is one dancer, six times

The backup crew is one dancer shot in Seedance on a flat teal backdrop, which keys out cleanly. That one take is placed six times in a V. Each rank trails by two frames so moves ripple outward; the right-hand three are mirrored.

Footage never lands exactly on the beat. The fast hits in each take are pinned to the nearest beat, and the playback between pins stretches between 0.72x and 1.4x.

The edit is 64 function calls

Lyric typography is planned per line, all 76 of them, in one file. Each line has a mode — HERO fills the frame, STAMP lands like a seal, CALL takes the gang answer, CHOP splits a word.

The edit file braids the materials into 64 shots. A shot is one call:

shots.push(wide('CH1-1', BAR(26), 'S09'));                          // dance wide, two bars
shots.push(crew('CH1-1c', BAR(28), D1, 'stage', { haeba: 'S09' }));  // crew formation, one bar
shots.push(pup('CH1-3', BAR(32), 'stage', { flips: true }));         // call close-up

A chorus runs two bars wide, one bar crew, one bar wide, an insert, the call close-ups, the crew hits. The rap alternates puppet and meme one bar at a time. The memes — a permission door, an AGI scoreboard, GPU confetti, a "next stop: singularity" sign — are JavaScript shapes too.

Frames average about 70 ms. A full 16:9 pass takes a little over three minutes. The 9:16 version redraws the same scenes on a vertical canvas as vectors; type stays sharp, while the 720p dance footage softens under the zoom. Three 59-second Shorts are cut from the vertical master at the ranges the shot list names, with an 8 ms fade at each join.

The failures drew the architecture

None of this was designed up front. Three failures shaped it.

The code-drawn character was not pretty. The brush-painted face scored 4 out of 10 in review — "a cartoon filter with stickers." So the face is now the image model's painting, and the code only moves it.

Seedance could not lip-sync singing. With the song as an audio reference, the mouth still missed, and the dance missed the beat. So the puppet sings, the footage only dances, and the code keeps time.

The credits ran high. The project used about 1,545 Higgsfield credits; 426 of them went to photoreal mocap that the character change made obsolete. The final assembly used zero. Every one of the 64 shots came from material that already existed.

One smaller miss: lyrics painted inside the laptop screen and the chalkboards looked right in the plan and were unreadable at real size. They became on-screen type.

Split the work between the model and the code

The rule this project leaves behind is short.

The model owns the quality of one image, one take — a pretty face, a coherent style, a body that moves like a body. The code owns time and repetition — beats, syllables, cut points, exact re-renders, thousands of consistent frames.

Keeping them apart makes revision cheap. Changing a cut costs no credits: edit one line, re-render that range.

If you are making video with AI, ask this before asking a model for a finished piece: what in this scene can only the model do, and what can the code do more precisely?

Tools

• Song: Suno (take A2, 186.2 s); lyric check with whisper large-v3
• Images: GPT Image 2.5 (character, five locations)
• Video: Seedance 2.5 via Higgsfield (8 dance takes, 4 crew takes, 6 face-mocap takes)
• Mattes: BiRefNet_lite (PyTorch MPS), MediaPipe pose
• Render: p5.js 2.3.3, p5.brush 2.2.3, Puppeteer, headless Chrome, ffmpeg