Audio synch to Video Speed

Moderators: Disciple, zlovatt

Post Reply
RichParisWilson
Posts: 3
Joined: December 30th, 2007, 6:32 pm

So, I'm getting in a bit of a muddle here, and hoped somebody could help me leap over the last hurdle.

Basically, I want to synch' the Audio to the speed at which the video image is displayed. So, when the track is LOUD, the image speeds up, and when the track is quiet, the image slows down. I realise the image will be jumping ALL over the place, but that's the look I want.

So, to do this I've turned the audio into keyframes (right click on the track > Keyframe Assistant > Turn Audio into Keyframes), which creates a beautifully little null. I can do this to the treble and bass separately if I wish, but for now I've created a single null for both.

I then link the slider for Both Channels within the null to the position of the null, so that the null moves with each beat. I can link the null to the scale of a video layer if I so chose, with no problem, adding a small expression to ensure the scale is always above 100 (( linear(value,0,100,100,140) )).

But what I want to do is link the Slider within Both Channels of the null to the Speed at which a video layer is played. Currently, if I do this through Time Remapping, I just get a video that bounces around through the corresponding timecode, but what I want it to do is play faster and slower. Does anybody know the expression I need in order to do that?

Thanks a ton.
Mylenium
Posts: 139
Joined: July 20th, 2005, 12:07 am

Use the audio as a multiplier to normal playback rather than the value itself. Other than that you will have to reduce the number of keyframes to the most distinct ones, access the keyframe values directly and interpolate on your own. All of that can be found as an example (for markers) on Dan Ebberts' website.

Mylenium
[Pour Mylène, ange sur terre]
nab
Posts: 203
Joined: November 29th, 2005, 3:00 am
Location: Royan
Contact:

I don't think you can use the audio amplitude (or some "linear()-ized" value to change its range) as a timeremap multiplier because the animation will jump forward and backward, not slower and faster.
You need an expression that loops through the comp or layer duration and maintains a counter (say a number of frames) that increases at each frame by an amount based on the audio amplitude (you also need the range of your audio, the min max values as references for calculation).
I've got a script that does this job. After converting your audio to keyframes, you select your video layer and that's it (there is a controller to specify how "intense" the acceleration will be).
It's not on my website but here is a demo. The basic animation is a camera flying through some text layers (the names of the 70 top posters on aenhancers) along the Z axis. The audio amplitude of the track influences the speed of the camera movement (actually it drives the timeremap property of the precomp).

If you're looking for something like that, I can make the script available...
audioAmp_to_videoSpeed
RichParisWilson
Posts: 3
Joined: December 30th, 2007, 6:32 pm

Thanks to both of you. Really appreciate your thoughts.

Nab - yeah, when using the amplitude data for time remapping, I get a video that continually jumps around, which is an interesting effect in itself, but not what I had in mind. I'm VERY interested on getting my hands on your script. The demo vid displays an effect that is exactly what I'm looking. Good stuff (and great choice of music)

If you'd like to keep the script quiet, we can discuss this over email (richy@outcast.tv) - although I know other users of Ae Enhancers would love to see it too!
nab
Posts: 203
Joined: November 29th, 2005, 3:00 am
Location: Royan
Contact:

The script can be downloaded here 8)

Usage:
1. convert your audio to keyframes
2. select your video or precomp layer
3. execute the script

A null layer and a parameter called "Max Forward" are created at the top of the comp, this parameter controls the "speed"...
RichParisWilson
Posts: 3
Joined: December 30th, 2007, 6:32 pm

nab wrote:The script can be downloaded here 8)

Usage:
1. convert your audio to keyframes
2. select your video or precomp layer
3. execute the script

A null layer and a parameter called "Max Forward" are created at the top of the comp, this parameter controls the "speed"...
Nab - that's amazing.

The script works extremely well. It's a tad chaotic with loud music (aside from using Audition or Soundbooth to change the Bass, or Treble, etc before I turn the audio into keyframes, would I be able to alter the chaotic nature with a linear(value, _, _, _, _)) expression in there somewhere?), but it works fantastically well in most occasions, especially when there's a distinct beat and lots of silences.

Thanks a ton, and well done! If I ever use it in a music video (like I intend to), I'll credit you, and even send you some cash if I have a budget!
nab
Posts: 203
Joined: November 29th, 2005, 3:00 am
Location: Royan
Contact:

You could use a "smooth()" expression on your audio data (see the doc for usage) or play with The Smoother (from the Window menu). Sometimes this helps.
tedmihu
Posts: 3
Joined: June 28th, 2014, 10:16 am

Hello, I'm trying to make a script work and the part I keep having issues with is selecting the audio layer and getting an audio amplitude null from it. The code is below:

Code: Select all

var selectedItems = app.project.selection;
var bodyComp = app.project.items[1];
var mouthComp = app.project.items[2];

for (var i = 0; i < selectedItems.length; i++) {
    var compName = selectedItems[i].name.substr(0,(selectedItems[i].name.length - 4));
    var myComp = app.project.items.addComp(compName,1080,1920,1,1920,24);
    myComp.layers.add(selectedItems[i]);
    myComp.layers.add(bodyComp);
    myComp.layers.add(mouthComp);
    myComp.layer(1).timeRemapEnabled = true;
    }

for (var i = 0; <selectedItems.length; i++) {
    if (app.project.item(i) instanceof compItem) 
        app.project.item(i).openInViewer();
        app.project.item(i).layer(3).selected = true;
        app.executeCommand(app.findMenuCommandId("Convert Audio to Keyframes"));
        }

What I need the script to do is create new compositions based on the name of the voice over file minus the .wav extension, then import a "Body Loop" and "Head Loop" comp into each one, as well as the corresponding VO track for each comp. Then I want it to set the work area to the length of the VO track, and create an audio amplitude null, which I will then use to animate the time-remapped mouth loop with an expression. As a bonus I would also love for the script to add a 2x smoother for the "Both" channel keyframes. I've figured out how to do most of what I need in the code above but I can't figure out how to select the VO track in each comp and generate a null object from it.

Any help would be greatly appreciated as we are currently spending hours doing this manually.
Post Reply