Render times progressively increasing on batch script render

[HiRez]

Hi, I have written a JavaScript which reads in text files and does a series of text replacements and renders based on the text in the file. The script runs completely internally to AE (UI version). Each line of the input file is rendered by itself each time through the loop. The render queue is deleted before each render, and all caches purged after each render (I have confirmed it is returning the memory). The file is very simple: one background movie and two text layers, no fancy effects or expressions.

The results are fine, but the weird thing is that each pass through the loop the renders progressively get a little bit slower. It starts at around 30 seconds and each pass adds a second or two. This is a problem since I have over 3,000 (and more later on) of these animations to render and the extra seconds are really adding up. Quitting AE and restarting it is the only thing that seems to work to reset the render times, but that kind of defeats the purpose of scripting and batch rendering.

I guess I could have Python or AppleScript do all the looping and restart AE periodically, but that would involve rewriting the whole thing in a different language, which I don't have time for, so I'd rather avoid that. Any ideas? Anyone noticed this behavior before? Seems like a bug but I dunno.

I'm using a Core2Duo 2.16 GHz PowerBook, 2 GB RAM (plenty free while running), OS X 10.4.11, After Effects CS3 8.0.1.8. But, I have noticed this same behavior on a dual G5 tower.

[Darkmoon_UK]

I write Scripts to produce hundreds of data-driven graphic screens and have noticed the same thing - the slowdown is quite significant. So far my solution has just been to throw enough PC's at the render to get it done in time, but it would be very nice to get to the bottom of the issue.

I think its very likely that this is down to memory management - that internally, AE is building up some long list of asset references that doesn't get cleared, so with each iteration, JavaScript is searching through more and more redundant entries to get to the objects it needs.

If its bad on After Effects, spare a thought for Photoshop scripters - I was horrified to discover recently that actually, Photoshop *never* releases memory that has been consumed by loading images. That's right: One too many iterations of an image loading script and your memory fills right up and you have to restart the application. Surely a dark legacy of bad code in there somewhere.

Fortunately we can see from monitoring memory usage that this is not the same case in AE, but somethings going on - can anyone suggest any ways to flush caches or clear memory in some way?

[Darkmoon_UK]

w00t!

I was Googling for a reference to that Photoshop issue; and came across this thread:

http://ps-scripts.com/bb/viewtopic.php?p=8015

To quote:

Most scripting languages (and some others) have automatic memory management. The interpreter has the smarts (usually) to know when memory is no longer being used by the script. However, with some kinds of scripts, the interpreter needs a little help. That's where $.gc() comes in. When you call it, it forces the interpreter to immediately check and see if there is any memory that the script is no longer using and frees it up. This is useful in long running PS scripts where you are using a lot of data.

This post is written about Photoshop, but at these low levels, the workings of Adobes JavaScript engine seems to be common to all Creative Suite applications. This post from Adobe, although in a slightly different context, does confirm that $.gc() has an effect within its own JavaScript engine:

http://support.adobe.com/devsup/devsup. ... /54356.htm

I don't have time to test right now, but perhaps running $.gc(); at the end of each loop iteration, will at least keep some more memory free and speed things up? I am also wondering if object.gc(); can be used to perform this memory check on specific objects to optimize things?