I have a script that loops over a 6MB text file and takes each line to do a large amount of processing: creating structs, lots of queries, calling other CFCs, etc. The problem is this loop will eventually take up 100% of its alloted memory and eventually timeout with an “out of memory” error. I cannot figure out if it is something I am doing wrong or if it is the nature of CF. From what I’ve read on the web, CF is notoriously poor at garbage collection and will not flush its memory until after the request is complete, which leads me to believe there is nothing I can do.

I have looked over the code with others and can 100% confirm that all variables in a function are var scoped, each structure is cleared at the head of the loop, and there are no open loops or areas where memory would obviously be leaking from.

I can also confirm with the following code (found here) that with each 1000 iterations through the loop, 20MB of memory is being used, which eventually builds up until it’s maxed out.

<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset memoryUsed = (runtime.totalMemory() - runtime.freeMemory()) / 1024 / 1024>

It was my understanding that with each loop iteration, recreating the structures that get built within the loop clear them from memory, but this apparently isn’t the case. I have also tried tapping into Java to manually force garbage collection, which also doesn’t do a thing. My only solution at the moment is to increase the Java heap size to 2500MB – enough for the damn thing to run – but once it has finished and other requests are made, memory is never flushed. I then force kill CF and restart it. This is fine for my development box, but is not going to fly in production, and this script will probably be run a couple times a month.

So, aside from posting ~500 lines of code across a handful of files, any quick ideas I can investigate? I’m running CF8 on Ubuntu 9.04.