The .blend files of Orange are requiring more and more memory to render/composite and Ton has been struggling to find a solution. As it turns out, OSX (Orange uses an Apple XServe renderfarm) can only allocate 1.5GB per application at a time and windows seem to have similar problems.
He was able to come up with a solution though and it looks like it's a big leap forward for all of us who work with (really) large scenes.
Ton writes:
On my system, just 1 GB of memory, it uses almost 2.5 GB, and editing still is pretty responsive! Seems like this is an excellent way to allocate large data segments… and leave it to the OS to swap away the unused chunks.
Just a couple of hours ago I've committed this system integrated in our secure malloc system. While rendering now, all texture images, shadow buffers, render result buffers and compositing buffers are using mmap, with very nice stable and not-fragmented memory results. :)
Even better; with the current tile-based rendering pipeline, it's an excellent way to balance memory to swap to disk when unused, to be able to render projects requiring quite some more memory than physically in your system.
The full article is available on the Orange Blog. A fixe for Windows is already on their way.
Update: As Diffid pointed out, only Windows suffers the same problems. Other Operating Systems seem to be fine.
6 Comments
I've committed a patch to the bf-committers list and on the patch tracker to implement the new memory mapped management system also on win32 platforms. It works like a charm so far, hope the dev team commit it soon on also on CVS.
If anyone wants to try it just download the source file here:
http://www.infosquid.net/repo/mman_win.h
Then put an include in intern\guardedalloc\intern\mallocn.c at line 44 and build the whole package ;-)
A new full fledged & patched windows build by me is coming soon here on BlenderNation.
Your Blog says:
As it turns out, OSX (Orange uses an Apple XServe renderfarm) can only allocate 1.5GB per application at a time and other platforms seem to have similar problem.
Just to clarify "other platforms" comes down to M$ Windows:
Ton Quote from Orange Blog:
Now I can already see the Linuxers smirk! Yes indeed, doing renders on our Linux stations just went smooth and without problems. Linux starts with memory allocations somewhere in the lower half, and will easily address up to 3 GB or more.
We've used (to make virtual visit of General Hospital of Ciudad Real) a render farm made with Oscar under linux stations (60-70 Pentium IV), but we don't need more than 1 GB of RAM (and the computers have exactly this size of memory). In fact, we can study buying more RAM for the computers. If the University of Castilla-La Mancha can support this project, this could be good! :-)
Hi Diffid,
thanks for that. I've updated the post.
Bart
My mman_win.h implementation resolves the problem also on Win32 :-P
Lox: cool! I'm sure many people would like to have a go at your patched version.