Creating a distributed render farm with Blender
Jamesy writes:
Hello there! Have you ever thought how you would cope with a tight deadline for an animation project, no budget for a renderfarm and not enough compute power to do the job?
A friend of mine had a similar situation and needed help fast. So a group of us got together online and used cloud storage and blender to combine our computers and help get the project rendered on time. This method saved countless hours of render time and the project was able to finish before the deadline.
So if you have friends with computers – even if they’re spread over the entire Earth – this is something you could do that might not cost you anything but time.
This is something I have been waiting for since I started using blender, more or less 10 years ago. I am testing the last version at this very moment and have to say it is very promising and definitely a great addition to blender tools. Thank you very much for your great work!
James: A few suggestions:
A Cancel Rendering button that stops all nodes
Making using this machine optional
Is it possible to reuse a node once it is done with the portion of the image it was given? Complex, I know.
If adding the same node more than once is not allowed can you eliminate duplicates, or show an error message ?
Thank you again for this amazing add on!
Hi Victor,
Awesome! Working on the cancel issue, hopefully fixed this week. As for reusing nodes, not exactly sure what you mean there! Any chance you can elaborate? Do you mean using it for other renders from other files?
All the best
James
I mean that some nodes can be really fast and others way too slow, I don’t what criteria you are using to dispatch the jobs to the nodes, but I can see that when rendering stills faster nodes will finish and the slower nodes will still be struggling there for quite a while.
I noticed that after a few renders the addon learns to give faster nodes bigger chunks of the image than those given to the slow ones, which is nothing short of amazing, but still sometimes faster nodes will finish way before the slower ones just to sit there still.
Can they take over new buckets that are scheduled to be rendered by the slower nodes?
Sorry if my explanation is not clear enough, my English is very poor.
Errata: I meant: “I don’t know what criteria”.
Hi Victor, your english is fine, that explanation nailed it; thanks!
You’re right, we are changing the addon to perform load balancing sort of like you mentioned, it may be a little different but will have the same effect. Our actual goal is to get perfect load balancing on the first render so its useful for full res, full sample final renders of stills, first go.
That assumes the artist got the scene debugged and is happy with everything though, we can’t do that for them too ;)
James after a few renders the addon started giving the precise amount of work to each node and the performance improvement was optimal. That is more than enough for me.
I am sorry to bother you with these nuances but thought that maybe showing some statistics would be really cool, like number of total cpu/gpu cores in the render farm, just for bragging purposes LOL.
Is CrowdRender able to use gpu and cpu at the same time? If not, I watched a video where somebody just opened two instances of Blender and put them to work on an animation project one using the cpu and the other using the gpu. Could that idea help? Just being extra naive here.
This is so far the best addon ever! Thanks a lot.
Hi Victor,
Great to hear that you got it to work :) I’m working on supporting the daily builds which I believe contain the GPU + CPU code for rendering. They use an updated version of python that i have to support since we used compiled code that targets a particular python revision. Since the daily build I think has this feature uses the newer version, I need to update the builds. This isn’t exactly straight forward, but I’ll do my best! Look out for an announcement on our website if you’re subscribed, if not you can subscribe by going to http://www.crowd-render.com. There’s a link on the front page ;)
Great video. Am I correct in assuming you didn’t even use crowd render here?
And could you make a tutorial on how to leverage the power of Azure/ Google cloud/ Amazon with Crowdrender?
I don’t have many computers at my disposal, but could rent some for bigger render jobs…
Hi Erik :)
Yes, you’re absolutely right, at the time the deadline was looming and we couldn’t open the file with crowdrender, our addon had a few issues which we fixed…. after we’d rendered the project and the deadline had passed! It was more important to make the deadline and besides the technique is actually very easy and an option for anyone right now that needs extra compute power.
Crowdrender is catching up fast though! We’ll definitely be doing another tutorial real soon to show case the new build and for sure use with AWS. We have a working prototype that can use AWS that is getting very usable. Nice thing is you can combine your local hardware with the cloud and that of your peers. Very flexible!
Thanks for commenting Erik :D
All the best
We use a hybrid farm with https://cloudrender.farm that allows us to get better pricing than AWS/gcloud etc but with same render performance