CrowdRender: Why we built a network rendering platform

James Crowther shares CrowdRender, the new crowd-sourced render farm that he presented at the Blender Conference last year. It presents an alternative to crowd rendering platforms like Sheep It with some additional options like creating pools with your own hardware.Since 2013 we’d been working on a problem that had kept us, or rather out computers up at night, how to get content rendered. In particular we we’re spending a lot of render time doing small tests. The purpose of those tests was to make sure there were no bugs in our work prior to committing our work to completely render, start to finish. We’d been here many times, very long render times and the uncomfortable feeling of whether the work would be finished in a reasonable timeframe.

I’m not going to lie, we’re not the worlds greatest artists, but we are very good at solving problems. So we set about building an addon that could use whatever computers we had available. This made sense to us since we did have a few machines lying idle that we couldn’t use in rendering the single frames we wanted to do in our tests. But, we couldn’t find any software that could use multiple machines to accelerate rendering a single frame, not for blender at least. Another problem we found with the existing network rendering solutions was the fact that the whole file has to be updated each time a change to the scene is made. This wasn’t a problem for small files, but for larger files it becomes very bandwidth intensive, especially if a large number of machines is involved.

Many development hours later we built a system that can stream edits to a scene in real time from one computer to another and then stream back the rendered image parts into blender’s “Render Result”. This makes for fast turn around times when you are testing scenes prior to committing to a full render for production as well as being able to render the complete animation as well. It works with as little as just two machines and we’re working on supporting, dare I say it, unlimited!

We presented a demonstration of our early prototype at the 2016 Blender Conference – you can see our presentation here:

Since we presented, we’ve modified our prototype to make it ready for testing and built a website to support our addon. We’ve been lucky to have lots of artists become subscribers with many of them requesting to be a part of the alpha testing.

Our alpha testing is perhaps the most important part of our project to date, we’ve got a lot of ideas of how the beta version should work, but we want artists to lead us, tell us what they need so we can prioritise our work. When our subscribers test the addon, they’ll be able to give us very valuable feedback on what matters to them in helping them create and finish their work.

So the future, partly it depends on the response from the artist community as to what they need when it comes to rendering, either for testing new ideas, debugging or final rendering. We also have our own ideas we’d like to try. For example, we’re working with the team behind the short film Lukes Escape to help them group their machines together for a final render. The problem they face is, they have no office and the team is spread out across the world. So our solution has to work over the internet. To fix that problem we’re working on a solution that will help teams share their physical computers as well as their talents to accelerate rendering.

We’re very excited (as you can imagine) as we’re releasing our alpha in (hopefully) a few days from the time of posting this article. If you would like to test our alpha release, head to www.crowdrender.com.au where you can register. We’ll contact you via e-mail when we release the software.

What makes our platform different from sheepit?

Technically, there are quite a few differences, here are some of the main ones:

• Crowdrender is an add-on rather than a web application, this means you do not require access to the internet to use it, you can use it on your home network to connect your own computers together. Or you can use it to connect to other peoples computers via the internet. Eventually we’ll also allow you to connect to render farms.
• Crowdrender does real time updates, there are no file uploads other than an initial upload then the nodes are kept in sync using a stream of updates, the only time a file upload is required is if there is a problem streaming or the user adds a texture file or other data that cannot be replicated by using blender’s operators. This is probably the biggest difference, sheepit, burp, renderfarm.fi, net render and loki are all examples of 1st generation distributed render solutions. They aim to solve the problem of rendering content that is final and just needs rendering start to finish. Our add-on focusses on the part just before that, debugging and perfecting sequences, shots or stills. This is why we went to the trouble of building a system that can update other machines with the artists edits in real time.
• Crowdrender is actually an api, a platform to assist blender to distribute its sessions live across the internet and local networks, other functionality can then be built on top of this, like collaboration between artists in difference countries (including chat, audio), other forms of acceleration. We’re looking at physics simulation as the next target for us in a couple of years from now, distributed games and so on. Its a similar idea to platforms like facebook that provide a resource via an api. In facebook’s case it was a whole lot of personal data with apps onto of that, in our case its processing power with apps built onto of it, like blender for example.

Who is behind this project?

At the moment that is my self and Jeremy Folland, you can see us presenting at Blender Conference here. Jeremy is sitting at the computer and I’m standing up. We’ve incorporated a company and we’re using this as the way to bring crowdrender to artists.

What are the current limitations?

• that it will only allow two machines to connect.
• Only a single blend file is supported, no external files (textures can be packed into blend file) so use of libraries is out, point cache is also not supported, yet.
• Compositing can be done during the render, but the render result doesn’t contain separate passes, so tweaking the compositing setup using crowdrender is currently not supported.

All of the above are short term gaps that we have plans to fill. The development timeline is alpha release and testing until April 2017, Beta release by June 2017, MVP by (hopefully) Blender Con 2017