Fresh Blender News, Every Day
Free rendering on 250-CPU Grid
Published by Bart on November 28th, 2008 in 3D News, Toolbox
The Centre for Parallel Computing (CPC) at the University of Westminster, London are test-driving their 250-CPU Grid. It's built around Blender and they're inviting YOU to render your work there.
Zsolt Lichtenberger wrote:
We are the Centre for Parallel Computing (CPC) at the University of Westminster, London. As the name suggest, we are interested in parallel computing, particularly in computer grids. We would like to promote Blender on the Grid and we kindly ask for your help.
We created a simple rendering portal, which uses our 256 CPU AMD Opteron cluster. You can use it to render your Blender projects and we are offering this free of charge. Please give it a try!
We are looking for feedback because we wish to understand the needs of Blender users and to improve the portal.
Links
- Rendering portal
- Mailing list (email link)
Related Posts
Related posts are selected automatically based on their content.
too bad you cannot protect your work inside .blend files :( so once you give the file it becomes public..
Still! that's awesome to know that they will give out render time!
Great job guys!
@mav:
i guess you don't want to see your files in public, because they're for commercial use. but if so, you hopefully have a budget to rent a proper renderfarm (or have your own).
i think it's a great opportunity for hobby-artists to test a farm for free.
mav, that's not true. Where did you get that idea? Rendering on this service does not release your work to the public domain. They stated very specifically on their homepage that they respect your copyright and do everything they can to keep their service secure.
I tried it out, and it seems to work! After I submitted a job, it was running moments later and the render was complete very quickly. I'll definitely be using this service in the future.
It would be good to see Yafaray supported. Although blender internal is not exactly fast it doesn't do GI and where render farms are an added benefit to ordinary desktop owners is that they can deliver amongst other things GI renders in a far shorter time frame and Yafaray, I am finding is a rather nice render engine. :-)
Would it be possible to support yafaray?
O_0
C_C
P_P
6_6
9_9
The website is down (from this side of the ocean)
Can i render images or it only for animation ?
Very well. But I see everyone rush on these computers. Then rendering will likely be slower than on your own PC…
Are they intending for people to be able to use this long term?
@yellow
I thought that Blender Internal IS fast…
I am just hoping that they will release a nice tutorial on how to make one for my self :P
just submitted a simple job with lots of ray transparency and reflections :).
Nice!
Will this feature be available in the future?
I didnt tested yet!
Yes, free render time is like "wow" and "WOW!!!" :)
@Bart - how long do you think they will keep this free Blender rendering offer open? Some open projects would benefit from this resource. But even if it doesn't stay active for free use I think that as open source people we can, and probably will, choose to shovel the money out to a render farm when it's time to render our projects - that's what we do: find a way through every mountain. But I do hope it stays active. That would be sweet to render my stuff there. ;D
Ciao!
-b
Well I couldn't get it working after a few atempts :(
Just getting errors.
Would be nice for a quicky tut for how it works!
I did submit and render a job quite quick. Nice experience.
However, now it always fails I think they got their wanted load and experience. I guess someone will fix it next week ;-)
Carsten
Dear All,
First of all, thanks for the interest in the portal, we appreciate it!
As you can see we had problems. Turns out it was just a silly little thing. It should be fixed now. I just tried and it works.
However I'm afraid running jobs have to be resubmitted :(
Check it out!
@Yellow: If I get time allocated and you guys help me out with advice, sure. That's kind of why we started this.
@BLENDER_BOY: I know I know… Can you give it a try now?
@Orbit: I think you can render a single frame. But if you have a Core Duo or similar it will probably render faster on your desktop.
@Lancer: I hope so.
@Grzesiek: That is possible, yes.
@jahmaica: Yes, Go give it a try!
@banor: Obviously the dean has to see the benefit. That is where you guys come in: we would like to hear from you! We need feedback to improve it. Then we have to prove that the community actually needs this and it is worthwhile for the university to keep it running and for us to work on it.
@Jj chilidz The user manual is under the help tab. I think we can prepare a technical paper as well. I'll ask.
Zsolt
@Zsolt:
Thanks for your reply…I am still having the same error.
I can connect to the site but shortly after I do it displays the following error message
Look:
http://i38.tinypic.com/15o6qmd.jpg
I mean the website is reachable by my browser. There has to be something inside the portal itself that is causing the trouble.
this is probably a limited time action, isn't it?
This is brilliant especially if they keep it going ;)
@BLENDER BOY!
Can you retry please? Are you behind some kind of proxy? does it block access to port 8080 or something?
Well it worked great for me , very fast, but a few problems.
One I dont know what version of blender its running, which isn't too much of a problem, but the by far the biggest problem , it renders an avi image sequence, and the avi movie wont play in quicktime. I'm on a Mac and can find no way of putting the sequence together, except manually, which sort of makes it useless, unless anyone can help me there.
@jon: you can use either VLC to play it, or if you really need Quicktime use the Perian codecs for Quicktime - http://perian.org/ . Worked great for me!
I have tried several times now.
It seems to do its job just fine
but than deletes all the files due to "exceeding user limit".
Could you tell what these limits are exactly besides the 600 frames limit
and the 100MB space limit
(just for uploaded blends? or also for the rendered output?)
also do you know if Rendering and compositing with FSA and the temp .exr files works?
and which Blender Version is supported?
But aside all those Questions
Thank you very much for your kind offer.
Maybe I get to render some of my stuff I stopped working on due to too large render times.
Zsolt
In case you get time to add Yafaray. :-)
Is it Linux on your cluster?, if so what distro?, if precompiled binaries are not available then perhaps I could compile for you, so that it would just be the install across your grid with binaries and python scripts left for you to do?
vlc may play it but I need a movie file rather than a sequence so I can use it in after effects or other compositing programes.
I would need to put it all together by hand, which is very time consuming.
@Jon: I think that with Perian, you should be able to read in .avi files as well. Haven't tried it, but I've had it on my system for years and I've always comfortably played and worked with both .avi and .mov files here.
actually vlc is not playing it either
Thanks for the help bart, i've installed Perian and quicktime will play the online compressed avi online file you get with your render which is very poor quality, but will not play the image sequence.
I dont what to sound like I'm complaining after all its free, I just frustrated, it took me 7 hours to render a sequence of a movie I'm making on my Mac and only 10 minutes in the render farm so I was really excited, but I've been searching all afternoon to be able to play the avi sequence and I cant find a way.
If they're going to output a sequence wouldn't a jpg or png sequenece make more sense as most compositing programes can import these files and order them into a movie.
I'd definately use this service. But I have a problem, I'm still a 3D Noob and don't know how to save all my external textures and materials in 1 single .blend file. I've tried it before and took my .blend file to my friend's computer and when I tried to render my scene almost everything came out black with no textures.
Does the disk quota refer to the size of the output files, the .blend file, or both ? I have a 200mb file I'd like to animate, it's too big for my PC to handle.
@all: our friend NextGens from CPC fixed the bug this morning. Say hello to him :)
@BLENDER_BOY: I definitely want to sort this out for you. There is a redirection to https before the welcome page. Maybe a firewall issue or some security setting that kicks in. Have you tried going to https://rendering-portal.cpc.wmin.ac.uk:8080/gridsphere/gridsphere directly?
@jon The avi is just a convenience, There is a zipped jpg sequence of course. Click on the green things in the output column on the manage rendering jobs tab! The manual is under the help tab. It's also accessible through a link on the welcome page without logging in.
@marijn @Jonathan: Depends on you actually. The louder you are the easier it will be for us to convince those who make decision about our resources. So keep using it and let us hear your voice :)
@jon, Bart: Yea, the video is dodgy. There was talk about setting up an rtsp server or at east tweaking the servlet to force the proper mime type. We'll discuss this issue on Monday.
@testscreenings @Rhysy: blend only I think. What's your portal usernames? I'll increase your quota to 300MB for a while. The portal box has 160GB total at the moment but I think I can be bold enough to ask for a bigger disk tomorrow. Looks like Monday will be busy :)
@yellow: Yes, The cluster runs SLES10 SP1. The portal machine is a separate box which I'd like to upgrade now that there is interest. I prefer Ubuntu, NextGens is a Debian guy, and CentOS is nice too… Yafray would have to be compiled for the cluster but that's no problem. I'll investigate it when I get the chance. Have to finish another project and get time allocated. That's how it works.
@BigBoy: I guess this is a good place to ask. Unfortunately we at CPC know virtually nothing about using Blender.
Also, please keep posting bugs and asking for enhancements! This is important for us to improve the portal!
Does it render particles? Because i tried, so is it just me?
my portalusername is also testscreenings.
That would be very nice.
I´ll try some more.
And thanks again :)
Wow, this is too good to be true. I'm going to be using this for my CgTalk entry. This is awesome. I'm off to see if I can get something running.
Off the top of my head:
* When a job is submitted, display its position in the que.
* Add an option to receive an email when job is completed.
Okay, I've submitted two jobs, (1) a 120 frame animation, (2) a 1600×1040 image.
Both failed with this message:
"Submitting workflow to DAGMan failed, error message of command "condor_submit_dag BlueBalls.dag" follows:
ERROR: failed to transfer executable file /home/pgportal/pgportal/condor/bin/condor_dagman"
Zsolt : Wow thanks ! My portal name is Rhysy.
@Alex: our Blender is version 2.45. Is that helpful?
@Twinsnakes: I have added the feature requests. Submission fails at the moment. We are working on it.
@testscreenings @Rhysy: Your disk quota should be 300MB now
Many thanks Zsolt. I actually managed to get the file size down to 65 mb - I did try this before but it seems Windows wasn't reporting the file size correctly. Can't wait to try this out !
Submission should be fixed now; it works for me at least :)
Bender 2.45 is not the latest (2.48a is what everyone is using ATM)
Also, fluid and cloth simulation animations can not be rendered because the baked movement data of the fluid/cloth is not stored in the .blend file itself, but it is stored separately in the /temp folder.
This means you need to add the feature to bake the fluid/cloth before rendering, otherwise it will be motionless frames and nobody can use it for that purpose.
Thanks for your support.
Thanks for the reply!
However the project i wanted to render with particles was made in version 2.48.
maybe you guys could upgrade to 2.48 also?
Anyway, still a great service!!
Fair enough; I've just upgraded the version of blender we use to 2.48a.
I hope I haven't created too much disruption on running jobs doing so :/
Enjoy!
Thanks!!!!
I hope you keep this service going, it really is fast and easy!
Thanks for the answer Zsolt,
I got an avi sequence, not a jpg seqeunce the first time I tried, my next render produced no output file and no error messages.
hope that helps.
I have try it with a 90megs .blend but it says: erros, not a blend file. :(
I submitted OK, but I get the error message :
zip warning: name not matched: frames
zip error: Nothing to do! (try: zip -1 -r frames.zip . -i frames)
When I hit visualise it looks like it's still doing something, though it says finished in the main window. Should I just try again ? I know the .blend is fine.
It works now. I did 120 frames in about 3 minutes with maxed out settings.
@jon I don't really know Blender but there seems to be a setting that creates jpg. We got this avi sequence with a few public .blend files we tried, and jpg with others. Maybe somebody here can help you out.
@endi: Which version of Blender are you using?
@Rhysy: Please try again, We had to restart the portal and that messes up running jobs. Hopefully it will work. If not I'll look into it tomorrow.
@Twinsnakes: Great! Cold you share a rough performance comparison?
hello
when i pressed the download avi button in the render manager, it only downloads a blank .avi movie
what should i do?
Very great, it's work fine and quickly. But few font don't render, with the thru font.
Cheers Zsolt : it seems to be working now. I'll check back tomorrow to see how it goes.
Zsolt, I use the latest version. But I think my blend must work on 2.45. Maybe.
I get the following error in the logs:
=>> PBS: job killed: walltime 302 exceeded limit 300
Running version 2.48a… a 7M file… I am able to render at half the size on my laptop.
Props to the portal progenitors, persevering, with perspicacity!
Just tried it again, with a file that failed on the weekend. I realised it had a cloth modifier on it, so I exported the cloth mesh as a point cache (.pc2), removed the cloth modifier, re imported the point cache and re saved. It worked fine when I uploaded it again. So, that's one workaround if you want to render cloth. It's pretty damn fast! I hope this service stays around.
ATM the management Tab only shows a help message for me. So I cant check my tasks.
I did subscribe to the mailing list, I think we should not use this comments for error messages.
Carsten
Ermm, erm. I TRY to subscribe…
- Results:
Invalid confirmation string. Note that confirmation strings expire
approximately 3 days after the initial subscription request. If your
confirmation has expired, please try to re-submit your original request or
message.
Carsten
If you experience strange things, try to log out and log in again!
;-) Carsten
A few frames rendered fine for me, everything looks as it should. Unfortunately I get the error message "122 frame(s) killed due to exceeding user limits" - not sure what those limits are !
@Zsolt:
First, thanks for offering this service and putting time and resources into it and Blender!
Second: I was able to successfully submit and get results for two rendering jobs:
however: my results come out as a bunch of .avi files ( I guess I will check my blender file and see if changing the default output to something like png or jpg yields different results).
Third: If at all possible, support for fluids would be awesome! I'm guessing that there are other things as well that require baking that might not work either but I'd think others here would be able to point to those better than I. I've been trying to find a nice distributed renderer that would handle fluids for a while and haven't found anything simple enough ( I think I had it working once with Yadra but I think I just copied all the fluidsim data to the /tmp directory where each node was expecting the data to be but I don't remember for sure)
thanks, again!!
Dear All,
We spent today on fixing some portal related issues on the cluster and eliminated a few bugs the portal code.
As you know job submission is broken. The service is quite popular and the disk on the portal machine is full. We had a discussion and decided to allow you to use our cluster filesystem. This means 2 things: up to 11TB of storage and 1.2 Gb/s throughput. :)
We had to take the portal down to migrate it to a cluster node. Since it was unusable anyway we decided to take it down without an announcement. This leaves a bad taste in my mouth but it will make migration much faster. Sorry for the inconvenience! We are working on it right now.
For all those who I have yet to reply to: I'll do my best to reply as soon as possible but getting the portal up again has to take priority!
Zsolt
Well with all that processing power, isn't it a shame that blender is still single threaded when baking? My first thought was lets crank up the resolution insanely high and do some awesome water shots - baking is still single threaded and so it would still take a few days just to bake.
I guess you could break down a large complex fluid simulation into a number of jobs each with only tens of frames. Then have the system's job management and LSF parcel the jobs out to individual cluster nodes (or node pools, depending on the cluster).
I strikes me the Blender community could use a grid application similar to SETI's seti@home BOINC application. I'd be happy to share my spare CPU cycles with fellow blender. (Yeah - right like we have spare CPU cycles…..)
thx for the head up bart, might be good if they supported this branch of blender wich integrates kerkythea, most of the lighting models take ages to render .
http://cobalt3d.free.fr/images_3dblender/kerkythea/documentation/Blender2Kerkythea_en.htm
doing it on a renderfarm will make them doable. if they get this working you can forget yafray ;)
uploaded a fitness room full of mirrors let see how the britsh cope with that lol, I dont see any indication of how long it will take? I better check tomorrow.
Hi Zsolt,
I'm getting the same problem as "testscreenings", when my render finish i get this error: "X frame(s) killed due to exceeding user limits". What does it mean? I uploaded a 49Mb file, with 560 frames.
Thanks for the help, and for this great project of you =)
Icedeb, I think the frame limit is supposed to be 500. I've just realised I've been trying to render 501 frames, not 500, which is probably why my animation keeps failing. :)
@Rhysy: but i tryed to render only 20 frames too, and got the same problem =\
the idea of this project is great, but it will take some time to make it really good and have no bugs.
**(sorry for the bad english ^^)
Dear All,
We are up and running again. But I'm sure you know that already. We didn't even have to announce anything, you rushed to the portal when it was restarted :)
The portal was moved to a new machine last night and uses the cluster filesystem now. This improved job submission considerably. We've updated the welcome page with useful information.
Workflow visualization is broken. Fixing it is a low priority at the moment.
Much more important is the quota management. We are working on fixing this issue.
@Icedeb: I have added useful information on the welcome page.
@ysvry, @Icedeb: The walltime is 5 minutes. My guess is that a it takes too long to render, hits the wall and fails. Also, make sure it is possible to render the number of frames you want from your .blend file.
@Rhysy: Your frame limit is also the default end frame you can see on the 'New rendering job' tab. This should be 600.
Zsolt
any one worked out how to get jpgs as the output files?
i tried 3 times, only got black screens back, i try again at 3 times reflection.
@ elwol , better make a multiplayer game that plays in blender and let the player share their free cpu cycles. something along the lines of the sims in space lol.
Awesome, it worked ! I think I was probably hitting the 5 minute per render wall, it now works perfeclty. This probably would have taken 12 hours or more on my own PC, it was done in less than 2 thanks to this (I went away for a while so I couldn't time it exactly).
Many thanks !
seem to get some non black jpgs now lowered the revlection counter to 3. Will gradually up it see where the limit is.
I think this is great too can this be inbuild in blender so people allowing the feature will automatically form a render matrix? that would be uber cool.
thx for the good work will this stay free or will it go comercial? and what will the prizes then be?
another question could you make the wall time inversley related to the number of rames you render so say if you render 300 frames you can have a frame render for 10 min instead of 5 for the 600. Think it will result in the same processor usage per person, right?
i have use it and i think it s great…tried particles and some simple animation and it render out OK.
TKS
@Zsolt: thanks for the help. But how can i know if it will take more or less then 5 minutes? I tried to render a file with 1 frame and got the same problem…
Hi Zsolt, i changed my file a little to make it faster to render. But now i get this error:
cat: /tmp/tmp.rHvHas5956/*.stderr: No such file or directory
zip warning: name not matched: frames
zip error: Nothing to do! (try: zip -1 -r frames.zip . -i frames
I don't know what this mean, Is it because of my file again? If it is, how can i fix it?
Thx again =)
Same here….
ERR log of: SchPAL600 / Frames_451-600
cat: /tmp/tmp.DYLUF24510/*.stderr: No such file or directory
zip warning: name not matched: frames
zip error: Nothing to do! (try: zip -1 -r frames.zip . -i frames)
Carsten
Same for me more or less, with the same file as previously (rendered at a higher resolution but still below the 5 min limit).
@Rhysy - Dec 1st, 2008 at 6:35 pm
May