New Cycles benchmark: BlenchMark
Mark van Dijk wrote a new Cycles benchmarker that can automatically upload test results to his website to compare them to others.
A few weeks ago I saw some screenshots from old Blender versions. I noticed a button called “benchmark”, but when i searched for it in my own copy of blender (v2.70a) I couldn’t find it. I went searching for a benchmark addon, but couldn’t find it either. I was looking for a project to learn python, so why not make that addon myself? I started right away, things went smoothly and after some testing I decided last weekend to go public.I call the addon BlenchMark.It’s a benchmark for Blender, and my name is Mark (that’s why the M is capitalized). It let’s you run a benchmark, see your results and upload them to the webiste I made for it. On the website you can see how you did in comparison with other renderdevices. If you run this benchmark in every new Blender version it becomes visible if Cycles is getting faster or not. There are some other useful statistics too.
The website is running at a subdomain of mine. When this addon really gets used by a number of blender users, I will move the website to its own domain. The addon is available for download at the website.
I’d like to thank the Dutch Blender community members for helping me with their testing of this addon. It really helped me getting this done.
I hope to see your benchmarks soon!
Mark
Link
Mark a nice addon and website.
Now I can see what a system and vidiokaart works better with Blender.
Thanks!
It doesn’t seem to send the numbers of GPUs being used. Am I wrong?
Nope,
You are not wrong. At first I decided to not implement that. Since yesterday I received that many request I will try to implement it asap.
Does this work with Blender 2.70a? I keep getting the error: “Auto-run disabled: Driver ‘-100*power'”
That’s because there are some drivers in the scene. They are disabled because of a securtiy trigger in Blender. It won’t affect the addon. It should work anyway.
I had the same error. Try:
User Preferences -> file -> enable ‘Auto Run Python Scripts’
(I exluded my Download folder)
Save User Settings.
Worked for me ;)
Tried to enable the add-on on blender 2.71 but couldn’t find in on the list, copying the .py file to addons folder or installing it from user preferences panel. What is the add-on name on Blender and is there a reason for me not to find it?
I had the same problem. Discovered it in the “disabled” menu. The reason I didn’t find it: it’s called bLenchmark and not benchmark.
Yep,
It is called Blenchmark. Thanks for the clarification jfeoöjföoeijf!
Wow,
In 6 hours after this article posting I received over 200 benchmarks, comments everywhere and someone even took the time to make a tutorial on Youtube!
I never expected that kind of a response. So to all of you: Thank you! you made my day! :-)
This is a huge motivation to improve this addon, and I will do so!
Thanks again!
Mark
No, thank you! It was sorely needed :)
Just shout if you need some help, it’ll eventually become a burden to maintain I’m sure.
Oh and a suggestion for the future – would be cool to have a couple different scenes so we can see if there’s any difference between different kinds of rendering (i.e. fur, sss, transparency, volumes, interiors…)
Thank you,
I’ll shout if I need help.
I have had some other suggestions too. I’ll list them on the site, and surely consider implementing some of them.
Mark
Excellent, glad to hear it! What’s the link of the YouTube video?
here you go: https://www.youtube.com/watch?v=urh2tIjEcjI
Hi Mark, I found out that your precondition for not changing the settings of the core benchmark file can be circumvented without the user knowing. I ran the benchmark on my GTX670 with the standard settings ending at 1:34 render time and uploaded to the site. The standard tile size was pretty small for cycles, so I tested what an optimized render tile size would deliver. Changing it to 256×256 brought me up to 1:03 and this reminded me of the auto tile size add-on for Cycles. So I reinstalled that add-on and I started the blender fresh and added the blenchmark add-on and did not change any settings – and I ended at 1:05. If people run auto-tile size add-on with autosizing as default or 256×256 as default in the add-on, they will get benchmarks up to 30% better as I got.
While it is of course nice to have such performance boost, it can significantly dilute the real benchmark as conditions are not the same and people do not know and upload results and other wonder why their cards are slower than others cards in same set-up :-). I have no idea if this is a real problem and also not how to prevent it, but I thought I flag it to you.
Great project, keep it coming :-))
Hi,
I noticed this as well, but more in the case that users changed these settings over a set of uploads, despite that the fact that it is not permitted. On the website I see the same card getting faster and faster. In my head I allready bugfixed it (reset all those settings as soon as the hit “Benchmark”. Still have to implement it. :-)
Thanks for you reply!
Mark
The Auto Tile Size add-on, if enabled, will automatically optimize your tile size without you needing to do anything about it. This means overwriting the set tile size in the blend file for the benchmark.
Best thing to do would be to check what the current tile size is, and not allow them to submit the benchmark if it’s different from what you originally set it. Or, whatever data is sent to the server, include the tile size and reject the submission if it’s wrong. This could be done for any other render settings too, though I don’t think people would “cheat” just to improve their hardware’s benchmark.
Hi Greg,
I understand what this addon does, and will have to figure out how it works exactly in order to let those addons co-exist.
Regarding the cheating, well, don’t get me started. :-)
99% of the people is not.
Just run this before your script sets the tile size:
try:
context.scene.TileSizeEnable = False
except:
print (“Auto Tile Size is not installed”)
This is a great tool, but we need to see more data to compare setups. Can you add some pages to your site that graph all the GPUs on one page, and all the CPUs on another page, a bit like tomshardware benchmarks?
Thank you for your suggestion. I think I’ll make some additional graphs for that!
Mark
I’ve released a new version (1.0.2). Fixed several issues.
For details and the new download: mad-creations.nl/blenchmark
Kind regards,