In real life, a shutter will add strange effects to fast moving objects. Sergey is now working on emulating this in cycles - in the picture above, the pole is perfectly vertical but moving from the left to the right.
Adaptive Samples writes:
Sergey is working on another cool motion-blur-related(ish) feature: The rolling shutter effect, which you find in most photo and video cameras these days.
This is presumably to help with integrating renders with fast moving camera footage, but of course you can use it for whatever crazy purpose you desire ;)
It’s not in master just yet, but there’s a solid patch awaiting review.
25 Comments
Oh this could be incredibly great! All this time I'd been worrying about finding ways to remove rolling shutter in video (I've found no good ways BTW) when it hadn't even occurred to me to think about adding it to the generated elements to match! That's genius! I do a lot of match-moving and other integration with video footage, so I'm always excited to see advances that could help with that! Fantastic!
There really shoul be a hint to surely follow the link as the linked slowmo Guys Video on that page is just so awesome! Just a link and some text might be not interesting enough as the detailled description on this page deserves being mentioned ;-)
Yeah but then we're still missing to have the rolling shutter in count when matchmoving, right?
cool feature!, another one to increase the plausability of mixing 3D with real footage would be to add a way to select the chroma subsampling 4:2:0, 4:2:2, or 4:4:4 wich is the way most cameras in real life work (https://en.wikipedia.org/wiki/Chroma_subsampling)
cool feature!, another one to increase the plausability of mixing 3D with real footage would be to add a way to select the chroma subsampling 4:2:0, 4:2:2, or 4:4:4 wich is the way most cameras in real life work
Hm, not so sure about the displayed effect.
Because in generally i dont want camera's to behave like this. (by default disable please)
But if things like that do happen occasionally then the cube as shown above wouldnt be normally blender sharp.
The effect happens if a camera scanline moves to the next line and within that time the object has moved.
For a lot of camera's this isnt a fixed time either often shutter time p frame varies on light conditions too.
Even when a camera is set to 25p or 50p or 60p, it doesn't mean it actually uses that, often the time is shorter.
The new frames then are taken at 25framespsec, but camera shutter open/close happens faster.
Thereby a camera's can keep up with light conditions without changing diaphragm (wich would change DOF too).
So then we need something in Blender to also calculate the real time used in the shotage
(and not from some studio condition), i dont know if such software exists.
But if we assume its static say 25frames p /sec or 60frames p/sec then since its fast moving it should be blurred as well.
Hmm, what you say makes sense. I could see RAW footage embedding that level of metadata, but I doubt any camera returning RAW would have issues with rolling shutter.
Great addition. I trust it will have a full set of controls to match and surpass any physical camera. Can this be ported to the compositor too? Equally interesting to have would be a full set of pin cushion and barrel distortion rendering capabilities to match. I'm sure some of these tools are in the compositor, but if this is in the renderer, I'd expect some of this capability there too.
Isn't it strange that software developers everywhere create effects for us to mimic all the physical limitations of real equipment? Rolling shutter, film grain, film colour casts, lens flare, lens vignetting, motion blur, and arguably depth of field too? And then somebody just suggested barrel and pincushion distortion too!
Makes you wonder why anybody wants OLED 4K 60fps TVs, because all we're going to put on them is stuff that looks like it was made on old vintage kit. Odd!
That reminds me of why people still often use 24fps, for its "cinematic" experience.
While like you said they actually use a highdef screen at home lol :))
You're right it is a kind of crazy situation, but the flipside is true also; a stone sculptor could spend many hours trying to sculpt a perfect cube, struggling with getting rid of all the little bumps and defects to make it look better, and if the result was accurate enough it would be pretty impressive. Whereas we can conjure a cube in Blender and everyone would say "Why would you want that? Add a lot of bumps and defects to make it look better!" Perhaps a clumsy example, but I guess my point is the grass is always greener. :)
Perfection is unreal. humans have an innate ability to sense when something isn't real.
Some drum machines have a random offset to the beat to counteract that. While you could argue that rolling shutter is an unreal effect, we have a lifetime of experience with the effect, so we know what we are looking at when we see it.
"we have a lifetime of experience with the effect,..."
Yes we do, because we grew up in the era of seeing imperfections. But now we're deliberately and unnecessarily keeping things imperfect, when we should be getting used to better things. Why should a modern child watching TV be subjected to imperfections from yesteryear? It's not their 'normal'. They could be watching TV / film shot with a global shutter, zero film grain, accurately corrected colour / white balance, oversized lenses that cause no vignette... Hey, perhaps we should go a bit further back and start adding more artificial scratches, wobbly framing and irregular speed playback so we can look a little more 1930s?
Well, when I think of rolling shutter I mainly think of modern cell phone cameras.
The Drum machine example is a great one, because mainly they're aren't re-engineering a technical limitation but rather mimicking human imperfection. I think what modern technology allows us to do is not only create imperfection, but CREATE imperfection. Film grain looks pretty in some cases, oil paint on silk is a lovely look, and there's just something about a weathered and rustic piece of wood. As artists we are no longer bound to one film grain within the camera, we can choose where the individual grains go if we wanted to micromanage it to that level of detail. Digital will preserve the imperfections we create, perfectly. We can also create effects reminiscent of wood grain or oil paint. And yes we can make an animation where a character is taking a video on her cellphone, and that video has an 'accurate' rolling shutter.
"Yes we do, because we grew up in the era of seeing imperfections" - do you think we grew out of this era?
Look at almost every interior architecture render (or photo) you have ever seen. Why don't they bother to make the bed? We have the technology to create renders of beds where they actually straighten out the pillows and bed spread, but it just looks better with a rumpled blanket draped over a corner. it's more organic. when everything is crisp and hard and sharp and perfect, it just looks out of place, like it doesn't fit in this imperfect world we live in.
For the specific example of rolling shutter, I agree that it is a little silly to be putting effort into reproducing technical limitations. but it's a part of our cultural inheritance and it formed the way we envision something moving fast. (see here: http://www.photoplaza.nl/lindolfi/Lartigue1.jpg and see here: http://www.pinkelephanttraining.co.uk/sitedata/117/wacky-races-characters.gif)
I think pincushion distortion is just a side effect of using an equirectangular lens no ?
no, pincussion is an effect that happens with real camera lense at a high telephoto zoom: http://www.dpreview.com/glossary/optical/pincushion-distortion
Can this be used to also fix/remove rolling shutter to make tracking easier ?
No, this is just to reproduce the effect in rendered scenes. In order to reverse rolling shutter, you need to know the exact motion of everything in the shot. you can do some basic full-frame compensation for camera motion, but it's very difficult to compensate for object motion.
The worst update i have ever seen in cycles! In real life, we pay more money to buy a global shutter camera(like bmcc, red) to reduce the jello effect. And today we try our best to get back the jello effect in cg camera....
The lens distortion node has the same "problem". But you just need exactly that when you deal with problematic footage (as most of us will because global shutter sensors are just too expensive). So it's a great feature to be able to correct the perfect CG elements to fit with the imperfect footage.
The worst update of cycles i have ever seen! In real life we pay for a more expansive global shutter camera(such as bmcc and red) to reduce the artifacts(jello effect) due to the design of Cmos. And from now on we can bring back this artifacts to our cg camera. What a "Great" feature……totally useless……
Not everyone can afford access to a high end global shutter camera, so if your baseplate footage has some wobble to it from a cmos camera, it's jarringly obvious when you composite in cg elements that are perfectly straight.
I'm a little surprised by the negative reaction from so many people. It's just an option. Like dithering, or rendering at 10 fps, or setting the resolution to 160x240 , or motion blur, or compositing lens distortion, or rendering in black and white.
Obviously you wouldn't use it on an entirely CG shot but if you're trying to composite CG into shaky DSLR footage this could be very useful. No one will force you to use it.
This is a great addition, but surely will not be the default.
I work on both the production and post-production end of independent films, and we are often using CMOS cameras without a global shutter. I use SynthEyes tracking software, which has a great calculation method for determining rolling shutter and then generating camera matchmove data for Blender with the rolling shutter compensated, and now having the ability to match the calculated rolling shutter in Blender means even more accuracy for my composited images.
Kudos to Sergey for tackling this! (Naturally, if they have this enabled by default, I may retract this statement. *grin*)