BlenderNation

Advertisement

You're blocking ads, which pay for BlenderNation. Read about other ways to support us.

First Glimpse: Tomato Tracking Branch

47
By on Google Summer of Code

A first impression of the GSoC camera tracking by Sergey Sharybin is now available. The video shows how you add trackers and how Blender will do it's best to follow them through the footage. Awesome stuff!

Link:

About the Author

Avatar image for Bart Veldhuizen
Bart Veldhuizen

I have a LONG history with Blender - I wrote some of the earliest Blender tutorials, worked for Not a Number and helped run the crowdfunding campaign that open sourced Blender (the first one on the internet!). I founded BlenderNation in 2006 and have been editing it every single day since then ;-) I also run the Blender Artists forum and I'm Head of Community at Sketchfab.

47 Comments

  4. LswaN on

    Yes, this does indeed look amazing. Definitely looking forward to an export script or native control of the camera, so that we can put this feature to work!

    Reply
  5. Rio on

    This is looking very promising. However there is still one thing crucial thing missing that is required for succesfull matchmoving and that is the ability to enter the camera's film back as it will dramatically affect the camera's field of view.

    Take the movie he used to demonstrate for example. From the file naming convention i know it is shot on one of the canon DSLR's however i don't know which one, it could be the 5DMKII or the 7D. Big deal you'd think, a camera is a camera. Well it's not. The sensors in these camera's differ in size. the 5DMKII is a Full Frame sensor while the 7D has an APS-C sized sensor which is smaller.

    If he'd used the same say 50 mm lens on both camera's the field of view would be radically different. Compared to the 5DMKII the 7D looks like the lens used is much longer, about 80mm (50mm x 1.6) and thus your solve would be wrong.

    I hope this will be taken into account as it is something I have been missing a long time in blender and it keeps me from using it as a layout and prep program. This and also the lack of proper animated COLLADA import and export aswell as FBX import.

    Just my 2 cents, Keep it up!

    Reply
  6. BlenderBoy on

    Rio is right. Although possible to track w/o film back info. the track won't match exactly. This should be considered during further dev. Nevertheless, this is a welcomed addition to Blender... good job devs. I personally am a fan of SynthEyes; Russ Andersson's algorithm is much better than MatchMover (in my experience) and it's pretty inexpensive compared to other solutions out there.

    Reply
  8. D@nny on

    Looks very interesting. But for now it needs a whole lot of tweaking, but ik looks very promising!
    Keep up that great work, guys!

    Reply
  9. Potado on

    This looks so awesome! Now we don't need any of those third party programs to track for us.

    Will it be possible for you to export your tracking data to other programs like after effects also?

    Reply
  11. Chucker on

    @Potado, well there is a Export Camera and empties exporter available for Blender 2.5

    This is great, is there nothing Blender will not be able to do?

    Reply
  14. ibkanat on

    Dont worry the guys working on libmv and blender know what they are doing they will get it right(camera settings(flim back, etc)).

    Reply
  18. Keir Mierle on

    I'm the BDFL from libmv, which is the library Sergey is plugging to do the image processing. I wanted to emphasize to everyone that the first release will not be as fast or robust as SynthEyes or the other commercial tools, and will require manual calibration. We will continue to improve things, but the first release will be a basic tool that will work in the simple cases.

    A solid matchmoving pipeline is a big and complicated piece of software that will take some refinement before we get some of the better stuff. So please don't be too harsh when you first try it out in a month or so; realize that we are going to release early with something basic rather than wait indefinitely until it is perfect.

    Thanks for everyone's patience with this! It's been a long time coming.

    Reply
  21. Keir Mierle on

    BlenderNation ate my last post after I hit submit. What a waste of time. Here's approximately what I wrote:

    I am the lead developer of libmv, which is the library Sergey has plugged into Blender to do image analysis for matchmoving, in addition to camera solving. I just wanted to post on here to emphasize that the first release will be quite basic when compared to what you get with the big commercial packages like SynthEyes. In particular, there will be no rolling shutter correction, no autocalbration (you will have to print out a checkerboard and calibrate your camera), no autotracking (you will have to manually place and correct the trackers), and you will have to manually specify the 2 frames from which to start the 3D reconstruction.

    Starting out with the basics will let us get something to you guys faster, but please keep in mind that building a robust automatic matchmoving pipeline is a large and complicated task that will take time to get right. The first release will not be good enough to replace the commercial tools.

    Reply
  22. Dane on

    Alright :-) camera tracking, now all we need is a real open source video editor (light works has much promise, it is coming along beautifully in that direction) and someone could produce a live action video with some intense CG using entirely open source tools :-)

    of course as mentioned above it wont be as robust as some commercial tools, but you don't need that to get the job done, its just nice :-)

    Reply
  25. comeinandburn on

    Go Sergey ! Thanks for the update... this is going to be a godsend for doing VFX/3D integration using Opensource.

    Reply
  29. rorkimaru on

    @Dane Openshot is pretty good as a video editor. It doesn't have everything but between Blender and it you can get a lot done.

    Glad to see camera tracking is coming. I've a few projects I've kept on ice because camera tracking wasn't feasible so hurray for the future ^_^

    Reply
  30. Rio on

    @ejnersan: This looks very good, imho i think it should be integrated in the blender camera system ASAP as the lenses in blender don't make sense at all right now. This 'simple' fix should give blender a considerable boost for use as a previs, layout, matchmove tool.

    @Keir Mierle: The basics is all i need :) Advanced features are nice (see pftrack 2011 nodal workflow) but if we can't control our solves on a fundamental level the program is worthless to us. What I expect to see in all matchmoving programs is this (this is my own list btw i'm just a single matchmover):

    Tracker related:

    - Placing tracks manually
    - adjusting track pattern and search regions
    - offsetting the tracker pattern from the tracking point
    - manually set if the tracker has to take tracking pattern rotation, scale or skew in account
    - setting a pattern failure threshold.
    - which color channel to track
    - selecting different tracking algorithms if available
    (better speed, better accuracy, optical flow (i know is maybe a bit advanced))
    - centering the viewer on the tracker to be able to judge its quality.
    - the ability to enable/disable the tracker for different time regions
    - the ability to track 1 frame at a time. in the demonstration the tracking went faster than real time although it can be a nice time saver, it makes it very hard to judge a track and stopping it in time

    Solver related:

    - Setting the initial frames manually
    - Setting solve frame range
    - including/excluding tracks from the solve (should already be possible in the tracking stage but doing it here to will prevent tracking information loss)
    - setting camera filmback and focal length as well as letting the program figure it out the best it can
    - placing constraints on groups of trackers (planar and distance)
    - filtering out bad matching trackers by threshold and adjusting the solve accordingly

    I'm not sure which parts of this applies to blender and which to libmv.

    Sorry if I'm being redundant with this list. I'm sure you've got your own list and people working on it but I'd really like this to work so we can all expand our tool set. It's great what is being accomplished here. Please keep it up!

    Rio

    Reply
  31. Simon on

    @ejnersan:
    you are doing a great job! I can't understand why the devs are so hesitant on this. It is extremely important when working with film/photo material. When they start working on mango they'll soon realize that.

    Reply
  32. dave62 on

    @ejnersan good to know that someone keeps focus on this nice patch. i was very happy when i saw it some month ago at matts blog but i wondered when it get into trunk..
    even without any tracking stuff in blender it would be a great addition to proper combine cgi with real footage.

    keep up good work! thx

    Reply
  37. Nik on

    Wow hey, could marker tracking like this be used for facial motion capture? Like if it just creates a bunch of animated empties for each face marker... That would also help in cases where the camera is stationary, but there are moving objects in the scene.

    Reply
  39. Whimsy Collective on

    I wonder if it will support planar tracking and automatic tracking Libmv or PTAM (or PATM). Also, Adobe research labs at some Universities had something similar Libmv/Voodoo. By the way, have you seen Mocha for After Effects?

    Reply
  42. SavageCode on

    On video editors, VideoLAN is developing one, which I assume should be very good and solid, just like VLC, let's hope the project can go on until official release.

    Reply
  44. eonmach on

    Looks awesome, a 2D matching engine sounds like a big idea, but IMHO I think that a 3D matching utility is an entirely different beast.

    Reply

Leave A Reply

To add a profile picture to your message, register your email address with Gravatar.com. To protect your email address, create an account on BlenderNation and log in when posting a message.

Advertisement

×