Weekly – June 8

by - 2012/06/09 57 Comments Development, Weekly meeting

The previous blog posts already show our weekly progress well, so here’s some additions to this – like an evidence we have a full shotlist now!

This weekly I spoiled the fun a bit by ringing the alarm bells; if we want 9 minutes of film to be done in 10 weeks, we need to start delivering finals now… and not only a few shots, but more like 10-15 shots per week, 50 seconds every week! The two main bottlenecks to overcome urgently;

– Mastering Cycles renders for big environments (too slow, needs optimizing or baking or just use more background mattes & compo trix)
– Cleaning footage (chroma key, mask out issues)

We’re already putting the brains of several developers together on better keying algorithms. If you like to check on the issue, get the footage from the post below, and check on the blurred stuff especially (like from hair of people in back). How to get this keyed best? Or is just good old handwork with feather masks and painting skills?

/production/first-original-footage-frames-in-4k/

Examples of – quite good already – results from Sergey you can find as the bottom two pictures in the gallery above. Sergey continues working on it, especially to explore ways to smoothen edges in keymattes with keeping good alpha-over properties. I think we need simply something like a “blur inside” node… we’ll see!

-Ton-

  1. Dusty says:

    Would love to see a second or two for fully keyed frames with a background to see what the results are so far.

    Stills look more composed than when things are in motion and moving faster than the mind can pull apart a frame.

    • ton says:

      That’s a good point, but hard to define how much you can trick people… our trained eyes should accept it a bit, at least.

  2. Aaron says:

    I witnessed those bells being rung. :P I forsee some late nights around the corner!

    I hope you quickly find solutions. It’d be a huge shame to have to drop Cycles from production.
    With regards to cleaning up shots by hand; I suppose it may prove a little faster, depending on what needs to be done. Can’t expect a computer to do it all, unfortunately!

    All in all, everyone’s done an amazing job with improving Cycles, keying, and the film itself! So, again, it’d be a shame if thnigs have to be dropped, or cut short.

    My best wishes to you all during this hurdle!

  3. Sparse Voxel Octree says:

    – Cleaning footage (chrome key, mask out issues)-(seen on BA):

    http://inf.ufrgs.br/~eslgastal/SharedMatting/

    – Mastering Cycles renders for big environments

    http://sglab.kaist.ac.kr/CORR/

  4. Donnerland says:

    hm.. there a to be some DoF in the CG-Part

  5. Bao2 says:

    Ton, take a look to how primatte works:
    http://en.wikipedia.org/wiki/Primatte_chromakey_technology

    • Tycho says:

      Its me or someone suppressed the content of this pages today : last time i check there was a full explanation but now all is gone.

      • Tycho says:

        One don’t like mango and suppress the interesting parts. Old links before todays modif : http://en.wikipedia.org/w/index.php?title=Primatte_chromakey_technology&oldid=496775754

        • Sparse Voxel Octree says:

          Damn a conspiracy

        • guest says:

          What else could it be? But the one who didn’t like Mango also doesn’t like knowledge. Why would someone delete information like that?

        • Bao2 says:

          Saved to my hard disk. But I already understood fully the algorithm and I could rewrite how it works. There is really silly people trying to control the Earth these days…

      • Emu says:

        Seems that the article has been edited by a slightly privileged user because it violated Wikipedia’s rules. It was way too long and was more of a user’s manual than an article in encyclopedia.

        I can agree with that. On the other hand, now the shortened article is pretty useless. So if you people do understand the algorithm (not user interface) well, it would be very kind of you to rewrite it briefly in a pair of paragraphs. And the original article could be moved to Wikibooks.

        I’m sorry for being off-topic, but I just didn’t like this conspiracy theory.

        • Tycho says:

          But why the day bao says it two hours after ? If it was an other day ok but …
          I dont know its perhaps a coincidence.

          • Emu says:

            Well, we both see the same logs. But as I understand it, there first was somebody from 98.169.89.217 who just added a {{Howto}} template, meaning the article was badly written. He did so an hour after Bao’s post, so most probably after reading it. Then, a user called Asav noticed (in my opinion) this template and made the easiest fix to it, namely deleted most of the text.

  6. marcoG says:

    It’s time for Cycles to get bucket rendering implemented! ;)

    Joking aside, facing these problems will get Blender better and better, that’s Mango’s goal so…keep rockin’.

  7. Ace Dragon says:

    Well you do have Brecht mentioning that he is looking into smarter sampling methods and performance improvements for Cycles, so maybe that will help to an extent.

    Perhaps it will help when Cycles starts getting optimizations noise-aware sampling, SIMD, QBVH trees, importance sampling, light priorities, and enhanced solutions for reducing noise in Cycles like changing the glossy blur and clamping value on a per-material basis or even using a map that may or may not be automatically generated.

    • Ace Dragon says:

      The spot where I wrote SIMD is supposed to be ‘SIMD instructions’, in case anyone gets confused when seeing that.

      • Tycho says:

        Or subdivision surface that adapt the level based to the distance to the camera. Texture can be also shrieked if they are to far away. So a pre pass bake (or distil the scene to improve mesh and texture before cycles render it)

  8. Danny says:

    Have you guys tried the cmiVFX tutorial for greenscreens?
    http://www.youtube.com/watch?v=0Wyta-hkCmY

    • Danny says:

      Nevermind. Just find out that the tutorial is made by one of you Mango guys….sorry! :p

  9. Gez says:

    I think our keying tools are pretty capable already. I’d stop thinking about a silver-bullet keying node and try to improve other tools needed to create good mattes.
    Masks are essential to define regions. Looking at the samples you shared so far it looks like you tried to pull a single matte from plates, and that’s pretty useless because a good key for hair might not work for out-of-focus backgrounds, transparencies, etc. (maybe I’m wrong, I didn’t see any screenshot of the nodetrees involved)
    I think that just tweaking masks and keying nodes to provide a straight-forward way to combine mattes from regions would solve most of your keying problems.
    An extra problem is noise. I tried to pull mattes from your sample images, and I couldn’t get decent results without applying some denoising.
    Question is: is that noise part of the source material or is it exaggerated by an incorrect colorspace transform? I’m not sure about the answer, but I wouldn’t expect so much noise on a greenscreen shot by a capable DP with a kickass camera.
    Anyway, I wouldn’t stuff a denoise option in the keying nodes. Having specialized denoise nodes would be much better than a simple pre-blur (I’m thinking about wavelet denoising on separated channels, that would be awesome).
    IMO adressing those problems you’d be good to pull good mattes from your material.
    Oh, and a lightwrap node. Lightwrapping is already doable using existing nodes, but it would be very handy to have it in a single node.

  10. Rogério Perdiz says:

    Here goes my keying attempt on the image above:

    .blend:
    https://www.box.com/s/104edb022a1dc440cfdf

    Screenshot:
    http://img221.imageshack.us/img221/4818/rogperkeying.jpg

    At least for just this image… it’s not easy but it can be done :)

  11. guest says:

    Check our “Greenscreen Nightmare” on YouTube, this has helped me with a few really bad keys. I almost always do manual tweaking on my mattes. I had to crunch the red channel once to fix a watermelon helmet someone had on their head in front of a greenscreen.

    • guest says:

      Sorry, “check out”, not “check our”. It isn’t from me. It is from BlendedPlanet.

  12. Rogério Perdiz says:

    Hey, I’ve posted a Keying method (a few hours back) that perhaps gave a better result then the one you’ve shown. It went to the spam, when you can please give it a look and let me know how is looking, my monitor isn’t no

  13. young_voter says:

    So Cycles is not ready for primetime.

    • Tycho says:

      Dont desperate it will surely be ready :)

    • stephen says:

      Well, isn’t keying/roto/mattes more of a compositor/tracker issue rather than Cycles?

      Besides, Cycles is still being developed. Have a look at what it does so far and if it covers what you need it to, great, use it. If not, then don’t. “Primetime” is a bit vague…

  14. DavidT says:

    I tried using a typical three-pass key on the example above (using A003C014_120507_00500.png sRGB image from the 2K release). I got a little better results than the example in this post. You will notice the glasses and hair aren’t as crunchy and some of the facial hair shows up more. The top left corner just needs a small garbage mask.

    Here is the composite (https://skydrive.live.com/redir?resid=C0FF37B8A3617BDE!122) and the alpha (https://skydrive.live.com/redir?resid=C0FF37B8A3617BDE!121). The blend file is at http://www.pasteall.org/blend/14401.

    The node tree is more complex with the three-pass method, but it would be quick to use. You just need to add these nodes to a shot and tweak the three key nodes (maybe some of the other nodes for slight adjustments).

    I used Blender 2.63 revision 47635 to set this up.

  15. Andrius.B says:

    May I ask how long do you plan the final film to be? 3 min? 5 min? 10min?

    Just wondering.

    Personally, I think it is MUCH better to deliver a shorter film at higher quality. Quality over quantity any time :)

  16. 3pointedit says:

    You often need to shoot green screen components at a high frame rate, which makes for a strobe like fore ground element. Mostly terrible looking.
    I wonder if motion estimation, derived from the tracker (planar?), could impart information for a vector blur node? This would smooth movement and make the key edge more natural, if pulling a transparent edge is to hard.

  17. ton says:

    Here’s an example of a blur for alpha mattes; which should “shrink inside”:

    Original alpha:
    http://www.blender.org/bf/blur-inside1.png

    Blur-inside with size 30 pixels:
    http://www.blender.org/bf/blur-inside2.png

    Used as alpha-over:
    http://www.blender.org/bf/blur-inside3.png

    Obviously it should be masked to only do left side of picture, but it’s a test to show how alpha blur should behave.

    This is simply a couple of ops:

    – blur the alpha
    – subtract 0.9
    – clip with 0
    – multiply with 10
    – blur again

  18. guest says:

    guess there’s some source code for inspiration there:
    http://www.youtube.com/watch?v=kkQ1r5g49d4

  19. Ira Flowers says:

    A good ‘non-alpha’ trick to get hair detail for a key can be rather simple.

    First you check the different RGB channels to find out which channel gives you the best detail. If it’s blond hair usually the green channel is the best. If it’s dark hair the Blue channel tends to work well. But going to the green channel is usually the best bet for dark and light hair.

    Then you isolate the ‘green’ or ‘blue’ or ‘red’ channel and follow it with a gain and/or lift node. If you want blond hair you adjust the gain until the greenscreen is nearly all white. This will require adjustment because you can blow out the channel and lose detail.

    One you have have the ‘blond’ detail that you like add a garbage mask around the hair detail and then add a grade node and adjust the white point to the desired color for the hair. Then screen or add it via a comp to the background.

    Then comp the primary key over the ‘blond’ detail comp. Adjust the ‘blond’ comp (via transparency) to taste.

    If you want dark hair adjust the lift until you can clearly separate the dark hair from the background. Add a garbage mask around the hair detail you want. Then multiply it over the background via a comp.

    Then comp the primary key over the ‘dark’ detail comp. Adjust the ‘dark’ comp to taste.

    These non alpha tricks can be very helpful and will give you professional looking composites.

    Check out this video from FXGUIDE about comping and keying in Nuke and they go over similar principles.

    Non-alpha tricks are very useful and combined with good keying can really make the difference. I feel Blender has a good tool set to allow you to get professional keys. Just with a little bit of now how it can be done.

    Also, feathering can be a double edge sword and make your edges look like Vaseline. Be careful.

    Keep up the exciting work!

    Ira.

  20. Ira Flowers says:

    Here is the link I forgot to add to my post

    http://www.tinyurl.com/nuketricks

    • 3pointedit says:

      Thanks Ira that was very informative, I had not considered reconstruction transparent elements via screen or mult.

  21. 3pointedit says:

    What chance Mango use UV pass in compositor and add a cycles renders there?

  22. gillan says:

    for this music video I used in an intensive keying in Blender
    http://www.luceri.it/2012/05/blender-vfx-in-videoclip/
    here, where you can download the blend with my basic settings and with advanced color bleed
    I hope can be useful

    • 3pointedit says:

      Wow gillan that is a pretty good key on difficult hair. I will be interested to look at the .blend too.

    • Tycho says:

      I’m sure you can apply for helping them. Your movie clip looks cool btw and I like the compositing you make for mango. Just imagine your competence with greater tools than today.
      If you got time enough.

  23. Est says:

    The best chroma keys are done on layers, using localized settings for different areas of the images and using masks to layer them.
    I can recommend bayesian matting, too. There are better algorithms now, but for chroma-keying it has good quality. It’s not very difficult to program and quite efficient for large images.