Starting to play with Corrective Blendshapes

I started to play with "corrective blendshapes" concept (shall we call it cBS ?). The idea of cBS is to perform minor correction on the skinned mesh so that the artifacts coming from skinning are corrected when the character reached a key position.

If you messed around Maya/Blender and skinning, you know that painting weights have limits : after a while you will simply consider you can't make a better weight-paint.

In some situation, even the best painting of weights cannot lead to a good result. I don't even mention the fact that "skinning" doesn't take into accound the muscles; volume conservation and other things.

As far as I think, the cBS approach is a good solution. I particularly like the fact that artists still have control over the result : rather than relying on physics to compute the volume conservation or other crazy properties, the final mesh simply comes from some additional artwork from artists. So it is easy to aschieve what we really want to see.

The main issue with Corrective BS comes from the fact that the Blendshapes are applied after the skeleton got transformed and the skinning computed. The whole challenge is to compute the shape of the Blendshape at rest pose.

I wrote a plugin for Maya that is exatly doing this Job. Here is the current workflow :

1. Put the character in the position you want
2. Take a snapshot of this mesh in this position
3. Export this mesh (as obj format, this is good enough) so you can rework it in your favorite modeler (ZBrush is really good for that)
4. In ZBrush, for example, modify this mesh by working the clay so you do compensate the artifacts around the joints; the inter-penetration of triangles etc.
5. Export from ZBrush this new mesh and import it back to your Maya scene
6. invoke the plugin by specifying this corrected mesh : the plugin will compute the BS at rest pose; add this BS to the current frame with weight = 1. On the screen, you'll see exactly what you modified in ZBrush. Except that this time it is fully part of the animation

Of course, this correction is only valid for the character position that you worked on. For other positions, you will set the weight of this BS to 0. In order to get sonsistent animation, you just need to keyframe this weight so that it slightly goes to 1 when the specific position is reached.

My goal in this project is to find a way to perform these corrections in realtime. You can indeed imagine that a lot of small cBS be available. The challenge would be to find the best match that would correct the current animated character. I do have some ideas on the topic...

Before correction (original skinned mesh):

After correction:

Antother example :

after:

Finally, this method is also interesting to add some sort of "Dramatic Emphasis" to the motion. For example, I tried to make the forward foot bigger than expected, like the Cartoon style "squash&Stretch". This emphasis are driven in the same manner : applying the proper cBS at the proper time...

To be continued.