Tuesday, July 19, 2011

Character Development Workshop, Part 3: Skinning and Facial Setup


When we last met Generikat he was rig-ready and geometry-ready, but the geometry was not yet bound to the rig. Let’s do that now.
Select the root-bone and shift-select the geometry and Skin→Bind Skin→Interactive Skin Bind – I used the default options.

Now a capsule appears around the root bone of your character. You should see “root” highlighted under the “influences” section of the tool settings window. The capsule gives you a visual representation of the influence envelop around each joint. Successively click through the influence list, selecting each bone and adjusting the capsules size so that they more tightly fit around each joint. Select the different parts of the capsule manipulator to resize in different ways, using the SHIFT key to constrain in different ways.

Note: with Reflection turned on you only need to adjust capsules for one side of the rig.
After you adjust the capsules, test the rig by moving various controls. Also, select the nose and the head joint and Skin→Bind Skin→Smooth Bind (options): Change Bind To to Selected Joints and bind.

To further refine the skin, select the geometry, make sure you are in Shading→Smooth Shade All mode and Renderer→Default Quality Rendering and go to Skin→Edit Smooth Skin→Paint Skin Weights Tool. Select joints on the influence list and paint the geometry to adjust the area of influence. For quick results, paint areas of influence with Paint Operation: Replace and a value of 1 (a pressure-sensitive tablet will vary the value) and then go back though with Paint Operation: Smooth to even out the falloff. Paint one half of the body, and then use Skin→Edit Smooth Skin→Mirror Skin Weights to copy the edits to the other half.

Tips: Click on a joint with MMB and use MMB to rotate the joints to test while using the paint skin weights tool. Click with LMB to switch back to the brush. Alternatively, set keyframes for various callisthenic poses throughout your timeline and paint on different frames to view the skin under different deformation conditions. There’s also a handy RMB-menu to use over joints while using the paint skin weights tool. The icons in the Paint Skin Weights tool settings window enable you to smooth selected vertex weights (weight hammer), copy and paste weights from vertex to vertex, and transfer vertices from influence to influence, all without leaving the tool! As a result of these updates to the tool, you will find less need to go into the Component Editor to adjust weights than in previous versions of Maya.
Now go into the attribute editor for the skinCluster node and under Smooth Skin Attributes change Skinning Method to Weight Blended. This will allow you to use the paint skin weights tool to paint in areas of dual quaternion (DQ) skinning. Basically dual quaternion holds volume around joints better, so you will paint the shoulders, elbows and knees with dual quaternion. By leaving the other areas alone we default to the classic linear skinning method (which most skinning artists find has more predictable results in non-joint areas). Go back into the Paint Skin Weights tool and switch Weight Type to DQ Blend Weight and then paint the elbows, shoulders and knees white (DQ) while leaving all else black (classic linear). Note: as of Maya2011 Mirror Weights does not yet mirror blend weights, so your best bet is to do it manually to both sides.

Continue refining the skin envelope, being careful to test every deformation as you go.
To clean up your skinning, finalize by going to Skin→Edit Smooth Skin→Prune Small Weights and then Skin→Edit Smooth Skin→Remove Unused Influences.

Basic Face Blendshapes, Method 1: using Maya

This is the “old” method, still the preferred method if you don’t have Mudbox.
First we’ll duplicate the entire character mesh for facial blendshapes. We will create facial sculpts that will act as morph targets for the character’s facial expressions.
Notes on head extraction: Because this is a seamless mesh, we’ll copy the entire character. Only vertices that move will be evaluated in a blendshape, so duplicating the entire body mesh won’t create too big a performance hit (although it will increase scene size). If you’d like to separate the head you can perform an extract operation, duplicate that for blendshapes and then combine the second mesh back to the body; however, this method will leave you with a mesh that you cannot delete history on, nor adjust skin weighting, so it is not recommended. The third option would be to download the popular detatchSeparate script from creative crash: http://www.creativecrash.com/maya/downloads/scripts-plugins/polygon/c/detachseparate-mel/download_page which will allow you to extract the head with no history, leaving the body untouched. However, the head will have to be reskinned afterwards. Best practices is always to design characters with heads modeled as a separate mesh to begin with, if you know you will be doing blendshapes for facial animation.
      If you haven’t already, group all your meshes into a group called geoGRP.
      Duplicate the geoGRP group and move new one to the side (CTRL+D)
(By duplicating the group, we are copying the eyes as reference so we can create eyelid blendshapes properly)
      Shift+D to duplicate with transform 5 more times
      Select those six, duplicate them and move them up to be a second row (12 in all). The top six will be the upper face blends, the bottom six the lower face blends.

      Create the BlendShapes (see below) by using the move tool with soft select and reflection turned on. Name each copied group the name you want the blendshape to be (browsUp, browsDown, etc). Be careful not to move any vertices but the ones in the local area of the particular blendshape. For example, brow shapes should only move vertices around the brow.

The Upper Face Shapes:
      Brows down – brows uniformly down and slightly forward over eyes
      Brows up – "surprise" brows, pushed up and back along forehead
      Sad – inner brows raised, outer brows lowered
      Angry – outer brows raised, inner brows lowered
      Closed – eyelids closed as in a blink

The Lower face shapes:
      Narrow – lips pursed or puckered as if blowing or saying "oooo"
      Wide  - lips long and thin, corners pulled back around the contours of the face
      FV – lower lip curled inward at the top, to fit under the upper teeth in a closed position (as in saying "f" or "v")
      Frown – ends of mouth pulled slightly downward
      Disgust – a snear: upper lip slightly curled outward and nasal folds lifted higher around nose
      Smile – ends of mouth pulled upward slightly


By making blendshapes very local they are as modular as possible; that is, meant to be used in combination to create the widest variety of possible shapes. Notice we didn't make mouth open / mouth closed shapes; that's because we weighted the jaw area to be driven by a jaw bone, which we will animate for open/closed mouth action.
Note: to dial in facial asymmetry (so important!) you’ll additionally have to break out each face shape to right/left side only versions. This can be time consuming - you can't just mirror geometry in Maya or you'll reverse the vertex order which blendshapes depend on. Because of this, the use of Mudbox (see below) is highly recommended for blendshapes.

      Select all the mesh copies, shift+select the original (base) model and perform Create Deformers→Blend shape (options)
      On the basic tab, give it a name under BlendShape node: faceblends
On the Advanced tab, change Deformation Order to Front of Chain (the blendshape deformer will evaluate before the skincluster)
      Open Window→Animation Editors→Blend Shape to access sliders for these new blends, or use the channel box (with base shape selected).
      You can now delete or hide all the target blend heads (you can change the targets to update the base head, so you may wish to keep them around on a hidden layer).

Basic Face Blendshapes, Method 2: Using Mudbox

If you have Mudbox, this is the preferred method, as it makes the breaking out into left and right side shapes much easier.
Select your skinned character mesh and File→Export Selected to Mudbox… (Maya2011) or File→Send to Mudbox→Send as New Scene (Maya2012). (In Mudbox2012 there is a bug that may cause errors importing a skinned character ... in this case, duplicate the mesh first and send the duplicate.)
In Mudbox, immediately create a sculpt layer. Name it Brows Down.

Using the grab tool, grab the vertices in the vicinity of the brows and pull them down (hit W to see the wireframe) to sculpt a brows down shape. Turn on Mirror: X under tool properties (click the tool icon and look in the Properties palette) so that you can sculpt with symmetry.

Under the layers palette menu, Duplicate Selected twice. Hide all sculpt layers except the current one while you sculpt. Do this by clicking the bullet under the eyeball column in the layers palette.

The first duplicate rename Left Brow Down. With this layer current, select the Erase tool (under the Sculpt tools palette) and erase away all changes made to the right side. Toggle the layer’s visibility on and off and look for any vertices popping on the right side to make sure you got them all. (May have to turn off Mirror in the erase tool properties.)

Do this for the other duplicate layer, naming it Right Brow Down, and erasing all changes made to the left side.

Repeat this basic process for all the blend shapes.

Upper face:

  • Brows Down
  • Left Brow Down
  • Right Brow Down
  • Brows Up
  • Left Brow Up
  • Right Brow Up
  • Sad
  • Left Sad
  • Right Sad
  • Angry
  • Left Angry
  • Right Angry
  • Closed
  • Left Eye Closed
  • Right Eye Closed
  • possibly others, such as Squint ...

Lower Face:

  • Narrow
  • Wide
  • FV
  • Frown
  • Left Frown
  • Right Frown
  • Smile
  • Left Smile
  • Right Smile
  • Snear
  • Left Snear
  • Right Snear

·         possibly others, such as Cheek Puff or Lip Curl In/Lip Curl out
Note: Best not to subdivide the mesh after you start working on blendshapes – sculpt layers are tied to the particular subdivision level you edit them on.
Return the current layer to the base mesh (all other layers visibility turned off) and go to File→Send to Maya. (In Mudbox2012 you can Update Current Scene.) In the Send options, change If there are sculpt layers: to Export sculpt layers on current mesh level as Maya blendshapes. (In Mudbox2012 this is under File→Send to Maya→Preferences ...)
If you sent the character back to Maya as a new scene, Maya will prompt you to save and start a new scene. Your mesh comes in, complete with blendshapes. Test them out.
Save this temporary file and then reopen your character file. Import the version of your model with the blendshapes on it – it should be in the same position as your skinned model. Select the skinned model, shift-select the blendshapes model, and Skin→Edit Smooth Skin→Substitute Geometry. You now have a skinned model with blendshapes.

Setting Up Animation Control Objects for Facial Animation

Next we’ll create a control object that animators can use to manipulate the jawbone and the faceblends. By doing one control object, you’ll be able to see how this process works.
Using the EP curve tool set to linear, create a square on the grid in the front camera. Modify→Center Pivot. Move and scale this into place next to your character and freeze transformations. Name it mouthControlBox.

Create→Locator and point constrain it (maintain offset turned off) to the NURBS square, then delete the constraint. This is to place the locator in the dead center of the square. Freeze transformations on the locator. Name it mouthControl. Parent it under the mouthControlBox. In the attribute editor for mouthControlBoxShape node, under Object display, turn on Template so the box itself can't be selected. You can turn this back off if you ever need to move the whole control.

Now select the locator and move it in x to the edge of the square, taking note of what value you reach in the channel box. In my case, I created a 5x5 unit square, so the locator travels 2.5 units to reach the sides. This is where we will set the limits on the locator.
Go to the Attribute Editor for the locator and under Transform Attributes→Limit Information turn on limits for Trans limit X and Y, setting min and Max to -2.5 and 2.5 (or your numbers noted in the previous step). Lock and hide translate z, scale and visibility attributes. You now have a locator constrained within the limits of the NURBS square. Try it out.

Now we'll use Set Driven Key to drive four blend shapes from one control object, simplifying the control system for the animator by consolidating several channels into one keyframeable object.
Animate→Set Driven Key→Set...
With the mouthControl locator still selected, hit Load Driver. Select the mesh and in the channel box select the lowerFace blend node. Hit Load Driven.

The translateX of the mouthControl will be used to drive the narrow and wide blend shapes. Select TranslateX in the upper field and both Narrow and Wide in the lower field to set a default key at a value of 0. Hit the Key button.

Now select mouthControl in the SDK window to bring its attributes into the channel box. Set translateX to the maximum: 2.5. Select lowerFace in the SDK window to bring its attributes into the channel box and set the value for Wide to its maximum: 1. Hit Key.

Now change the mouthControl.translateX value to its minimum: -2.5 and the lowerface.narrow shape to its maximum:1 and hit Key. Test your slider by moving the mouthControl's translateX between its minimum and maximum to see it slide between the Narrow and Wide shapes.

Now set a default 0 key on mouthControl's translateY and lowerFace's Frown and Smile shapes.

Set a Key with mouthControl.translateY set to its maximum: 2.5 and lowerFace.smile set to its maximum:1.

Now set a key with mouthControl.translateY at its minimum: -2.5 and lowerface.frown at its maximum: 1. 

Now select the locator in your window and test your fully funtional control slider. Pulling it to the top of the box invokes a smile, toward the bottom for a frown, to the right for wide shapes and to the left for narrow shapes. The corners blend between harmonious shapes (narrow smile, widefrown, etc) but the overall setup prohibits the simultaneous invoking of shapes that are in opposition to each other. For example, simultaneously invoking both narrow and wide shapes would produce strange results, since the vertices are moving in opposite directions, so you would never use these incompatible shapes simultaneously. This setup prohibits such awkward distortions and makes the blendshape system easy for the animator to control and key.
You can use the Create→Text to create a label for the control, positioning it over the control box and templating it.

With this procedure, you can create the rest of the facial setup in the same way. One interesting case study is Package Man, downloadable at http://www.rigging101.com/.
His facial setup includes a slider that controls the rotation of the jawbone, linear sliders controlling the eyelids and eyebrows, and a slider controlling bend/squash blendshapes on a lattice deformer around the head for cartoon squash 'n' stretch. You can set up sliders to drive nonlinear deformers the same way (Create Deformers→Nonlinear→...). If you broke out your blendshapes into left and right shapes, you'll want to have separate sliders for the left and rights sides, or you can get fancy and use utility nodes such as the Condition node to create a box controller that slides between the left and right versions of a shape. In short, this same procedure can be used to drive just about any attribute value on a character rig! Here's a screen shot of the Package Man setup:

 When you are done setting up your facial rig, group the whole thing and parent it under the head control. It is best to add an attribute to the head control (or root control) that drives the visibility of the facial rig group, so you can turn it on when doing facial animation, and turn it off otherwise to avoid screen clutter.


Now we've followed the development cycle of Generikat from a line drawing to a fully rigged, animatable character. This set of tutorials was meant to demonstrate the most common practices in character modeling and setup using a simple test case. Characters of higher complexity can be set up by utilizing the same procedures demonstrated here.