I consider it an imperative that any interactive storytelling system include the ability to present displays of the faces of the characters as they interact with the player. I begin, however, with an aside on the subtlety of facial expressions as demonstrated by this photo that appeared on The Late Show with this comment:
“Alexandra Ocasio-Cortez watching while you brush your teeth while leaving the water running.”
The comment is hilarious because it matches the facial expression perfectly. Consider the subtlety of the facial expression; marvel at your ability to detect the emotional expression it represents.
I have three primary specifications for the next generation facial display technology.
Artistic, not Photorealistic
The
facial display should be based on what artists have learned about the
human face, not what anatomists have learned. We should not seek
photorealism; we should seek expressive power. Compare these
photorealistic faces on the left with the artistic faces on the right:
It should be obvious from these examples that the faces on the right are more expressive than those on the left. Indeed, I find the faces on the left to be cold and dead.
The recent release of the photorealistic version of Disney’s The Lion King gives us an even more powerful demonstration of the emotional weakness of photorealistic representations of characters. Compare the original cartoon Scar with the photorealistic Scar:
The upper Scar has energy, life, and emotion. The lower scar looks pretty much the same in every shot. BORING!
3D, not 2D
We
need to make the move to 3D faces. I believe that I have taken 2D faces
as far as they can be taken; there just isn’t much room for
improvement. Note that every one of the heads in the upper display of
different characters is tilted. It’s possible to fake face tilt and
rotation slightly in a 2D display, but I see no point in squeezing that
dead horse any harder.
There are surely plenty of tools for creating such faces; I expect that people more knowledgeable than I am will have plenty of suggestions.
Static, not Animated
A
negative recommendation: I don’t think that we should include animated
face displays. With turn-sequenced systems such as we’ll be building, an
animated face will be inappropriate. What’s the face supposed to do if
the player decides to take a break? Look bored?
How Cartoony should faces be?
Scott McCloud wrote a classic book, Understanding Comics, which should be required reading for anybody working in interactive storytelling. Scott presents this triangle of faces from the comics, which provides us with a useful tool for figuring out what kind of facial display we want.
This triangle maps the faces along three dimensions. Read the book to grasp their significance. We’ll definitely want to discuss the kind of faces we want.
Blending Facial Expressions
The
Face Editor explained elsewhere has a number of interesting features.
The new version will be different in that it will present a
three-dimensional representation of a face rather than the 2D faces in
the Face Editor. There are a few additional features that might be
useful. One would be the ability to blend together two different
expressions. The basic algorithm is simple enough to permit such
blending. I propose that it be done with the Blend operator. Thus, for a
given situation, the story builder can specify a facial expression
presenting a combination of two standard expressions, like so:
DisplayExpression(FirstExpression, SecondExpression, BlendingFactor)