Wednesday Wonders: Facing the music

For some reason, face morphing in music videos really took off, and the whole thing was launched with Michael Jackson’s video for Black or White in 1991. If you’re a 90s kid, you remember a good solid decade of music videos using face-morphing left and right.

Hell, I remember at the time picking up a face-morphing app in the five dollar bin at Fry’s, and although it ran slow as shit on my PC at the time, it did the job and morphed faces and, luckily, it never got killed by the “Oops, Windows isn’t backward compatible with this” problem, so it runs fast as hell now. Well, whenever I last used it, and it’s been a hot minute.

If you’ve never worked with the software, it basically goes like this. You load two photos, the before and after. Then, you mark out reference points on the first photo.

These are generally single dots marking common facial landmarks: inside and outside of each eye, likewise the eyebrows and mouth, bridge of the nose, outside and inside of the nostrils, top and bottom of where the ear hits the face, major landmarks along the hairline, and otherwise places where there are major changes of angle.

Next, you play connect the dots, at first in general, but then it becomes a game of triangles. If you’re patient enough and do it right, you wind up with a first image that is pretty closely mapped with a bunch of little triangles.

Meanwhile, this entire time, your software has been plopping that same mapping onto the second image. But, at least with the software I was working with then (and this may have changed) it only plops those points relative to the boundaries of the image, and not the features in it.

Oh yeah — first essential step in the process: Start with two images of identical dimensions, and faces placed about the same way in each.

The next step in the morph is to painstakingly drag each of the points overlaid on the second image to its corresponding face part. Depending upon how detailed you were in the first image, this can take a long, long time. At least the resizing of all those triangles happens automatically.

When you think you’ve got it, click the magic button, and the first image should morph into the second, based on the other parameters you gave it, which are mostly screen rate.

And that’s just for a still image. For a music video, repeat that for however many seconds any particular transition takes, times 24 frames per second. Ouch!

I think this will give you a greater appreciation of what Jackson’s producers did.

However… this was only the first computerized attempt at the effect in a music video. Six years earlier in 1985, the English duo Godley & Creme (one half of 10cc so… 5cc?) released their video Cry, and their face morphing effect is full-on analog. They didn’t have the advantage of powerful (or even wimpy) computers back then. Oh, sure, they had pulled off kind of early CGI effects for TRON in 1982, but those simple graphics were nowhere near good enough to swap faces.

So Godley & Crème did it the old fashioned way, and anyone who has ever worked in old school video production (or has nerded out over the firing up the Death Star firing moments in Episode IV) will know the term “Grass Valley Switcher.”

Basically, it was a mechanical device that could take the input from two or more video sources, as well as provide its own video input in the form of color fields and masks, and then swap them back and forth or transition one to the other.

And this is what they did in their music video for Cry.

Although, to be fair, they did it brilliantly because they were careful in their choices. Some of their transitions are fades from image A to B, while others are wipes, top down or bottom up. It all depended upon how well the images matched.

In 2017, the group Elbow did an intentional homage to this video using the same technique well into the digital age — and with a nod from Benedict Cumberbatch, with their song Gentle Storm.

And now we come to 2020. See, all of those face morphing videos from 1991 through the early 2000s still required humans to sit down and mark out the face parts and those triangles and whatnot, so it was a painstaking process.

And then, this happens…

These face morphs were created by a neural network that basically looked at the mouth parts and listened to the syllables of the song, and then kind of sort of found other faces and phonemes that matched, and then yanked them all together.

The most disturbing part of it, I think, is how damn good it is compared to all of the other versions. Turn off the sound or don’t understand the language, and it takes Jackson’s message from Black or White into the stratosphere.

Note, though, that this song is from a band named for its lead singer, Lil’ Coin (translated from Russian) and the song itself is about crime and corruption in Russia in the 1990s, titled Everytime. So… without cultural context, the reason for the morphing is ambiguous.

But it’s still an interesting note that 35 years after Godley & Crème first did the music video face morph, it’s still a popular technique with artists. And, honestly, if we don’t limit it to faces or moving media, it’s a hell of a lot older than that. As soon as humans figured out that they could exploit a difference in point of view, they began making images change before our eyes.

Sometimes, that’s a good thing artistically. Other times, when the changes are less benevolent, it’s a bad thing. It’s especially disturbing that AI is getting into the game, and Lil’ Coin’s video is not necessarily a good sign.

Oh, sure, a good music video, but I can’t help but think that it was just a test launch in what is going to become a long, nasty, and ultimately unwinnable cyber war.

After all… how can any of you prove that this article wasn’t created by AI? Without asking me the right questions, you can’t. So there you go.

Image: (CC BY-SA 2.0) Edward Webb