Will Disney Research’s AI FRAN Revolutionize Re-Aging of Actors?
Deepfakes trolled Hollywood for years. They’re cheaper, faster to use, and often better than more complex Hollywood technology. Now a new Disney AI has copied the fundamental technology of deepfakes–and improved on it. What does this mean for the future of digital filmmaking? And does it banish uncanny valley forever?
Creating realistic digital humans is the holy grail of computer effects. It’s an art form that VFX companies have spent millions of dollars and countless hours trying to perfect. But often their digital humans fall into uncanny valley, a phrase used to describe CGI humans that are not-quite-right. They’re too plastic; their skin too smooth. Worse still, faulty-looking digital humans can have abnormal facial movements. Whether it’s building a digital human from the ground up or re-aging an existing actor, the process has proved quite difficult to master, even to this day.
Disney Research Hub offers hope with a recently revealed video on YouTube, detailing their new “production ready” de-aging / aging tool. Dubbed FRAN (Face Re-Aging Network), the new software takes its cues from deep fake software to make rapid changes to an actor’s age (making them either younger or older). Speed and photorealism are the name of the game today, with the number of productions increasing with the advent of streaming services. Speed also keeps the cost down. Speed, photorealism, and reduced cost in re-aging is likely to be required to keep pace with other technologies like ILM’s groundbreaking StageCraft that revolutionized production with virtual sets.
A recently published paper shows that Disney Research Hub has improved on deepfake technology. The paper details how FRAN uses thousands of generated synthetic faces of various ages. Like deepfakes, this software records a library of images that uses a machine learning algorithm to alter an actor’s face based on age. The algorithm takes an image, scans it, and calculates which part of the face will be most affected by age and how it will change over time. A major advance is the re-aged person still looks like the actor. In comparison, other re-aging tech can wipe out important facial characteristics in order to re-age a person. Tightening a jaw line, for instance, to make an actor appear younger does not necessarily reverse the natural aging process the actor underwent organically. FRAN resolves that problem.
FRAN’s algorithm can also incorporate the original lighting of the footage into the age changes due to the practice of overlaying these age changes onto the actor within a shot, which deepfakes usually can’t emulate.
With recent criticism of a de-aged Luke Skywalker in The Mandalorian and the subsequent tweaks made in deepfake by Youtubers at a fraction of the cost, it seemed likely Disney would study deepfake tech to surpass it. Unfortunately, the smooth, plastic look of the aging effects has not been completely banished. But the technology is improving. See for yourself.
