Artists who bring to life heroes and villains in animated movies and video games could have more control over their animations, due to a latest technique introduced by MIT researchers.

Their method generates mathematical functions referred to as barycentric coordinates, which define how 2D and 3D shapes can bend, stretch, and move through space. For instance, an artist using their tool could select functions that make the motions of a 3D cat’s tail fit their vision for the “look” of the animated feline.

Many other techniques for this problem are inflexible, providing only a single option for the barycentric coordinate functions for a certain animated character. Each function may or will not be one of the best one for a selected animation. The artist would have to begin from scratch with a latest approach every time they wish to try for a rather different look.

“As researchers, we will sometimes get stuck in a loop of solving artistic problems without consulting with artists. What artists care about is flexibility and the ‘look’ of their final product. They don’t care concerning the partial differential equations your algorithm solves behind the scenes,” says Ana Dodik, lead creator of a paper on this system.

Beyond its artistic applications, this system might be utilized in areas akin to medical imaging, architecture, virtual reality, and even in computer vision as a tool to assist robots work out how objects move in the actual world.

Dodik, an electrical engineering and computer science (EECS) graduate student, wrote the paper with Oded Stein, assistant professor on the University of Southern California’s Viterbi School of Engineering; Vincent Sitzmann, assistant professor of EECS who leads the Scene Representation Group within the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior creator Justin Solomon, an associate professor of EECS and leader of the CSAIL Geometric Data Processing Group. The research was recently presented at SIGGRAPH Asia.

A generalized approach

When an artist animates a 2D or 3D character, one common technique is to surround the complex shape of the character with an easier set of points connected by line segments or triangles, called a cage. The animator drags these points to maneuver and deform the character contained in the cage. The important thing technical problem is to find out how the character moves when the cage is modified; this motion is decided by the design of a selected barycentric coordinate function.

Traditional approaches use complicated equations to search out cage-based motions which might be extremely smooth, avoiding kinks that would develop in a shape when it’s stretched or bent to the intense. But there are various notions of how the artistic idea of “smoothness” translates into math, each of which results in a unique set of barycentric coordinate functions.

The MIT researchers sought a general approach that enables artists to have a say in designing or selecting amongst smoothness energies for any shape. Then the artist could preview the deformation and select the smoothness energy that appears one of the best to their taste.

Although flexible design of barycentric coordinates is a contemporary idea, the fundamental mathematical construction of barycentric coordinates dates back centuries. Introduced by the German mathematician August Möbius in 1827, barycentric coordinates dictate how each corner of a shape exerts influence over the form’s interior.

In a triangle, which is the form Möbius utilized in his calculations, barycentric coordinates are easy to design — but when the cage isn’t a triangle, the calculations change into messy. Making barycentric coordinates for an advanced cage is particularly difficult because, for complex shapes, each barycentric coordinate must meet a set of constraints while being as smooth as possible.

Diverging from past work, the team used a special form of neural network to model the unknown barycentric coordinate functions. A neural network, loosely based on the human brain, processes an input using many layers of interconnected nodes.

While neural networks are sometimes applied in AI applications that mimic human thought, on this project neural networks are used for a mathematical reason. The researchers’ network architecture knows output barycentric coordinate functions that satisfy all of the constraints exactly. They construct the constraints directly into the network, so when it generates solutions, they’re all the time valid. This construction helps artists design interesting barycentric coordinates without having to fret about mathematical elements of the issue.

“The tricky part was constructing within the constraints. Standard tools didn’t get us all the way in which there, so we actually needed to think outside the box,” Dodik says.

Virtual triangles

The researchers drew on the triangular barycentric coordinates Möbius introduced nearly 200 years ago. These triangular coordinates are easy to compute and satisfy all of the mandatory constraints, but modern cages are far more complex than triangles.

To bridge the gap, the researchers’ method covers a shape with overlapping virtual triangles that connect triplets of points on the surface of the cage.

“Each virtual triangle defines a legitimate barycentric coordinate function. We just need a way of mixing them,” she says.

That’s where the neural network is available in. It predicts mix the virtual triangles’ barycentric coordinates to make a more complicated, but smooth function.

Using their method, an artist could try one function, have a look at the ultimate animation, after which tweak the coordinates to generate different motions until they arrive at an animation that appears the way in which they need.

“From a practical perspective, I believe the most important impact is that neural networks offer you numerous flexibility that you simply didn’t previously have,” Dodik says.

The researchers demonstrated how their method could generate more natural-looking animations than other approaches, like a cat’s tail that curves easily when it moves as a substitute of folding rigidly near the vertices of the cage.

In the longer term, they wish to try different strategies to speed up the neural network. Additionally they wish to construct this method into an interactive interface that will enable an artist to simply iterate on animations in real time.

This research was funded, partly, by the U.S. Army Research Office, the U.S. Air Force Office of Scientific Research, the U.S. National Science Foundation, the CSAIL Systems that Learn Program, the MIT-IBM Watson AI Lab, the Toyota-CSAIL Joint Research Center, Adobe Systems, a Google Research Award, the Singapore Defense Science and Technology Agency, and the Amazon Science Hub.