In the opening scene of Toy Story, released in 1995, a cluster of boxes is scattered across a child’s bedroom. The sun streams into the room as a Mr Potato Head doll demands money from a seemingly stricken cast of plastic and plush toys outside a cardboard bank.
Into the picture arrives the hero – a cowboy sheriff made of plastic and fabric with a pull-string to make him speak. The sheriff casts a shadow over the villainous potato who flees from the law.
It’s a scene plucked from a child’s imagination. It was also the culmination of decades of development in computer animation.
This year two of the men behind those advancements, Ed Catmull and Pat Hanrahan, are the recipients of the Turing Award. The award recognizes “lasting and major” contributions to the field of computing and is considered to be the “Nobel Prize” of computer science.
The award is given by the Association for Computing Machinery and comes with a $1m cash prize split between the winners.
Dr Catmull, one of the founders of Pixar, the studio behind Toy Story, and Dr Hanrahan, one of Pixar’s early employees, were notified of their win in early March.
It gave the two old friends and former colleagues just enough time to meet for a celebratory meal before coronavirus lockdown measures were put in place in California, where they both live.
The digital revolution we have seen in all kinds of movies, television, games – probably no one made more of the difference to that then Ed and Pat,” says David Price, author of the book The Pixar Touch.
To make Toy Story and other computer-animated films possible, Dr Catmull, Dr Hanrahan and their teams had to develop ways to get computers to visualize three-dimensional objects.
During his postdoctoral studies, Dr Catmull created a way to make a computer to recognize a curved surface. Once developers had a mathematically defined curve surface they could begin to add more features to it – like texture and depth.
“Step by step you figure out what kind of lighting should be applied. Then you begin to put in the physics of it because plastic reflects light one way and metal reflects it in a very different way,” Dr Catmull explains.
Dr Catmull had always had an interest in animation and film.
After earning, his doctorate and working in a graphics lab in New York, he eventually became the head of computer division of Lucasfilms, founded by George Lucas. The creator of Star Wars and Jurassic Park saw the potential of computer animation in movies.
But Dr Catmull’s says his dream to make a feature-length computer animated film was still seen as “wildly impractical”.
“Most people dismissed the idea as an irrelevant pipe dream.”
Pixar is born
In 1986 the Apple founder Steve Jobs came along. He bought Lucasfilms’ computer division and turned it into a standalone company, Pixar.
At first, the firm tried to sell computer hardware. When that failed to take off, Pixar refocused on computer imagery.
Dr Hanrahan was one of the company’s earliest employees. He was put in charge of creating a minimum standard for the way computer code is used to describe images.
“Pixar was a magic place,” says Dr Hanrahan who now teaches at Stanford University.
He oversaw the creation of RenderMan – the software Pixar uses to create its 3D animation – working with teams from across the industry.
Shading and light
Crucially Dr Hanrahan worked out how to visualize how light reflects off of different surface. On surfaces like human skin some of the light passes through or is absorbed.
Getting this level of light and shade right gave the images a computer could create a realistic look.
RenderMan has been used to create animated films like Toy Story and Pixar’s A Bug’s Life. It was also essential for visual effects of live-action films including Terminator 2, Titanic and Jurassic Park.
Developments in computer animation drove the video gaming industry as well as advancements in virtual and augmented reality. And its progress has been tied closely to advancements in machine learning.
“Waiting for computers to catch up
According to Dr Catmull, sharing work across the industry and with other sectors allowed for the bigger breakthroughs, particularly processing power
Computers of the 1980s and 1990s had only a fraction of the processing power laptops and smartphones have today.
“[A lack of processing power] definitely was a limiting factor,” Dr Catmull explains.
“You had to almost bide your time working on the algorithms for the compute power to catch up with the ideas that we had.”
But even today computer animated films rely on small armies of animators.
“It’s a very labour-intensive process, we still have to do a lot of things manually,” says Dr Hanrahan.
“If you just want to have a character walk around a world and have a human-like motion that makes you think it’s natural, that’s a huge problem…we have no idea how to do that well.”
Developments in robotics have helped to make improvements in this area, demonstrating how crucial it still is to share learning across fields.