Saturday, January 23, 2010

The VFX Predictinator, Part 1

Avid readers of this site may have noticed a curious omission during last year's awards season. I didn't run the numbers to try and predict which film would win the Academy Award for Best Visual Effects.

In early 2008, I took an exhaustive (and exhausting) look at the previous 23 years of visual effects nominees and tried to determine what was a better predictor of the film that wins the Oscar: critical acclaim or box office popularity. (If you're interested in all of the explanations and data, be sure to read the full articles, "Predicting the VFX Oscar Part 1," which has links to Parts 2 through 5.)

Here is a brief summary for those who need to catch up. I've always wondered if there exists a statistical relationship between the winner of the visual effects Academy Award and some sort of quantifiable criteria which informs the Academy's voting. I wondered if one could accurately predict which nominee wins the award. And, most importantly, which is the more accurate predictor of the visual effects Oscar - critical acclaim or box office success?

And please keep in mind that this is a game of prediction; this process does not take into account the actual quality or achievement of each years' nominees. We are looking at Academy voting trends, and what criteria influences Academy voters' choices.

Analysis of 23 years of Academy Award nominees for Best Visual Effects brought me to this conclusion: critical acclaim is generally a better predictor of the winner of the Academy Award than box office popularity. And, to state the obvious, this theory surmises that critical acclaim (which initially drives the wave of publicity, such as "For Your Consideration" advertisements, Oscar "buzz," and the self-fulfilling prophecy of "If people are saying it's Oscar-worthy, then it must be Oscar-worthy") ultimately informs Academy voters and influences their vote.

For the 80th Academy Awards, I ran the numbers (in Part 4), and the data overwhelmingly indicated that "Transformers" would bring home the gold. The silly giant robot film had the greatest critical acclaim (a modest 57% on the Tomatometer, but still higher than its competitors) and earned the most money at the box office (barely edging out "Pirates 3" but destroying "The Golden Compass"). In addition, in eleven years of the 23 years charted the film that earned the most acclaim and box office won the Oscar ten times. Armed with this ammunition, I felt pretty confident predicting "Transformers" would bring home the gold. Of course, I-- and the rest of the moviegoing world-- was a bit stunned with "The Golden Compass" win.


Not only did my prediction fall flat on its face, but my heart was broken (not because I'm particularly fond of shiny robots, but because I was a sequence supervisor at ILM for the film). The "Transformers" loss left my theory dead.

As the 81st Academy Awards nominations were announced, I had a sinking feeling that the theory would fall apart yet again. The three nominees for the Oscar were "The Dark Knight," "Iron Man" and "Benjamin Button." Looking at the numbers, all three films earned some fairly solid acclaim ("Dark Knight" earned 94% on the Tomatometer, "Iron Man" earned 93% and "Button" garnered 72%), and although "Dark Knight" destroyed at the box office, they each made a respectable amount of money ($513, $318 and $125 million, respectively). According to the numbers, with its domination of the box office and the highest amount of critical acclaim (by a hair), "The Dark Knight" should have wrapped up the statuette.

As we drew closer to the awards, I realized it was not meant to be. The stinging, theory-destroying win of "The Golden Compass" would probably be repeated, with "Button," the least acclaimed nominee and the least earning film winning the Oscar. (On a personal note, I actually felt that "Button" deserved the Oscar by a hair, just slightly more than "Dark Knight." But to quote Clint Eastwood in "Unforgiven," 'deserve's got nothin' to do with it.')

So I decided not to run the numbers and not write up a blog post. Ultimately, I feel somewhat vindicated since my fears came true; "Button" won the award, once again foiling my theory that the film with the most amount of critical acclaim has the best chance of winning the award.

Just before the Oscars, I heard through the grapevine that my humble "Predict The Oscar" methodology was mentioned on The VFX Show podcast, show #68, from February 13, 2009. As an aside - thanks, guys, for the mention! When my wife heard about FXRant's mention on an actual, respected podcast, her immediate response was "Wow, somebody actually reads your blog? And then talked about it?!" Later, after thinking about it for a few moments, she said, "You know, you're doing this all wrong."

"Doing what all wrong?" I asked.

She said that I was looking for correlation between statistics and the ultimate winner of the Academy Award-- and getting upset when the correlation breaks between the data and reality. A far more interesting exercise would be to devise a formula to actually predict the winner, based on quantifiable, known criteria. We could look at several years' worth of data, craft a series of calculations based on certain values, and weight those values accordingly to skew the formula in the right direction.

She, like always, was correct.

After analyzing the data and looking for the trends in the winning films, we added several new criteria and weighted the criteria until we hit paydirt. Using nine carefully weighted scores, we devised a formula which was able to predict the visual effects Academy Award winner with a strong degree of accuracy.

How strong?

In 20 years of Academy Awards data, the formula is 100% accurate. It correctly predicted the winner of the visual effects Academy Award every single year. We're calling the formula The VFX Predictinator.

Stay tuned for Part 2 coming soon! It will be worthy of these exclamation points! Trust me!

And now... here's Part 2!


Akira said...

Can't wait! :D

Natasha Price-Lopez said...

Wow Kearsley! Want to work some numbers for my investments??

Mike Van Eps said...

What about the 'Actors and Actresses just check the first movie listed on their ballot because they don't really care about VFX' theory? Not saying that I believe it but I have heard it.

Todd Vaziri said...

Mike Van Eps said...
>What about the 'Actors and Actresses just check the first movie listed on their ballot because they don't really care about VFX' theory?

The 'alphabetical order' theory!?! Blast it! I have to back to the labs and re-jigger the formula! :)

In all seriousness, if I see winning films trending toward the first one listed on the ballot, then I'll add it to the formula!


vfx fan said...
This comment has been removed by the author.
vfx fan said...

I think that a vfx-nominated movie that is also nominated for Best Art Direction has the best chance of winning. See "The Golden Compass" and "What Dreams May Come" among others.

The Academy is far more likely to look at the vfx of a film in terms of its beauty and might see Best VFX as a sort of secondary Best Art Direction award.

I don't understand what was so great about "The Dark Knight." I didn't even think its fx were that impressive. I was more outraged that "Speed Racer" didn't even make it into the first 15 runners-up. That was the most visually original movie I've ever seen -- and it's what the Wachowskis are truly good at.

And don't get me started with "The Golden Compass." That movie had no business being in the final three. I honestly would've chosen "Evan Almighty" (which I believe was in the final 7) to be in the final 3 in place of "Compass" for its highly impressive water simulations. Failing that, "300."

michaelB said...

they gave the oscar to golden compass because they didn't want to honor a micheal bay film with giant robots. yes pearl harbor won an oscar for sound editing( if i'm not wrong) but that was an historical film, giving the gold to a micheal bay movie about a bunch of hasbro toys was too much. and because this, i think transformers 2 will not even be nominated( sadly, because it has a dozen of great vfx scenes). about avatar: like i wrote in the apposite thread i think that the WETA deserve the oscar of course, but the na'vi are not photorealistic like cameron promised, with grace's avatar and neytiri's boyfriend( not jacke, the native boyfriend) looking particularly bad.

vfx fan said...

The Na'vi/Avatars were photorealistic in the same vein as ILM's Hulk (which I thought was also a very amazing and overlooked character both technologically and artistically), meaning they were photorealistic and cartoony at the same time. Of course, with the six-year advancement of technology, the blue guys would inevitably be more amazing.

But Weta was responsible for all kinds of other awesome stuff, like entirely CG jungles and vegetation that I could've easily believed were real or at least miniatures.

Still, I think Benjamin Button is the most photorealistic animated character today. I could've easily believed that they got a short old man to do the part (although they technically did).

TylerMirage said...

I agree with vfxfan. Although "Hulk" received negative visual effect reviews, I honestly believe that the Hulk character was at the time, and still holds up to this day, one of the most photorealistic CG human-like characters. I mean, I thought the film was okay, but then I enjoyed Leterrier's "The Incredible Hulk" much better, as it was more of a Hulk-film. However, Despite having five years of advancements, Rhythm & Hues' Hulk character, although looking more like the Hulk, was not as photorealistic as ILM's previous attempt. Maybe it was the neon-green skin that made him look Shrek-y or fake?

And also, "The Golden Compass"'s win over "Transformers" = FAIL. Like Todd put it, the entire movie-going world was shocked at this. Some awards just HAVE to be. "Avatar" HAS to win, Heath Ledger HAD to win as The Joker, and "Transformers" HAD to win. didn't. I realize that I am not a judge at the Oscars, and thus do not know all of the rules and regulations, but I mean, really? I'm fine with "The Golden Compass" being in the final three, but I'm sorry, "Transformers" HAD to win. It just HAD to. WHY??????

I'm still rooting for "Transformers 2" to be nominated, but for "Avatar" to win. *cheers silently*

michaelB said...

i agree with tyler and vfx about hulk. the only problem i had with ang lee's hulk was the fight with his father at the end of the movie: i think that everything in that sequence looks too obviously computer generated. back on topic i think that benjamin button( even if it was only the head) and davy jones showed that is possible to fool the audience with cg carachters. so i was expecting the same thing at least from one of the na'vi...

Anonymous said...

yeah on these times it's kind of hard to choose one, but choosing a good movie it's not just action and effects, the script is one of the principal things the judges qualify and the popularity doesn't count too much, for example with generic viagra, in pharmacy the best one it's the one with less secundary effects, not the one make you have more time.
Nice post

Dale H Bernier said...

I think expanding the category to include 5 films is a wise choice and long overdue. Look at the films that make money these days and you are looking at a long list of blockbusters centered on visual effects.

VFX has become the stars of the show in movies such as Tron, Transformers, and Inception. This needs to be acknowledged and commended in the industry so expanding the nominations for best visual effects is definitely a step in the right direction.

I work for a midsize VFX studio in Montreal, Canada and we have some really talented artists doing amazing work on films and commercials. It is nice to know that this is being valued more in the industry, especially considering how exploited VFX artists are these days. We are keeping our company small and we refuse to underbid in order to win jobs, so our VFX artists are never exploited or underpaid.

We hold VFX artists in high regard and it’s nice to see that the Academy, and the film industry in general, is starting to do so as well. I have more thoughts about the industry on my blog at

So feel free to check it out!