In case you haven't heard, the new "Star Trek" trailer has been released. To see it in full HD glory, take a peek at "Star Trek" at apple.com. "Star Trek," directed by J.J. Abrams, will be in theaters in May, 2009.
And here are some completely random frames from the trailer, as chosen by the Randomizer 2000(tm) software, featuring ArbitraryBoost 2.0.
You will undoubtedly hear this bit of advice repeatedly on this blog: the single best education for visual effects artists involves watching non-visual effects films and dissecting and analyzing the creative and technical choices made to achieve believable shots. In this article, I'll concentrate on organic, realistic camera work in live-action films and how photoreal visual effects artists can learn from cinematographers how to compose actors within the frame, and make their camera work feel natural and true to the style of the rest of the film. I'll also show you an example of my work from "Mission: Impossible III," and how I analyzed the film's camerawork to make my effects shot fit into the rest of the film.
One of the basic tenets of photographic camera work involves the framing of actors with appropriate headroom. Traditional Hollywood movies have used the rule of thirds to help guide cameramen into shooting pleasant compositions. Framing an actor's eyes to fit onto a line that is 1/3 of the way from the top edge of the frame is sure-fire way of achieving a 'proper' composition. Using this technique, the actors fill the frame in a pleasing manner with proper headroom; the audience can clearly read the actor's face. Generally speaking, this type of framing of actors' is the standard for Hollywood films. Unless a filmmaker has a stylistic reason for framing his actor differently (to give the shot a different, off-balance emotional feel), you will see this type of framing as the standard.
A shot from "Romancing the Stone" (1984), directed by Robert Zemeckis.
The same shot, with an upper-third overlay. Notice how Michael Douglas' eyes perfectly line up with the upper third of the frame.
Better yet, this kind of framing consistency means that when one shot cuts with another, the audience doesn't need to frantically scan the frame to lock their eyes on what is important: the actor's eyes. Actors' eyes are consistently featured at the upper third of the frame, which leaves the audience only to scan the frame horizontally from shot to shot. It's a subtle way to help guide an audience member through a sequence. Remember, in most cases, traditional Hollywood narrative films need to keep the techniques subordinate to the story. The editing, sound effects, cinematography and set design all need to service the story and characters, rather than call attention to themselves.
Another shot from "Romancing the Stone."
The same shot with an upper-third overlay.
This lesson is extremely valuable to anyone who creates visual effects or animated shots from scratch. Without any live-action footage to use as a basis, it is up to the artist to frame the shot properly. In these cases, it is important for visual effects artists to mimic their live-action counterparts and understand this basic convention of cinematography.
Let's watch this scene from "A Fish Called Wanda," directed by Charles Crichton. Watch how effortlessly we cut from angle to angle in this brilliant sequence, completely devoid of dialogue.
Watch it again, this time with the sound down (as to not distract you from analyzing the visuals), and pay close attention to the compositions, particularly on Michael Palin's close ups. Notice how the camera operator consistently frames its subjects in a pleasing manner, consistently striving for pleasing compositions, not only for actors but props and set pieces (like a gun, a pulley, ropes, and a giant block of concrete).
Notice how the cameraman is constantly reframing his tilt angle to give Palin a pleasant composition within the frame. These shots were photographed with a human being behind the camera, with separate pan and tilt wheels being cranked to frame the shot properly. This lends the shot an organic, hand-created quality, something that is frequently missing from visual effects shots that are 100% synthetic.
Raymond Stella, camera operator, sets up a shot for "Back To The Future Part II." Notice the pan wheel (on the left of the mount) and the tilt wheel (at the rear of the mount).
Why is there sometimes a discrepancy between the feel of real camera moves and synthetic moves? Why do purely synthetic visual effects shots sometimes stand out from ordinary non-effects shots? There are two primary reasons: purely synthetic camera compositions are not framed properly, and the camera's pan/tilt movement is frequently robotic.
As we just discussed, professional cinematographers have the tenets of composition in their DNA; on the other hand, very few digital artists have a photographic background, so it is important for these artists to have a solid foundation of knowledge when it comes to these concepts. Proper framing of subject is something any digital artist can learn.
A more difficult and subtle technique to learn, however, is the skill of adding the organic pan and tilt adjustments to a shot. Rather than pan and tilt wheels being 'performed' in real time on a set, synthetic camera moves are programmed using motion curves on a computer. By default, these curves are exquisitely perfect with either linear perfection, or computer-created acceleration and deceleration slopes. And, if the artist doesn't go in and 'mess up' those curves, they will end up creating a camera move that is sterile and robotic.
Antiseptic, inorganic camera moves are not deal-breakers, but they subconsciously take the viewer out of the experience. A visual effects shot with crystal-clean camera movement, within the context of non-effects shots, is disruptive to the viewing experience. An audience becomes conditioned to the natural, organic nature of a film's camera movement, which makes the visual effects shot stick out, even if that visual effects shot looks completely photoreal and convincing. This is one of those situations in visual effects that, when noticed, causes audiences to say, "That shot feels fake. I don't know why it looks fake, but there's definitely something fake about it."
How can shots with robotic camera moves slip through the cracks? How can visual effects supervisors allow this kind of camera animation get through to the big screen, given all of the hard work and constant reviewing that the shot gets during production? The answer is simple - the vast majority of the time, a visual effects shot, while in production, is viewed hundreds of times on a loop, without any context. The shot just plays over and over at an artist's desk, in dailies and in client reviews. Yes, the shot is cut into a sequence (both on the effects house's side, and on the client's side), but the vast majority of the time, the shot is being analyzed and critiqued in a vacuum. And in a vacuum, the camera move might look just fine. In fact, having an overly-smooth camera move in an effects might actually increase the chances of a shot getting approved. Subconsciously, the artists want their work to be seen, and seen clearly (not only for reasons of vanity, but also from a story point of view), and keeping the camera moves uncomplicated certainly helps the readability of the shot. Adding organic tweaks and twitches to the camera move may be correct and more realistic in context, but it may actually decrease the readability of the shot. Personally, my artistic sensibility is to err on the side of naturalism over readability. Our job as visual effects artists is to make shots that feel as spontaneous and organic as possible, as if the action is truly happening right in front of our eyes; creating organic camera moves is part of that process.
Here's another section of the "Wanda" scene, played back in slow motion with an upper-third horizontal line overlay, so we can better analyze the camera movement.
Notice how the operator is striving to hit the proper eyeline framing. Look at all of the micro movements in the tilt that occur throughout the shot. The operator knows the key marks that Palin will hit, but he doesn't necessarily know precisely when he'll hit those marks. The result is the kind of organic, humanistic camera movement that subtly reminds you that this is a movie made by human beings. These are the kinds of cues that are important to add to synthetic camera moves.
Depending on the shot, the camera operator might miss a moment entirely. Watch this clip from the "Wanda" sequence to see what happens when the operator is a bit behind the action.
For a moment, Palin's eyes are completely off screen. Watching this shot on a loop, one might be surprised that a shot like this would actually make it into the movie. In fact, this is one of those "I would never be able to get away with that kind of move in visual effects" moments. No visual effects supervisor or director would approve an effects shot with that kind of camera move in their shot-- they would complain that the move is far too wild, and we lose focus on what's important: Palin's horrified reaction to what he has just done. However, within the context of the rest of the scene, we 'get it,' and Palin's eyes briefly leaving frame just adds to the emotional shock and absurd comedy of the scene.
Of course, every cinematographer and camera operator has a different style. One cannot study the camera style of a single film and expect to understand the vocabulary of camera movements. "A Fish Called Wanda" was classically shot; in creating this character-driven comedy, the filmmakers made a conscious decision to allow the actors to move within the frame, shoot relatively static, and not call attention to the filmmaking process by keeping the camera moves unobtrusive. On the other hand, a modern action film like "Mission: Impossible III" contains a visual style quite different.
Director J.J. Abrams and cinematographer Dan Mindel created a film whose camera movement was essentially a character in the film. Rather than allowing the camerawork to be invisible and unobtrusive, the camerawork in "MI3" does some heavy lifting; the frenetic nature of the camera style adds to the chaos and unpredictability of the action sequences. Rather than letting the actors work within a static frame, the camera takes us along for the ride and interacts with the characters and the action. When Ethan Hunt (Tom Cruise) is running down the Chesapeake Bay bridge, the camera is running right along with him, trying to keep up, mirroring the emotions of the characters. This style is sometimes called cinema verite, or documentary-style filmmaking, where the camera is an active participant in the action, rather than objectively and coldly recording the choreography.
I was sequence supervisor of the visual effects for the bridge attack sequence for "MI3" at Industrial Light & Magic. To ensure that our visual effects shots fit seamlessly into the rest of the film, we had to pay close attention to the style of camerawork created by Mindel and camera operator Colin Anderson. Here is a portion of the bridge attack sequence, which is entirely comprised of first unit camera work. (Although there are a few ILM shots in this clip, we didn't alter the camerawork in any way.)
Notice the style of camera movement in the clip, contrasted with the "Wanda" clip. The "MI3" camera is constantly pulsing and breathing, mimicking Hunt's point of view. It's a moment of chaos, and the characters are trying to react to this chaos, and so is the camera. In this clip, the camera is constantly searching, readjusting, sometimes losing 'proper' framing. There are a couple of micro-crash zooms in the clip, as well, which is something one would typically see in a documentary. This is exactly the sequence I studied to help create the camera move for one of our big effects shots of the sequence-- a shot that I composited, and that would require a synthetic camera move.
The final two shots of the above clip were very important to me, since they had a very similar tone and feel to my shot, with similar action and camera choreography. The style in which Mindel and Anderson photographed those two shots would be my inspiration for the camera move for "be064," the production code for the shot of Hunt getting blown forward by the shockwave of the truck explosion behind him.
Because of the staggering complexity of the stuntwork, special effects, and ultimately the visual effects work, the base background plates were photographed as smoothly as possible, rather than 'performed' as the rest of the movie was photographed. Visual effects supervisor Roger Guyett needed to make sure that we captured pristine photography of the elaborate stunt that the actor, himself, would be performing, along with the amazing practical special effects explosion.
Here are the base photographic plates for the shot, as featured in the "MI3" 2-disc DVD special edition:
Because the focus of this article is visual effects camera work, I will not be able to dive into the massive complexities of this particular shot. Hopefully, I'll be able to talk about the specific challenges of this effects shot in a future article.
As you can plainly see, the base photographic plates for be064 were shot without panning or tilting, slightly wider than necessary, and captured with two cameras with spherical lenses (I would later merge these plates into one large, wide anamorphic plate). I also built my composite with this smooth camera move, without any frenetic camera moves. Once my smooth pre-composite was in good shape, I would add the camera moves in the 2D composite.
Abrams had some very specific camera direction for key moments in the shot. At the head of the shot, he wanted to be tight on Hunt (similar to the framing of the second to last shot of the first "MI3" clip.) Then, just before the missile hits the truck, he wanted a crash zoom out, to reveal a wider view of the action. Then, after the explosion occurs, he wanted the 'operator' to pan/tilt and zoom into the unmanned aerial vehicle (UAV) that whizzes right over Hunt's head. After the flyby, Abrams wanted me to bring the camera back to our hero, still reeling from the shock of the explosion.
Needless to say, that's a lot of choreography for a pretty short shot. To add to the complexity, I would need to add an explosive camera shake to the shot (in reaction to the massive explosion behind Hunt).
Here were my specific ideas for the camera moves for the shot:
• I was simulating the look of a handheld camera operator sitting atop a dolly, which gives us the smooth movement backwards with all of the energy of handheld.
• Because of the handheld nature of the shot, I made sure to maintain generally 'proper' framing, but also keep it active and chaotic, and allow Hunt's head to break frame from time to time.
• At the head of the shot, when we're still zoomed in tight on Hunt, I made sure to make the camera bobble realistically amplified, which is what you would see in a real-life camera shake with a long lens. Because the field of view is so tiny, even small camera bumps are seen as giant waves of movement. Of course, after the crash zoom out, I made sure to diminish the intensity of the camera bumps.
• On the crash zoom out, I made sure that the zoom and the pan/tilt readjustments that I would make post-zoom were not simultaneous. In real life, the operator zooms out, and then needs to make a split-second decision on framing. It's subtle, but it's there.
• On the explosive camera shake, I had to juggle two challenges which were at odds with one another. I had to make sure that the camera shake did not distort the image to a degree that we would somehow miss the important point of the shot - that the actor is doing his own stunt. This was a very important emotional beat to the shot; I didn't want the audience to think that I was tricking them, or hiding something with an over-the-top camera shake (which audiences, sadly, see all the time). I also had to make sure that the shake was appropriate for a giant explosion that was strong enough to blow an actor off of his feet, smashing into a parked car with such velocity that it shatters its rear window. If anything, I erred on the side of preserving the readability of the shot. Besides, all of the other effects I added (the dust shockwave, manually animating the briefcase across the ground, etc.) helped sell the intensity of the explosion.
• After Hunt hits the car, I wanted to tilt down with his body to achieve a better composition, but notice how I am just a bit late on the tilt. Keeping the tilt just a beat late gives the shot a bit of authenticity, as if the cameraman was operating the camera live, and didn't know exactly where his body would be at any particular time.
• On the UAV flyby, I tried to mimic a camera operator who could see the UAV out of the corner of his eye, and just barely was able to frame for the fast moving jet. And, if you look carefully, we never really get a good composition of the UAV at all, which was fine with us, since the UAV's flyby was an accent to the shot, not the main driving force of the shot.
• Finally, on the pan/tilt back to Hunt and the burning truck, I wanted to impart the energy of a camera operator who carefully rests the camera on his original framing, doing a bit of work to get the proper framing.
Ultimately, the biggest challenge for me was animating a camera shake for the explosion that was big enough to logically fit with the action (a human being blown off his feet), and yet not obscure the action and the big selling point of the shot, which is that Tom Cruise performed his own stunt. Abrams wanted the audience to truly feel what Agent Hunt was going through, to see the anguish on his face, and see what happens to his body after this amazing event.
Integral to this whole process was creating an organic, naturalistic camera move that complemented the rest of the film, and helped sell the spontaneity and reality of the situation.
Here's a clip that ran on the entertainment show "Extra" back in 2005, in the days leading up to the release of "Star Wars: Episode III."
Did you miss me? Did you blink? Like these things usually go, the reporter from "Extra" and I talked for well over an hour, and really got into the nitty gritty of that particular shot... and it was all boiled down to two lines. At least they got to show my breakdown of the shot, which featured some sweet footage of actual lava blasts from Mt. Aetna in Sicily, photographed in HD by Ron Fricke (director of "Baraka") in late 2004. I placed most of the elements in 3D space in After Effects, but rendered out those passes and comped the shot in our in-house compositing tool, CompTime.
The entire Mustafar sequence was a real joy to help create - we had an amazing team comprised of visual effects supervisor Roger Guyett, compositing supervisor Pat Tubach, and comp sequence supervisor Mike Conte, all of whom were extremely generous.
In case you haven't heard, the new "Star Trek" trailer has been released. To see it in full HD glory, take a peek at "Star Trek" at apple.com. "Star Trek," directed by J.J. Abrams, will be in theaters in May, 2009.
You will undoubtedly hear this bit of advice repeatedly on this blog: the single best education for visual effects artists involves watching non-visual effects films and dissecting and analyzing the creative and technical choices made to achieve believable shots. In this article, I'll concentrate on organic, realistic camera work in live-action films and how photoreal visual effects artists can learn from cinematographers how to compose actors within the frame, and make their camera work feel natural and true to the style of the rest of the film. I'll also show you an example of my work from "Mission: Impossible III," and how I analyzed the film's camerawork to make my effects shot fit into the rest of the film.
A shot from "Romancing the Stone" (1984), directed by Robert Zemeckis.
The same shot, with an upper-third overlay. Notice how Michael Douglas' eyes perfectly line up with the upper third of the frame.
Another shot from "Romancing the Stone."
The same shot with an upper-third overlay.
Raymond Stella, camera operator, sets up a shot for "Back To The Future Part II." Notice the pan wheel (on the left of the mount) and the tilt wheel (at the rear of the mount).
A more difficult and subtle technique to learn, however, is the skill of adding the organic pan and tilt adjustments to a shot. Rather than pan and tilt wheels being 'performed' in real time on a set, synthetic camera moves are programmed using motion curves on a computer. By default, these curves are exquisitely perfect with either linear perfection, or computer-created acceleration and deceleration slopes. And, if the artist doesn't go in and 'mess up' those curves, they will end up creating a camera move that is sterile and robotic.
Of course, every cinematographer and camera operator has a different style. One cannot study the camera style of a single film and expect to understand the vocabulary of camera movements. "A Fish Called Wanda" was classically shot; in creating this character-driven comedy, the filmmakers made a conscious decision to allow the actors to move within the frame, shoot relatively static, and not call attention to the filmmaking process by keeping the camera moves unobtrusive. On the other hand, a modern action film like "Mission: Impossible III" contains a visual style quite different.
• On the UAV flyby, I tried to mimic a camera operator who could see the UAV out of the corner of his eye, and just barely was able to frame for the fast moving jet. And, if you look carefully, we never really get a good composition of the UAV at all, which was fine with us, since the UAV's flyby was an accent to the shot, not the main driving force of the shot.
Here's a clip that ran on the entertainment show "Extra" back in 2005, in the days leading up to the release of "Star Wars: Episode III."
The entire Mustafar sequence was a real joy to help create - we had an amazing team comprised of visual effects supervisor Roger Guyett, compositing supervisor Pat Tubach, and comp sequence supervisor Mike Conte, all of whom were extremely generous.