Cinemetrics; How Big Data Will Change Filmmaking

• Return to part 1, Movie Barcodes

Cinemetrics

This is where your mind gets blown. Last year I attended an international conference for film teachers called CILECT, at Columbia College in Chicago. A speaker there shared this video with me, and my jaw hit the floor.

What you’re about to watch in action is a computer program that can ingest movies, and create the author of this program calls a movie fingerprint. This project, called Cinemetrics, is a bachelor’s degree capstone project by a student at the Royal Academy of Arts in London named Frederic Brodbeck.

Movie Fingerprint Explanation

Wow, right?! Let me explain the fingerprint here a bit more carefully.

• The size of the circle lets you know how long the movie is; the bigger the circle the longer the movie.
• The color bands shown here are an average of all the colors throughout the movie. This view is regardless of what scene those colors were in. We can also look at the colors scene by scene like you saw in the Shinning.
• The pie slices movie based not on the speed of the cuts, but on the speed of the action within the frame. If a person runs across the screen in one shot that would make the pie piece move quickly.

Let’s take a look at these two movie fingerprints for the film Solyaris, and it’s remake thirty years later. We can see that the original is a longer film, has a little more movement in the footage, and that the color scheme used is a little lighter.

Or we can compare two movies that both take place in outer space. Obviously Aliens has a lot more movement to it, but both have a similar color pallet and both are about the same run time. This kind of comparison is fascinating to me.

We can learn more about a director’s style. These are the movie fingerprints for five popular Wes Anderson films, What did we just learn about Wes Anderson’s style? If you wanted to make a movie in the same vain as Wes Anderson these fingerprints would help a lot.

The film industry is primed for a Money Ball moment.

More Data In the Future

In the future we’re going to have more data, not less. We will be analyzing actor speech pace, and even the number of shots that show an actor’s face. We’re able to rapidly collect, and sort vast quantities of data. When a director asks you to “cut it like When Harry Met Sally” you’ll be better able to implement what that note means.

Applying Data to Teaching Editing

Why is all this important to EditStock? And why am I so personally invested in exploring data like this. The reason is because I want to apply that data to the way we learn editing. I want to build a brand new way of measuring our own progress as editors.

This new style of learning is important not only new editor’s, but also experienced ones who are working right now to improve their own craftsmanship. In fact, I think the very act of choosing to be an editor means you’re interested in becoming a better craftsman. Most editors chose this career specifically because they can learn, and grow, and try new artistic ideas.

In terms of education, I want to give you a couple hard example of what I’m talking about.

Right now the way we teach the creative aspect of editing is by giving students a mountain of dailies and saying go have at it. That’s not a whole lot different from your foreign language teacher giving you a book and saying, “stare at this until you figure out how to read it.” It is possible, but it takes a long time because you have to figure out the grammar on your own.

Instead, we should be taking this big complicated idea of “editing” and breaking it down into smaller parts, the way we teach sports and instruments. We should build puzzles based on the editorial grammar that we learn from big data. The wording for those puzzles is extremely important. For example, I’m going to give you this piece of data,

DATA: Wide shots are most likely to be used by editors when a shot contains big physical actions. Examples include when an actor stands up, or when a ball is thrown across a room.

Now I’m going to give you a puzzle,

PUZZLE: we’re going to play an unedited shot from the short film Handicapped John. Pick the four spots where an editor is mostly likely to use this wide shot.

Did you get the solution? Let’s take a look at the four spots we picked. See if you agree.

The big moments were right as John walks down the hall, when the stretcher rolls by, when the two lean over the cubicle wall, and finally when John gets up to leave. I know what you’re thinking, there are other spots where we could potentially use the wide shot. For example when John’s hands flail as he explains what went on in the bathroom. That’s why we worded the puzzle to ask for the four most likely spots. While you could argue about using John’s hands in the wide is good, you wouldn’t need to argue that the wide shot is needed for when he stands up out of his chair at the end.

Now let’s look at a cut by an editor who used EditStock’s feedback system. Let’s focus our attention on this editor’s use of the wide shot. There are other issues with this cut. There are even issues with the length of the wide shots this editor used, but that’s not what we’re here to fix. We’re focusing in on one issue that this editor needs to understand before we can move on to other things. If you think back to our baseball analogy, we don’t fix an outfielder’s swing at the same time they shag fly balls, though both maybe problem in the player’s game.

Puzzles alone will not be enough to push editors towards bettering themselves. We also need to offer students a meaningful score so that they can track their own progress, and see what areas they are weak in.

When I first introduced some of these ideas at the LA Creative Pro User Group about a year ago the very first comment I got from the audience was doubt. The commenter said something along the lines of “Editing is subjective. There is no right way to edit a scene. Editing is art.” I hear comments like this a lot actually. To that comment I say this:

The tools that EditStock is developing are not intended to create the perfect editor any more than an English teacher’s goal is to have a student write a perfectly written book. Rather, our goal is to create a literate editor, who is knowledgable, and conscious about the film grammar they choose in their cuts.

Hopefully in the next few years we’ll be able to show off some of the tools we’re building but for now we’ll save those details for another presentation. Stay tuned…