Expectations versus reality. So often, that's what Hollywood box office reporting boils down to: comparing how much money a movie made in its opening weekend to how much it was expected to make. It's a way of providing provide context because. While it might sound impressive to say that Pitch Perfect 2 grossed $69 million in its opening weekend in North America it's far more impressive if you point out that it was only expected to make $45 million. Similarly, Jurassic World's record-setting opening of $208.8 million seems inherently amazing, but doesn't seem even cooler if you know that it was only supposed to make between $115m and $125m? Such comparisons give the impression that a sincere cultural moment managed to trounce fancy statistical prediction models, but how exactly do we know how much something was expected to make? And why are those expectations thoroughly trounced every other week?
I've never had a firm grasp on the answers to those two questions. We know what something was supposed to make because an industry trade like The Hollywood Reporter said so, citing advanced ticket sales on sites like Fandango or a report from the box office research entity Rentrak or a statement from a studio's distribution chief. Why are the expectations usually a bit off? Let's just chalk that up to "margin of error," maybe cynically joke Hollywood studios prefer to keep their projections on the absolute low-end so that any opening weekend total above that will look all the more impressive by comparison.
But then Jurassic World came along and beat expectations by as much as $93 million. There are various reasons why Jurassic World did what it did, as I argued elsewhere ( here's the link). However, how did Hollywood not see this coming? How could Hollywood's pre-release projections have been off by nearly $100 million? As Deadline put it, "The guessing game of whether a picture beat estimates is all based on whether the tracking numbers are any good. They're not, which creates a set of expectations based on bad information, affecting all box office reporting."
If you ask many in the film industry they'd tell you that the current pre-release tracking system has been in dire need of improvement for years now. "Audience tracking, a cottage industry created in the 1970s, works principally by polling viewers at test screenings to gauge interest and awareness of a film, or over opening weekend to provide a window on a movie's demographics," Variety explained in 2013. "It's a good measuring device for marketing a film, but it simply isn't capable of drilling down deeply enough to provide the kind of analytics that newer digitally driven data-collection can." Tracking, provided to the studios by various third parties, used to primarily be the domain of New Research Group (NRG), but they've received competition in recent years from Box Office Analyst, MarketCast and Screen Engine, the most accurate of which is MarketCast (according to Deadline late last year).
These agencies identify a film's total awareness, unaided awareness, how many people rank it as the movie they most want to see that weekend, and how many people would definitely recommend it to friends. Those scores, gathered from polls conducted three times a week, are matched up to those of comparable older movies, like an earlier installment in the franchise, i.e., Furious 6 for Furious 7. In Jurassic World's case, the agencies compared it to Man of Steel, another franchise re-launch released on June 12 th. Such efforts only apply to studio-level wide releases thus leaving out indies playing in fewer than 600 theaters. A forecasting insider told Deadline that we all seem to fundamentally misunderstand what their goal actually is, "We're not paid to predict box office, rather identify pockets of strength, threats and opportunities in the marketplace for the studio. ... It's a five-week journey with daily phone calls." NRG's co-founder Catherine Paura echoed that viewpoint in conversation with Variety, "The real intention (of tracking) was to gauge advertising materials and where a film falls within the competitive landscape."
Motion Picture Group CEO Vincent Bruzzese told Variety that regardless of the agencies' motives their methods are not good enough anymore, "Simply knowing the awareness and interest of something only scratches the surface. It doesn't matter necessarily if you're going to definitely recommend a movie, so much as how you're going to recommend it." He further pointed out that someone who texts a friend immediately after a seeing a movie is likely a more enthusiastic recruiter for your movie than someone who liked it but only says so if asked about it directly, "On paper, both of those people would mark 'definitely recommend', and yet, they mean something completely different."
So what? What does it really matter if the projections are off because their methods are not nuanced enough? Well, according to Deadline it matters more than we realize, "Not only do tracking numbers affect how studios spend on marketing, but they are also looking at those numbers to help determine what kind of genre projects to greenlight. They are hiring directors and writers based on what looks likely to be successful."
Okay. So, this is surprisingly important to the way films get made. Presumably, the projections weren't always so unreliable. What's changed? Well, get ready to shake your fist in the air and yell, "Kids these days!" because social media and fickle millennials are the two culprits most often blamed for inaccurate tracking. A top distribution executive at a major studio told Deadline, "The new generation today has no problem multitasking, watching movies on the cell phone, and we, in the business, find that a little hard to take, so you find this generation is very fickle. They can multitask to the point of emailing, instant messaging, while they are watching movies - all these things. If you take a demographic like the young female audience, well, that's the toughest. They go in packs and text and email each other so many different ways. It's hard for them to agree to do anything."
Like I said, "Kids these days!" But how is that really connected to tracking agencies, "The tracking companies have not adjusted to the young and hipper generation, but they have to. I think that social media tracking is the way to do it and traditional media is very hard now because of TIVO. You have binge watching on these Netflix and HBO series. For whatever reason, I don't think they (the tracking companies) have taken the old media and traditional forms of research to online [...] They say they are, but I think they're making it worse somehow."
So, that's how the agencies would have been so wrong about teen-oriented films like Pitch Perfect 2 and The Fault in Our Stars, but how exactly did they fail with Jurassic World? The answer is partially down to simple statistics. Films which open between $15m and $50m are easier to predict because there are a lot of old films to compare them to. Anything which opens above $110m is harder to predict because only 24 films have enjoyed such a successful opening weekend. Furthermore, the tracking agencies don't include young kids under 12 in their samples nor do they account for families as a whole. That really hurt them with Jurassic World, a true four-quadrant blockbuster which Universal's distribution president Nicholas Carpou accurately described as not just "a drop-off movie for the kids" because "their parents want to see it too." Not helping matters any further is how those fickle millennials may tell a pollster one thing Thursday morning and then do something completely different by Thursday evening for late-night showings of a new movie.
To some degree, you can argue that box office performance can't truly be predicted. For the longest time, as E.T. producer Sid Sheinberg put it, Hollywood was in the business of "putting lightning in a bottle," meaning they made movies they hoped would catch on. Now, they're in the business of manufacturing opening weekends, but sometimes they are better at it than they realize or they recreate that old magic of catching lightning in a bottle. As one Universal insider told Deadline, "When you go into the weekend, you are armed with your expectations based on historical data, relying on movies released during the same time period as well as assessing different variables in the marketplace. But when the film gets a chance to be itself and grows through the weekend, you lose your historical data."
There are efforts to improve tracking as well as new companies which focus on social media, such as RelishMix, whose analysts charted the way Jurassic World exploded on YouTube and Twitter in ways quantitatively superior to Furious 7 during its' opening weekend. As such, they knew Jurassic World was going to beat Furious 7′ s debut of $147m, which had been a record for Universal Studios. RelishMix's lead social strategist Marc Karzen told Deadline:
"Surprise polling 500 or even 5,000 people about a film that they may or may not want to see doesn't correlate to box office well. It's a qualitative response of how much they want to see a movie (sequel) or if the marketing is memorable. But what's more powerful is how thousands and millions of people share trailers with their community, or not, on multiple social channels. That shows intent (to buy a ticket) and if the movie exceeds expectation, as Jurassic World did, the Internet and box office explodes."
We are still just talking about opening weekends, though. Last year's Best Picture winner Birdman joked Hollywood measures its entire worth in a single weekend's gross, but in truth most movies are lucky to ever actually turn a profit from their entire theatrical run. One studio distribution president vented to Deadline, "Since there are no secrets anymore, you also have the press writing and putting pressure on the studios on what the film is supposed to do. It puts everyone in a bad place. Tracking is the motivator and determining whether a movie is a success or not and to write a story that says, it might disappoint, well, what are they basing it on? They are basing it on tracking. But the press doesn't understand at times that we didn't get disappointed from an economic point of view. It's only a disappointment based on f***ing tracking, and the tracking is a f***ing disaster."
In the end, Jurassic World is the best-case scenario of a movie which demolishes expectations and sets records. However, there are those movies which fall short of expectations, like Melissa McCarthy's Spy, leaving the studios to tell the press, "No, we swear, we're really happy with this opening weekend," a defensive position partially forced on them by bad tracking.
Source: Deadline (about tracking), Variety, Deadline (about Jurassic World)
