The Biggest AI Debut In Hollywood

“The Wizard of Oz at Sphere” debuts just off the Las Vegas Strip on August 28, viewers will get to enjoy the 1939 film classic in a way that its creators likely never imagined.

The film has been upgraded to cover a massive 160,000 square foot wall of LED panels that stretches across three football fields, wrapping around the audience and soaring 22 stories high, while 750 horsepower fans whip up wind and debris to mimic the tornado.

The $104 or more per seat experience is truly more than it seems.

“The Wizard of Oz” represents one of the most important collaborations between a studio and a tech company, utilizing artificial intelligence to create a groundbreaking media experience.

Reuters chatted with nine individuals, including key players directly involved in the project and top experts from the entertainment industry, who shared the story behind a project that some seasoned professionals view as a possible turning point for AI tools in Hollywood.

“This definitely marks a significant milestone in the collaboration between AI and human creativity”, said Thao Nguyen, an agent for immersive arts and emerging technologies at CAA. “I believe it will set a standard for how we can reimagine culturally important media”.

Bringing Dorothy and the Wicked Witch to the enormous Sphere, a globe shaped entertainment venue equipped with cutting edge technology, took two years and united its creative team, Warner Bros Discovery executives, Google’s DeepMind researchers, academics, visual effects artists, and over 2,000 people in total.

The project developed amidst significant concern about AI’s effect on jobs in Hollywood and a strong wish to maintain human creativity.

Initially, some visual effects companies that were approached to participate in the project turned it down because they weren’t allowed to use AI at that time.

Reaching this point required the approval of Warner Bros Discovery CEO David Zaslav, along with his studio heads and legal team, who set the rules for AI usage.

“Wizard of Oz at Sphere” utilized archival materials from the original film, such as set blueprints, shot lists, publicity photos, and film artifacts, in addition to around 60 research papers to achieve a resolution that was ten times better than previous efforts.

“We had to rethink the cinematography, we had to rethink the editing, and we had to accomplish all of this without altering the experience”, explained Ben Grossmann, who managed the visual effects for the project.

Instead of using AI to eliminate jobs, they aimed to leverage it to revitalize a timeless tale and craft new experiences from existing intellectual properties.

The initiative kicked off in 2023 when Sphere executives were brainstorming which project could stretch the technological limits of the venue that had previously hosted U2 and Darren Aronofsky’s ‘Postcard from Earth’.

Carolyn Blackwood, the head of Sphere Studios, mentioned that “The Wizard of Oz” quickly became a favorite choice for the Sphere’s massive screen.

It was a great chance to reintroduce this classic film to a new audience, immersing them in L. Frank Baum’s enchanting world.

The team symbolically selected a timeless film that was a technical wonder for its era.

Although it wasn’t the first to use Technicolor, the striking shift from sepia to vibrant color in “The Wizard of Oz” was a significant moment in cinema history.

James Dolan, CEO of Sphere Entertainment, along with creative partner Jane Rosenthal, who co-founded the Tribeca Film Festival and is a well known film producer, had bigger plans than just digitally remastering a classic.

Rosenthal brought in Hays to collaborate with Google as a technical partner.

Dolan reached out to Warner Bros Discovery CEO Zaslav, who has been a friend and business partner since the early cable TV days, to suggest bringing “Oz” to the Sphere.

“I had just visited the Sphere with a friend and was truly amazed”, Zaslav shared.

Before handing over one of the most significant entertainment assets in the world, Warner Bros established strict guidelines.

Google was allowed to train its generative AI models on each major actor to replicate their performances, but the data would still belong to the studio.

None of the training data for “Oz” would be used in Google’s public AI models.

“A key factor in launching this project was establishing a secure space for experimentation”, Grossmann explained.

Initially, the visual effects team attempted to enlarge images using CGI, which would have produced photorealistic animated versions of the characters.

However, this method was turned down as it would compromise the authenticity of the original performances.

“AI became our last option since we couldn’t really achieve it any other way”, Grossmann stated, whose Los Angeles studio, Magnopus, has worked on photorealistic computer animated films like Disney’s “The Lion King”.

AI improved the resolution of tiny celluloid frames from 1939 into ultra high definition images. It brought back details, such as the freckles on Dorothy’s face or the burlap texture on Scarecrow’s face, which were hidden by Technicolor’s process.

AI also assisted in “outpainting” on screen images to fill in gaps caused by camera cuts or framing, like when it took a close up of the Tin Man chopping a door of the Witch’s castle with an axe to rescue Dorothy and completed the image of the woodman.

It took months of tweaking and the brainpower of Google’s DeepMind team to boost consumer grade AI tools, allowing them to produce sharp images with the Sphere’s 16K “super” resolution.

Musicians went back to the original sound stage to re-record the entire film score, making the most of the venue’s 167,000 speakers.

The vocal performances by Judy Garland and the other actors remain untouched.

Entertainment writer Joshua Rivera described it as “an affront to art and nature”.

“None of these critics have actually seen the film or grasp what we’re trying to achieve”, Rosenthal stated.

During a private midnight screening for Reuters, Grossmann provided a sneak peek of what’s ahead.

Some adjustments are quite subtle, like when Uncle Henry is by the front door while neighbor Almira Gulch is demanding Toto.

AI helps to reposition the performer, who is mostly out of sight, back into the frame, creating a broader view to fill the Sphere’s large viewing area.

Other changes are designed to bring the filmmakers’ vision to life in ways that just weren’t possible 86 years ago.

When Dorothy and her friends meet the Wizard in the Emerald Throne Room, a massive 200 foot high green head towers over the audience, with bulging eyes and a booming voice, making for a much more striking image than the original portrayal of an actor in green makeup projected on smoke.

“Every time we made a change, it was to ensure the audience could feel what Dorothy was feeling firsthand”, Grossmann explained.

The synchronized physical effects bring a whole new level to the experience.

Flying monkeys will dive into the Sphere as 16 foot long helium filled creatures controlled by drone operators, just one of the many Four D effects.

The outcome is a blend of cinema, live performance, and immersive VR.

“I believe this will transform how people perceive entertainment and experiences”, Grossmann remarked.