The future of Hollywood isn’t feeding prompts into vanilla gen AI models

For all the noise that’s been made about how generative AI is poised to revolutionize the filmmaking industry, there haven’t really been any projects created with the technology that felt like the sort of entertainment people would pay to see. Most AI firms’ video models are still only capable of churning out short bursts of visually inconsistent footage. And some of Hollywood’s biggest AI partnerships have suddenly evaporated in ways that make it seem like studios might not be able to rely on the new technology coming out of Silicon Valley. For the most part, short-form video slop appears to be the only thing that major production houses are capable of cooking up with gen AI. But that could change if studios take notes from some of the experimental projects that debuted at this year’s Tribeca Film Festival.

The future of Hollywood isn’t feeding prompts into vanilla gen AI models

AI wasn’t just slop at this year’s Tribeca Film Festival.

Though it feels unlikely that gen AI will ever be able to whip up a compelling movie whole cloth, Tribeca featured a number of films that demonstrated how human artists can leverage the technology in compelling ways.

While none of the AI-powered movies that screened at Tribeca were as terrible as the video slop companies like OpenAI and xAI have polluted the internet with, some of the projects were prime examples of why generative content tends to feel so lifeless compared to human-crafted art. Roar — an animated short produced by Illuminai Studios — felt more like a disorienting montage of AI-generated clips rather than a cohesive piece of cinema. And Asteria Film Co.’s ChikaBOOM! lacked the visual and sonic polish that’s necessary for fast-paced fantasy about a magician in training to really pull you in.

Roar and ChikaBOOM!’s overall roughness seemed to be a reflection of the inherent technological limitations baked into their respective AI-forward production workflows. But other films, like Google DeepMind’s *Dear Upstairs Neighbors *and OpenAI’s Mauvais Soleil, showcased how it’s possible for filmmakers to avoid those challenges when gen AI is deployed with a bit more ingenuity.

Written and directed by Pixar veteran Connie Qin He in collaboration with researchers from Google DeepMind, *Dear Upstairs Neighbors *tells the story of an exhausted young woman who’s trying to go to bed. All Ada (Márcia Mayer, who also produced the short) wants is to get a couple hours of peaceful rest before she has to wake up and get back to work. But every time she begins dozing off, the cacophony of noise coming from her upstairs neighbors’ apartment jolts her awake and leaves her wondering what they could possibly be doing in the middle of the night.

To give Dear Upstairs Neighbors’ world a distinct style, He enlisted Pixar production designer Yingzong Xin, who painted concept art in Photoshop and on paper using acrylics. Those illustrations’ expressionistic aesthetic was key to bringing Dear Upstairs Neighbors’ fantastical story to life, but it also presented a unique challenge to DeepMind’s researchers. With most AI video generation models, the illustrations’ painterly style would be difficult to turn into visually consistent footage. But DeepMind’s engineers developed custom versions of Veo and Imagen that were specifically designed to give Dear Upstairs Neighbors’ artists the ability to fine-tune their outputs.

You can see how filmmakers have to work around some of gen AI’s more typical limitations.

Because the customized models were trained on Xin’s concept art, they could consistently generate shots that adhered to He’s vision for the project. The text-to-video models were great at reproducing certain stylistic details, like the way sound is visualized when objects interact with one another. But to really build Dear Upstairs Neighbors’ scenes in a way that would tell a cohesive story, the short’s creative team had to do things a bit more traditionally. By creating rough animations with Autodesk Maya (the industry standard for 3D rigging and VFX), Dear Upstairs Neighbors’ production team could ensure that scenes would unfold exactly how they wanted them to. And by feeding those roughs into Veo, the artists could create scenes that were more visually polished and ready to be further enhanced with additional stylized assets generated with Veo and Imagen.

More than any other film at Tribeca, *Dear Upstairs Neighbors *felt like a case study in how generative AI can be used as a bespoke tool that actually assists artists as they develop their ideas. The film’s entire workflow relied on human-made art and people making the kinds of nuanced creative decisions that text-to-video generators aren’t capable of on their own. It’s important to bear in mind that *Dear Upstairs Neighbors *would not be nearly as visually impressive if it had been produced with vanilla versions of Google’s various models. The models worked well for this particular short, but that’s to be expected for a project that’s also very much a commercial for Google’s technology.

*Dear Upstairs Neighbors *was a much more enjoyable watch than the films OpenAI brought to this year’s festival. Alice Gu’s semi-autobiographical drama *Smoked *used Sora to re-create the Palisades Fire, and Youssef Michraf’s Mauvais Soleil features a number of photorealistic scenes generated with OpenAI’s creative tools. Watching both films, you could see how their respective filmmakers had to work around some of gen AI’s more typical limitations. Wide shots in Smoked’s fiery scenes looked a bit cartoony, but the effect worked somewhat better in close-ups — which were filmed using a Volume-like setup — of a woman and her son trying to escape the blaze in their car. Most of Mauvais Soleil’s shots last for just a few seconds and the only speaking character is an unseen narrator, but the film’s story about how a man’s life is being warped by artificial intelligence makes those details feel like intentional artistic choices.

OpenAI’s presence at Tribeca was somewhat surprising given the company’s recent decision to shut Sora down entirely. Sora’s sudden shuttering is what led to OpenAI’s feature-length film Critterz not being able to make its debut at this year’s Cannes Film Festival. It seems like OpenAI might be pivoting away from video-focused applications of its tech, but there are still other players in the gen AI space building tools that filmmakers can use to realize their projects.

These workflows really only function well when they’re guided by human artists.

With just $2,000 spent entirely on computing costs, writer / director Ash Koosha was able to singlehandedly produce Dreams of Violets, a docudrama focused on the nationwide protests that have rocked Iran throughout the past year. Using Kling AI, Claude, Gemini, and Nano Banana, Koosha tells a fictionalized story about a group of people who find themselves trapped in an alley as police stalk the streets brutalizing civilians. The project took Koosha just a few weeks to finish by himself, and while it’s supported by a powerful narrative, it doesn’t break any ground visually.

Watching all of these films, I got the distinct sense that there is no future where studios are cranking out commercially viable projects by feeding prompts to gen AI models. That kind of content probably isn’t going to go away, but it’s not the kind of stuff Hollywood’s heavyweights would want to put their names on. What seems much more likely is bigger AI firms like Google partnering with studios to build bespoke models that are tailored to very specific workflows. And those workflows really only function well when they’re guided by human artists with very clear creative visions.