Bespoke AI models are the next big thing in filmmaking

Though many AI boosters have convinced themselves that the technology can spit out films and television series whole cloth, claims of Hollywood being cooked feel very premature when you see what people are making with the most popular image/video models on the market. Models like Sora, Veo, and Runway just do not seem all that great for entertainment production.

But we’re starting to see more AI firms building a new breed of generative model — ones that are designed to address creatives’ needs throughout the development process while also avoiding issues like potential copyright infringement. What really sets these models apart from their peers, though, is the way they can purportedly be customized through training that turns them into bespoke tools that are tailor-made for every project.

Customizability was one of the major points that Netflix emphasized last week when the company announced that it is absorbing InterPositive, an AI startup founded by Ben Affleck in 2022. Though Netflix has not disclosed how much it paid for InterPositive, Bloomberg reports that the figure could be as much as $600 million. While Netflix productions have used gen AI before, the acquisition was notable because the streamer publicly highlighted its plans to make the technology a foundational part of its business. Netflix is presenting InterPositive’s AI as a tool that’s designed to “empower” filmmakers rather than eliminate them from the equation.

In a statement about the acquisition, Affleck explained how InterPositive’s team filmed “a proprietary dataset on a controlled soundstage with all the familiarities of a full production” that serves as the basis for the company’s core model.

“I wanted to build a workflow that captures what happens on a set, with vocabulary that matched the language cinematographers and directors already spoke and included the kind of consistency and controls they would expect,” Affleck said. “The results of this foundational work were deliberately smaller datasets and models focused on filmmaking techniques — rather than performances — creating tools that artists can use, control, and benefit from.”

In a workflow built around this tech, Netflix can create unique versions of InterPositive’s model by training it on dailies from in-progress shoots. Filmmakers can then use those project-specific models to generate and manipulate different kinds of visual elements later on in the postproduction process. Netflix says that these models can help directors tinker with a specific scene’s lighting, edit out unwanted details like prop rigging, or replace backgrounds entirely. And because the models are trained on raw footage from the movies or series they’re being used for, their outputs can (purportedly) match a filmmaker’s creative vision with ease.

This all sounds impressive on paper, but it presumes that InterPositive’s core models were trained on enough production scenarios to be able to generate outputs that align with whatever kinds of scenes filmmakers think of. Part of what makes this tricky is that there are no set standards for things like Good Lighting™ that can be unilaterally applied to every single kind of movie or series that Netflix might release. This might be why InterPositive’s models need to be trained on dailies before they can churn out anything that might be useful during postproduction. But it’s easy to see why a pitch like this would appeal to a studio looking to put out more projects while keeping costs down.

Affleck’s description of InterPositive sounds a lot like the basic idea behind Asteria, Bryn Mooser’s AI-forward studio that is currently producing Natasha Lyonne’s upcoming feature about a virtual reality game. Like InterPositive, Asteria’s flagship product is a proprietary gen AI model that can be customized by being trained on datasets consisting of a client’s original art. Asteria also just announced that it is releasing Continuum Suite, an AI-powered operating system that analyzes scripts to generate a complex database containing information about characters, scenes, storyboards, schedules, and budgets.

Asteria’s big selling point is that its vanilla AI model is “ethical” because its core dataset is composed of material that the company has licensed. But whereas InterPositive’s models seem more focused on tweaking details, Asteria’s have been used to generate things like complete characters and background objects that share an aesthetic derived from the model’s dataset.

This makes Asteria’s tech ideal for filmmakers who want to flesh out their projects with things that all look like they were designed by the same artistic team. In theory, it also helps studios prevent their partners from deploying gen AI in ways that could lead to lawsuits over whether someone has stolen IP. Asteria and InterPositive both see their products as tools that can help speed production timelines up without spending more money, and that framing seems to be the main thing that’s driving more traditional studios to hop on the AI bandwagon.

Compared to Netflix, most other production houses haven’t been quite as open about their interest in and experimentation with AI. But you can see the industry shifting in a pro-AI direction in things like Adobe’s recently announced partnership with multiple studios to develop “IP-safe” models that can be used across the company’s larger suite of production tools. What’s harder to pinpoint here is how, if at all, human artists will benefit from this shift.

Creating more things faster and more cheaply can help studios (and their executive leadership) boost profits. But those things don’t inherently lead to creative workers keeping their jobs, receiving bigger paychecks, or having more time outside of the office. As much as these newer AI companies talk about “empowering” creatives, they seldom go into detail about what that empowerment actually looks like. And until they do (or can), we should all keep looking at their products with some skepticism.