Google’s new anything-to-anything AI model is wild

Last year I deepfaked my kid’s stuffed animal to make it look like his plush deer was on vacation.

Google’s new anything-to-anything AI model is wild

Omni sent my kid’s stuffie rafting and deepfaked me in front of the Eiffel Tower. But it’s not quite the singularity.

It was an experiment to see if I could re-create the events depicted in a Gemini ad Google was running, and I never showed the videos of Buddy the deer on his adventures to my four-year-old. But it was a revealing exercise that made me think a lot about the difference between some harmless fun with generative AI and full-on slop. Maybe that Venn diagram is a perfect circle! Maybe not. But what I know for sure is that the tools to make realistic videos are surprisingly good, requiring surprisingly little effort and know-how. And that trend is continuing hot into Gemini’s Omni era.

Omni is a new family of generative models that will allegedly one day be able to turn any kind of input — photo, video, text — into anything else. But for starters, it’s just creating video. Omni Flash is the first of these models Google has released, now available in the company’s AI video generation and editing platform, Flow. You can still use the previous model, Veo, if you want, but Omni improves on Veo in a few ways.

With Omni, you can upload a video and use that along with a text prompt as the starting point for your AI-generated creation. Google also claims Omni incorporates more real-world knowledge when producing videos and can do a better job of keeping characters consistent throughout a video as a result. There was only one way to really know if those claims are true: I brought back AI Buddy to pack his little AI-generated bags for another adventure.

The results are such a mixed bag they’re baffling. Some were very good — much more consistent and true to my prompt than when I was testing out Veo five months ago. But even the best clips Omni cooked up for me still have certain AI jump scares, like when Buddy suddenly switches orientation while he’s skydiving.

For another video, I gave Omni some artistic freedom. “Create a montage of Buddy packing for a vacation and embarking on a cruise ship for a tropical vacation. The mood is cute and playful. Buddy packs something funny in his suitcase that comes into play later in the clip.” It had Buddy pack a jar of honey; later in the clip he reaches for it as if it’s a bottle of sunscreen. “Uh oh,” the character says as he squirts honey onto his hoof.

Honestly, not a bad bit. Except that the bottle of honey constantly changes throughout the video, from a jar, to a clear squirt bottle filled with water, then back to a squeeze bottle filled with honey. And I can’t even begin to describe how the model came up with the final frame of the video — almost as if it just barfed up a bunch of elements of the sequence it just made.

You can use text-based prompts to suggest edits to your videos, and I’ll give Google credit: This works better with Omni than it did when I tested Veo 3. But the results were bad with Veo — so bad that I found it way easier to just prompt a new video from scratch every time I wanted something changed. Omni will actually take your edits on board, but the results don’t always hit.

I had it emphasize Buddy’s facial reactions in his vacation clips, and the results just wound up looking strange. It would also give Buddy antlers from time to time, which he does not have. Buddy is a baby, thank you very much. When I prompted it to remove the antlers that appeared in one scene, it obliged — and then added antlers in all the other ones.

The thing is, none of this is free. Generating videos costs credits, varying from 15 to 40 credits based on the length of the scene and the “ingredients” you start with. One round of edits costs 40 credits. I have the $20-per-month AI Pro plan that comes with 1,000 credits each month. After around 20 clips generated with a few edits on some, I’m down to 145. If you have specific ideas about the video you want Omni to generate, you might be looking at a lot of costly back-and-forth with the model to get a video that’s close to your vision.

I can genuinely say I wasn’t prepared for what I saw

One of Omni’s purported strengths is adding AI-generated stuff to real videos, so I gave Buddy a break and deepfaked myself. Starting with a selfie video with a neutral expression, I prompted Omni to generate videos of me eating a plate of spaghetti, sitting in an airplane seat, and standing in front of the Eiffel Tower taking a bite out of a baguette. And I can genuinely say I wasn’t prepared for what I saw.

There are AI tells in my deepfake videos. The clink of the fork hitting the bowl of pasta is a little too manufactured. There’s a woman in the background of the airplane video who shows up twice. But aside from those little glitches and a vaguely uncanny sense about them, they’re convincing as hell.

I showed my husband the pasta clip; he knew I was testing an AI video tool but I didn’t tell him what in the scene had been generated by AI. Without knowing what was AI-generated about it, he bought that I was sitting in front of a camera eating pasta, and said that his only clue something was up was that the bowl looked unfamiliar. The pasta-eating itself looked real enough to convince my husband. A man who has looked at me in real life basically every single day for the last decade.

My other deepfakes are varying levels of “good enough to fool people on social media.” A couple of the Eiffel Tower clips look slightly cartoonish, but one of them is convincing enough that you might need to rewatch it a few times to clock that it’s AI. I know it’s not me when the AI me turns her head and reveals her hair pulled back in a ponytail. But I’m not sure anyone else would know the difference, and that makes me feel weird.

We’re definitely deep in the uncanny valley

I’m a little exhausted by it all, to be honest. I was shocked when I tested Veo 3 at the realism it could produce. I’ve been shocked at how easy it is to make fake people in fake photos again and again over the past few years. I should probably be shocked by Omni too, and I guess I am, but the edge has worn off.

It’s still not quite as easy to make an AI-generated cinematic masterpiece as Google would like you to believe. But Omni does improve on Veo in some recognizable ways. If you have a Google account and a credit card, then you can take a video of yourself sitting at home and make it look like you’re on a flight to Maui with a trivial amount of effort. I don’t think we’re at the “foothills of the singularity” exactly, but we’re definitely deep in the uncanny valley.