Tech companies desperately want to film you doing chores

This week, an AI training startup called Shift said it would clean New Yorkers’ homes for free. It has plans to expand into other cities as well, including London, and looking around my flat, I get the appeal.

Tech companies desperately want to film you doing chores

Startups are paying people for the real-world data needed to train their robots.

But there’s a catch. There’s always a catch.

In exchange for the cleaning, Shift wants footage of its cleaners at work: scrubbing dishes, wiping counters, dusting tables, mopping floors. It wants everything. Video of all the boring domestic labor we’d happily outsource if we could — and that robotics companies are racing to teach machines to do so they can sell us something to do it for us.

That’s harder than it sounds. Unlike chatbots, image generators, and other AI tools that have exploded in recent years, robots have to deal with the physical world. That means understanding space, motion, force, friction, weird shapes and materials, awkward lighting, and everything else that humans — and other organics — tend to grasp instinctively. It’s why things that are generally easy for us, like folding clothes, picking up an apple, or pouring a glass of water, have proven so maddening for roboticists to codify.

Teaching machines to do those things takes data. Lots of it. Text, images, and videos could be easily scraped from the internet at an industrial scale. And they were, often without compensating the people who made them. The physical world is harder to scrape, and harder still to scrape quietly without paying for it. This means access to high-quality data is a massive bottleneck for companies developing physical AI. It’s a lucrative opportunity, so companies like Shift are getting creative.

They’re not alone. In India, recent reporting revealed that home services platform Pronto has been using clients’ homes as a source of AI training footage for chores like cooking, cleaning, and laundry. Pronto says it only records footage if customers explicitly opt in — it’s not clear what customers get in return, other than a copy of the footage — but the practice still set off a wave of backlash in the market, with rival startups insisting they have never recorded inside homes to train AI and have no plans to do so.

Other startups are focused on trying to scale data collection. Silicon Valley-based Human Archive, for example, hopes to partner with companies like Pronto and have gig workers record their activities using not-so-stylish camera caps. The hats collect footage from the wearer’s point of view, exactly the kind of “egocentric” or first-person data robotics companies need to teach machines how people navigate physical space. Shift, meanwhile, also taps consumers directly, and claims to have paid tens of thousands of people across 15 countries to record their activities through its app.

Some companies are skipping useful work altogether. Instead, workers are paid to complete the exact same physical tasks again and again while cameras and sensors can capture every movement. Such staged data farms are designed to turn rote physical activity — folding towels, picking up cups, carrying boxes — into AI training material valuable enough to justify paying people to create it.

And some data is generated by robots already out in the world. Despite the hype, true automation is still a long way away — hence the need for all this data — but companies are keen to ship products anyway. They’ll use data from customers’ homes to improve the product. Many companies rely on remote workers to step in when the robots inevitably get stuck. They’ll use that data too.

Of course, the act of trading data for something of value is not new. Companies have been offering discounts, convenience, and free services in exchange for access to your data for years, from loyalty cards and cookies to dashcams, insurance apps monitoring how people drive, and that heinous smart TV that’s always showing ads.

What’s new is the kind of data companies are willing to pay for. For now, that means maybe letting a human clean your home in a snazzy hat for free so that, eventually, a company can sell you a robot to do it instead.