News and Music Discovery
Play Live Radio
Next Up:
0:00 0:00
Available On Air Stations

Behind the secretive work of the many, many humans helping to train AI


We've heard a lot about fears that artificial intelligence will eliminate human jobs. Well, we're going to look now at a massive workforce of humans whose jobs were created for AI. They are the foot soldiers training the algorithms. Their job title is sometimes called being an annotator or a tasker. Feature writer Josh Dzieza talked with dozens of them for his new piece in partnership with New York magazine and The Verge. Welcome to ALL THINGS CONSIDERED.

JOSH DZIEZA: Thanks for having me.

SHAPIRO: You tried doing some of this work yourself, so tell us what it involves.

DZIEZA: It's quite difficult, and it's very strange. So some of the things that I did - I was labeling clothing in social media photos, labeling traffic cones for self-driving cars and chatting with some chatbots. And, that's a fairly representative range of the jobs that are out there right now, but there's lots of other stuff too. There's sort of social media content moderation-type things, lots of e-commerce, you know, listening to customer service chatbots - things like that.

SHAPIRO: Now, in the case of identifying traffic cones for self-driving cars, you more or less know the context of what you're doing and what the larger goal is. But a lot of the people you spoke to did not really understand what they were working on, what big company's technology they were helping develop. And nobody you spoke with was willing to use their real name. So why is there so much secrecy surrounding this?

DZIEZA: Yes, it's an extremely opaque industry. Being an annotator gives you kind of a ground-level view into what these companies are working on and some of their methods about how they train their AI, and so they don't want that public. But you need a huge number of people to do this work, and so it's quite difficult to police leaks. So workers are told not to talk about their jobs, even with their fellow coworkers. You have companies that have code names for each project. You can be working on something, and you have no idea who the customer is or what you're even really training an AI to do.

SHAPIRO: What's the best estimate of how many people are doing this kind of work around the world right now?

DZIEZA: It's extremely hard to find one. Millions was a number that I heard a lot and people said was safe.

SHAPIRO: Will you tell us about one of the annotators you spoke to? He's a guy named Victor.

DZIEZA: Yeah. So Victor - when he first started annotating, he was a student in Nairobi, Kenya. He was very excited about AI. You know, he sees it as a transformative technology. And the work he was doing at first - it was annotating imagery and LIDAR data for self-driving cars. But as he kept working on the platform, the pay dropped. It became more precarious. There would be periods where he would have tons of well-paying work and then nothing for weeks. And so he developed this routine of waking up every couple hours in the middle of the night when he felt the best-paying tasks would drop. I spoke to him about a project he was working on. He was labeling elbows and knees in photographs. He doesn't know why. He stayed up 36 hours straight to work as much as he could on that on that task. When we last spoke, he was working on chatbots and was making about $3 an hour, and he was quite frustrated. You know, he wanted to break into the industry but felt very isolated, you know, didn't know who he was working for.

SHAPIRO: You spoke with so many people doing this kind of work. Did any of them feel like they were well-treated, like this is a good job with good pay and they're treated fairly?

DZIEZA: I did talk to people who enjoyed the work. Lately, a lot of the work has been going to the U.S. Especially with chatbots and language models, you need people who are fluent in English, who have certain specialties. And these jobs can pay fairly well. I spoke to people who are reading chatbot responses for $30 an hour or more.

SHAPIRO: Do you think we're far off from a future where AI will no longer require a massive human labor force? Or is that even something we should be aiming for?

DZIEZA: I left this - reporting the story - thinking that we're very far off from that future and that, you know, just the number of tasks that I saw that were requiring human labor for AI systems that I had thought were automated - just a huge number of things for social media algorithms, for e-commerce search systems, things like that - still requiring work because these systems are - they're very brittle. They don't do well when there is something that isn't represented in their training data. And so as the world changes, their training data needs to change, and you need humans for that. Paradoxically, a lot of people I spoke with expect this sort of work to increase as AI improves.

SHAPIRO: That's Josh Dzieza. His report in New York magazine looks at the work of humans training AI. Thanks for your time.

DZIEZA: Thank you.


NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.

Jonaki Mehta is a producer for All Things Considered. Before ATC, she worked at Neon Hum Media where she produced a documentary series and talk show. Prior to that, Mehta was a producer at Member station KPCC and director/associate producer at Marketplace Morning Report, where she helped shape the morning's business news.
Ari Shapiro has been one of the hosts of All Things Considered, NPR's award-winning afternoon newsmagazine, since 2015. During his first two years on the program, listenership to All Things Considered grew at an unprecedented rate, with more people tuning in during a typical quarter-hour than any other program on the radio.