Data Annotation: The Human Labor Behind AI with Heather Mellquist Lehto, PhD

Jessica and Kimberly sit down with Heather Mellquist Lehto, PhD. 

Heather is a mathematician, anthropologist, former Harvard faculty, Vatican AI advisor, and founder of Guilded AI. They asked her to pull back the curtain on data annotation: the human labor that makes AI possible and one of the least visible, least understood, and most exploited parts of the entire industry. From pennies-per-task gig work to expert PhDs clicking through unpaid tests, they dig into who is actually building these models, what they are being paid, and why the workers creating billions in value are locked out of the wealth they generate. Heather shares why she got fed up with the recruiting playbook, what she is building differently at Gilded AI, and why treating workers well is not just an ethical argument but a data quality one.

Topics Covered:

• What data annotation is and why it still requires human expertise at every level of AI development
• The difference between data annotation and reinforcement learning from human feedback
• How workers go from labeling apples to annotating molecular structures and advanced mathematics
• Why the effective hourly rate for data annotators is much lower than advertised
• Scale AI, the $29 billion valuation, and the Department of Labor investigation
• How Guilded AI is structuring equity so annotators share in the upside
• Garbage in, garbage out: why worker treatment is a data quality issue
• AI chatbot vibe checks as expert vetting, and why that fails everyone
• The Gilded Age, guilds, and what banding together could look like
• Why the perfect cannot be the enemy of the good

Referenced in This Episode:

• Empire of AI by Karen Hao
• The Worlds I See by Fei-Fei Li
• Surveillance Capitalism by Shoshana Zuboff
• Rerum Novarum by Pope Leo XIII
• Guilded AI
• Scale AI and the Meta investment