Lens 3: Lifecycle and Labor — Materiality and Extraction
Who does the work, carries the risk, or has no recourse?
- Slot: about 35 minutes in the morning, with a 15-minute group activity.
- Output: a lifecycle risk matrix.
- Worksheet: Lens 3 worksheet: lifecycle risk matrix.
- Slides: Lens 3 slide deck.
- Anchor sources: Crawford (2021), chapters 2 and 3; Bender et al. (2021).
The case describes workers who read texts about violence and sexual abuse. The page describes their working conditions. It quotes none of the texts.
Question
Who does the work, carries the risk, or has no recourse?
A generative AI system looks automatic. Behind it, people collect and label data, rate answers, moderate content, maintain hardware, and write the texts and images that the system learned from. This lens follows the lifecycle of a model and asks who works at each stage, who carries the risk, and who can object.
Core concepts
1. The lifecycle has six stages
The course uses six stages: design → data → training → deployment → evaluation → governance. The order is a convention for the worksheet. Evaluation happens before and after deployment, and governance applies at every stage. Each stage involves different people and different risks. Suresh and Guttag show that harm can enter at many points of the machine learning lifecycle, from the collection of data to the deployment of a model (2021). Weidinger et al. sort the risks of language models into six areas, from discrimination and misinformation to environmental and socioeconomic harms (2022).
3. Extraction concerns people who never agreed
Training data come from the work of writers, artists, photographers, and translators. Their consent was rarely asked. Crawford describes the collection of data at this scale as a form of extraction, next to the extraction of minerals and labor (2021, chs. 2–3). Copyright, consent, and provenance are therefore questions of labor too.
4. Recourse is unevenly distributed
A person who is harmed by an output can often not find out which data, which worker, or which decision caused it. Bender et al. warn that undocumented training data “perpetuates harm without recourse” (Bender et al. 2021, 615).
Evidence and limits
- Investigative journalism and interview studies document the conditions of outsourced data work (Perrigo 2023; Miceli and Posada 2022; Gray and Suri 2019).
- Developers’ own papers document that human feedback is part of the training (Ouyang et al. 2022).
- Most contracts between AI companies and suppliers are confidential. Single cases come from leaks, investigations, or interviews.
- Working conditions differ across suppliers, countries, and years. One case does not describe the whole industry.
- The companies dispute some details of the reported cases. The case below states where they do.
Case: safety labeling for ChatGPT in Kenya
This case comes from an investigation by TIME, based on interviews and documents (Perrigo 2023).
To make ChatGPT less toxic, OpenAI needed a detector for violent, hateful, and sexual content. The detector needed labeled examples. From late 2021, OpenAI contracted Sama, a San Francisco-based outsourcing firm, to label tens of thousands of text passages in Nairobi, Kenya (Perrigo 2023).
| Item | Reported fact |
|---|---|
| Task | Label text passages that describe violence, hate speech, and sexual abuse |
| Take-home wage | About USD 1.32 to 2 per hour, by seniority and performance |
| Workload | Workers reported 150 to 250 passages per nine-hour shift. Sama said 70. |
| Contracts | Three contracts worth about USD 200,000. OpenAI said it paid about USD 150,000, because the work ended early. |
| End of the work | Sama ended the work in February 2022, eight months early |
TIME interviewed four Sama employees who worked on the project. All four “described being mentally scarred by the work”. They had access to “wellness” counseling, but all four said that the sessions did not help (Perrigo 2023). OpenAI said that it works “hard to build safe and useful AI systems that limit bias and harmful content” (Perrigo 2023).
Why this case. The safety of a product for its users depended on work that exposed other people to harm. The users never saw this work. The case asks where the responsibility of a researcher who uses the product begins.
Activity
Build a lifecycle risk matrix. Work in groups of three or four, for 15 minutes, on paper or on a laptop.
- Choose the system. First choice: the system from one research-use description in your group. Use ChatGPT as in the ready-made case only when no description fits. (3 minutes)
- On the worksheet, fill one column for each of the six stages. For each stage, write who does the work, who carries the risk, and who has recourse. (7 minutes)
- Mark the stage where the risk is highest, and the stage where you as a researcher have the most influence. (2 minutes)
- Write one intervention for the stage where you have influence. (3 minutes)
This is the first choice. A use from your own research makes the analysis concrete.
- Choose the research-use description with the most open questions.
- Its author explains the use in two minutes: the task, the system, and what they do not know about it.
- Describe the use, not the data. Do not show confidential, personal, or unpublished research data.
- The column “who has recourse” often stays empty. Keep it empty and visible.
Use the safety-labeling case and analyze ChatGPT when no description in your group fits, or when the group prefers not to discuss its own use. The case documents one stage in detail: the labeling of data for a safety filter.
A sociology team plans to code 2,000 open survey answers with a hosted model. The matrix shows: at the data stage, the respondents never agreed to this use. At the evaluation stage, student assistants check the codes but get no credit in the publication. Intervention: ask the ethics board, keep a sample coded by hand, and acknowledge the assistants. This example is invented for teaching.
Output
A lifecycle risk matrix on one sheet for each group. For each of the six stages, it names who works, who carries the risk, and who has recourse. It marks the stage of highest risk, the stage of most influence, and one intervention.
Reading
Slides
Open the Lens 3 slide deck. Press S for the presenter notes.
Timing (35 minutes). Input on the core concepts, 10 minutes. The case, 5 minutes. Activity, 15 minutes. Plenary round, 5 minutes.
Prompts.
- “Which work in your own research is invisible in the final publication?”
- “If an output of this system harms someone, whom can that person contact?”
- “Which stage of the lifecycle does your institution control through procurement or rules?”
Link to the afternoon. The column “who has recourse” often stays empty. Keep the empty cells visible. Panarchy asks at which level someone could fill them.
Difficult moments. Some participants work with outsourced labor themselves, for example through crowdsourcing platforms. Invite them to share practice, not to defend it. Point to fair-pay guidelines of their discipline where they exist.
Fallback. Start with the group’s own example. If no description fits, use the case. Groups that finish early can add a seventh column for “after deployment”: who maintains the system, and who deals with complaints.