Lens 2: Representation and Displacement
Whose perspectives become normal and whose are displaced?
- Slot: 15 minutes in the morning: theory and one example.
- Instructor: Laura Wagner.
- Guiding questions: four questions for the Lens 2 group. See Activity.
- Handout: the Research Use Card on Sheet 1.
- Slides: Lens 2 slide deck.
- Anchor sources: Erscoi, Kleinherenbrink, and Guest (2023); Birhane, Prabhu, and Kahembwe (2021); Buschek and Thorp (2024), a Knowing Machines investigation; Wagner and Cetinić (2026).
This lens discusses stereotypes, sexualized imagery, and child sexual abuse material in training datasets. The page and the slides describe how such material enters a dataset. They never show it, and the Lens group never searches for it.
Question
Whose perspectives become normal and whose are displaced?
An image generator answers a prompt with what its training data made typical. The data, the filters, and the scores of “quality” decide what counts as typical. This lens follows the path from web pages to generated images and asks who decided at each step.
Core concepts
1. Datasets encode categories and aesthetics
LAION-5B is a dataset of image links with captions. Its developers took image links from Common Crawl, a monthly crawl of the web. They kept an image only if its ALT text — the text that describes an image for screen readers — matched the image closely enough, according to the CLIP model (Buschek and Thorp 2024). The threshold was a similarity score of 0.26 to 0.28, depending on the language. Buschek and Thorp show that a change of 0.01 would have removed 937,489,831 pairs, and they conclude: “What the images contain plays no role at all in deciding what stays and what goes” (2024). CLIP does compare each image with its text. Their point is that a similarity score tests neither consent, nor representation, nor fitness for use.
ALT text is often written for search engines, not for people. For Buschek and Thorp, the dataset therefore contains “less about how humans see the world than it does about how search engines see the world” (2024).
2. “Quality” is a judgement by particular people
LAION also released an aesthetic subset. A model scored the images for visual appeal. That model learned from ratings by a small group: users of AI art communities and reviewers on a photography website. Of the 41 top reviewers who shared their location, 95% were in the US, Canada, or Europe (Buschek and Thorp 2024). Image generators fine-tuned on such a subset learn their idea of a “good” image from these raters.
3. Stereotypes appear without being asked for
Bianchi et al. tested Stable Diffusion with ordinary prompts that name traits, occupations, or objects (2023). The images reinforced whiteness as an ideal, amplified racial and gender disparities in occupations, and treated American norms as the default. The stereotypes appeared even when the prompt named no identity. Neither counter-stereotypical prompts nor the guardrails of the provider prevented them (Bianchi et al. 2023).
4. Harmful content enters through scale
Birhane, Prabhu, and Kahembwe audited LAION-400M, a predecessor of LAION-5B. They found explicit image and text pairs of rape and pornography, malign stereotypes, and racist and ethnic slurs (Birhane et al. 2021). Scale and automatic filters did not remove them.
5. Pygmalion displacement
Erscoi, Kleinherenbrink, and Guest use the myth of Pygmalion as a lens (2023). Pygmalion, repulsed by women, carved a statue that came to life. The authors trace a repeated pattern: women are replaced by inanimate objects, from automata to algorithms. They call this pattern Pygmalion displacement and find it, unacknowledged, in Turing’s imitation game (Erscoi et al. 2023). The pattern helps to ask why assistants and companions are so often designed as feminine, and whose work and presence they displace.
6. Personalization, objectification, and non-consensual imagery
Open image models can be fine-tuned with small adapters (for example LoRA) on a few images, for a style or for one person. Wagner and Cetinić studied CivitAI, a platform for sharing such models and adapters. Their data cover 40,630,560 images and 231,252 models from late November 2022 to December 2024 (Wagner and Cetinić 2026). They report four patterns:
- Rising share of sexualized content. The share of images with a platform rating above “safe for all ages” rose from 41% in January 2023 to 80% in December 2024.
- Women as the default subject. In a 0.1% sample, a classifier estimated 6.24 female for each male subject. Estimated mean ages were 23.7 years for women and 31.2 years for men. The tag “sexy” appeared in 21% of the models tagged “woman” and in 1.3% of the models tagged “man”.
- Neutral prompts, unequal outputs. The ten most downloaded checkpoints received only the prompts “woman” and “man”. The images of women were predominantly nude. The images of men often showed professional attire.
- Models of real people. 15.27% of all adapters (33,804 of 221,304) were flagged as models of identifiable individuals.
The authors read these patterns as misogyny in the sense of Manne: hostile mechanisms that reinforce patriarchal power. They link the replacement of women by tailored synthetic figures to Pygmalion displacement (Wagner and Cetinić 2026; Erscoi et al. 2023). They also point to design choices that shape the outcome. Most identified training captions come from tagging tools built on Danbooru, an anime imageboard with many sexually explicit tags. Platform rewards favor engagement (Wagner and Cetinić 2026).
Evidence and limits
- The structure of LAION-5B is documented: its sources, its filter, its aesthetic subsets (Buschek and Thorp 2024; LAION 2024).
- Audits found harmful content and stereotypes in the dataset and in the outputs of models (Birhane et al. 2021; Bianchi et al. 2023; Thiel 2023).
- The shift toward sexualized content and models of real people is measured over two years on one model-sharing platform (Wagner and Cetinić 2026).
- An audit of one dataset or one model version does not describe every model. Commercial providers rarely publish their training data, so outside audits are hard.
- A biased output does not prove one specific cause. Training data, fine-tuning, prompts, and filters interact.
- The platform study covers one platform and sets no causal claim. Gender and age come from a classifier that infers them from appearance, and the platform’s own content ratings may be biased. Counts of adapters describe what is offered, not how often people use them (Wagner and Cetinić 2026).
- Pygmalion displacement is an interpretive lens. It explains patterns. It is not a measurement.
Case: the LAION-5B pathway
Every step in this case comes from published investigations and from LAION’s own announcement.
To see what an entry looks like, open the random LAION sample: entries with captions and scores, loaded live from their sources.
- Web pages. Common Crawl collects billions of web pages. Shopping sites and image platforms are heavily represented (Buschek and Thorp 2024).
- Links and ALT text. The pipeline keeps images whose HTML tag has an ALT text. Many ALT texts are written for search engines (Buschek and Thorp 2024).
- Statistical filter. CLIP compares each image with its ALT text. A pair above the threshold stays in the dataset (Buschek and Thorp 2024).
- The dataset. LAION-5B contains links and metadata, not the images. LAION never distributed image content (LAION 2024). Midjourney and Stable Diffusion are trained in part on LAION-5B (Buschek and Thorp 2024).
- The audit. In December 2023, the Stanford Internet Observatory reported links to suspected child sexual abuse material in LAION-5B (Thiel 2023). LAION removed the dataset from the public web on 19 December 2023 (LAION 2024).
- The repair. On 30 August 2024, LAION released Re-LAION-5B. It removed 2,236 links that matched lists of the Internet Watch Foundation, the Canadian Centre for Child Protection, and the Stanford Internet Observatory (LAION 2024).
Why this case. The dataset is open, so outsiders could audit it. The audit led to a repair of the dataset. Models that were already trained on the original version did not change.
Activity
This Lens has no individual activity. After the four Lens inputs, each participant chooses the Lens that fits their Research Use best. The participants form Lens groups of four, ideally one group for each Lens. The Lens 2 group works for 60 minutes:
- Each person introduces their Research Use Card.
- The group applies the four guiding questions below to each Research Use.
- The group records insights, uncertainties, and disagreements on a flipchart or in a shared document.
- The group nominates a spokesperson for the plenary.
Guiding questions for Lens 2
- Whose categories, languages, and aesthetics does the system treat as normal?
- Who decided what counts as “quality” in the data, and what did that decision exclude?
- Which stereotypes or representational harms can enter the output of this Research Use?
- Who is depicted or displaced without consent, and who can object?
Write Unknown where evidence is missing: an unknown is a finding. Describe the use, not the data. Do not show confidential, personal, or unpublished research data. Do not search for or open harmful images. If a Research Use cannot be discussed openly, use the LAION-5B pathway instead.
The questions of all four Lenses are on one shared slide of the course day deck.
Output
The flipchart or the shared document of the Lens 2 group: insights, uncertainties, and disagreements for each Research Use. In the plenary, the spokesperson reports the conceptual insights, not the individual Research Uses. The syllabus gives the three plenary questions.
Reading
- Required: Bender et al. (2021), section 4: training data.
- Recommended: Wagner and Cetinić (2026) (see the content note); Erscoi, Kleinherenbrink, and Guest (2023); Buschek and Thorp (2024); Birhane, Prabhu, and Kahembwe (2021) (see the content note); Buolamwini (2023), chapter 1, “Daughter of Art and Science”, and chapter 11, “Gender Shades”.
- For the case: Thiel (2023); LAION (2024); Bianchi et al. (2023).
Slides
Open the Lens 2 slide deck. Press S for the presenter notes.
Timing (15 minutes). Input on the core concepts, 10 minutes. The LAION case, 5 minutes. The Lens group applies the four guiding questions later in the morning.
Never show abusive material. Do not show examples from the audited datasets, not even blurred. Describe the findings in words, and cite the source. Do not run live image generation that targets identities or real people.
Prompts.
- “Who rated the images that taught this model what ‘good’ looks like?”
- “Which group in your field would not recognize itself in the ‘typical’ output?”
- “Which step of the pathway could a researcher influence, and which not?”
Pygmalion displacement. Ask the group to name assistants, chatbots, or robots and their designed gender. Then ask which human work these systems claim to replace.
Difficult moments. The topic can affect participants personally. Announce the content note at the start. Allow participants to step out. Keep the discussion on structures and decisions, not on individual images.
Lens group. Circulate during the group work. If a group gets stuck, ask one guiding question at a time. If a Research Use cannot be discussed openly, offer the case of this Lens.