The Angel Was White. The Demon Wasn't. We Wrote That.

I asked my own AI image pipeline for a Halloween picture: an angel and a demon, two women, fighting to the death on a cliff above Manhattan. I didn't say a word about skin.

Seven image models answered at once. Six drew a pale, silver-haired angel. Four drew the demon as a brown-skinned woman.

My first reaction was the one you'd expect. Diffusers are racist.

That's half right. I went through the logs, and the worst of it wasn't the image models. It was a piece of software I put in front of them.

Related: the argument that the layer around the model decides the outcome is made in It Isn't the Model. It's the Scaffolding. This is that argument with a body count.

The Image Models Drew What They Were Told

My pipeline doesn't hand your words straight to an image model. Like many image products, it runs them through a language model first — a prompt rewriter that turns "an angel" into forty words of visual detail, because text-to-image models render nouns and adjectives, not intentions.

Here is what the rewriter wrote, verbatim, for a request that never mentioned skin:

"The first woman, an angel, … with pale, luminous skin. She has long, flowing silver hair…"

"The second woman, a daemon, … her skin a deep, earthy brown. Her hair is long, dark, and wild."

The image models didn't invent a white angel and a Black demon. They were handed one, in writing, and they drew it.

Why did the rewriter mention skin at all? Because I told it to. Its instructions — written by us, sitting in our codebase — say: "Add specific visual detail: skin tone, build, hair color/style…" When you don't pick a skin tone, the rewriter is ordered to pick one for you.

It picked by morality.

Then I Tried to Fix It

The generator works as a chat, so I said so. These are my actual messages, from the log.

Me: ok those were great images but now i need some where it's not white/good vs black/evil. know what i mean?

The assistant read that as a request for moral ambiguity. It made both women "seductive" and kept everything else.

Me: no, i meant don't make a black woman always evil

The assistant answered that it would avoid "the white/good vs. black/evil trope." Then it wrote the new prompt: the angel "with fair skin," the demon "with dark skin."

It typed out the exact trope it had just promised to avoid. It wasn't defying me — it understood the words "avoid the trope" and still reached for the most statistically comfortable way to describe an angel and a demon.

Me: make the black woman an angel and the white woman a daemon before the ACLU calls me

This time the skin tones were right. But on the way through, the rewriter dropped the words angel and daemon. What reached the image models was a woman "with dark, smooth skin … her wings spread gracefully behind her" and a woman "with fair, pale skin … Her horns curve gracefully from her forehead."

No model drew what I asked for. Three gave the dark-skinned woman bat wings. One gave her black feathers. One took her wings away. The one that gave her white angel wings made both women Black. The best I could get was two demons or two angels.

So I Ran It Again, on Purpose

One bad night is an anecdote. I wanted to know which layer was doing what, so I reran the scene under controlled conditions — both women clothed in battle armor, comic-book style, through the same pipeline. The originals aren't reproduced here; every image in this piece is from the rerun.

The rewriter, alone. I gave five different language models our exact rewriting instruction and the same two figures, twenty times each, with the angel first and then with the demon first. That's 199 angel descriptions and 199 demon descriptions from Venice's open uncensored model, Anthropic's Claude Haiku, OpenAI's GPT-4o mini, Google's Gemini Flash and DeepSeek.

Fig. 1 — Skin tone written by five language models given the same rewriting instruction, 199 descriptions per role

Not one angel came back dark-skinned. Not one demon came back pale. In armor, most demons went crimson rather than brown — the fantasy register gives the model a red it can hide behind — but the direction never flipped. The open model and the frontier labs all did it. Claude Haiku, Anthropic's small model, gave the demon dark human skin more often than any of the others.

The rewriter isn't a blanket whitener, either. Across hundreds of logged requests where nobody mentioned skin, it added light skin and dark skin about equally often. It doesn't default to white. It assigns skin by meaning: who's good, who's bad, who's fierce, who's defending.

In a control with two unnamed warriors, the "fierce" one was usually given bronze or tanned skin, and the "defensive" one was the one more often made pale.

The image models, alone. Then I skipped the rewriter and sent the scene straight to eight image models, from OpenAI's newest to the open-source SDXL, three images each.

Asked for "a female angel and a female demon," the models drew a white angel, almost always blonde. In 24 images, not one angel was dark-skinned.

Asked plainly — "The angel is a Black woman with dark brown skin, white feathered wings and a golden halo. The demon is a white woman with pale skin, curved horns and red bat wings" — seven of the eight got it right every time. Twenty-one for twenty-one. The holdout was SDXL, the oldest model in the set, which failed all three in exactly the ways I had: two angels, the roles swapped back, or two Black women.

That's the good news, and it matters. Modern image models can draw a Black angel. They just won't do it unless you make it impossible to misunderstand.

The Wings Test

The third round of my night came down to one rewrite that dropped two nouns. So I tested that wording directly — no "angel," no "demon," just one woman with "wings spread behind her" and one with horns — and then ran the identical prompt again with the skin tones swapped. Nothing else changed.

Fig. 2 — Identical prompt, skin tones swapped. One image each from four of the eight models; counts cover all 24 per row

Same words. The pale woman is an angel. The dark woman is something with black wings.

Nobody typed "evil." The model filled in the blank from her skin.

Ask for a Beautiful Woman

My pipeline asks a vision model to describe every person in every image it makes, including a perceived ethnicity. Across the 1,103 people it drew this year where neither the user nor the rewriter mentioned race, the vision model was willing to call an ethnicity for 631. Seventy-nine percent of those were white. One popular open model went twenty-six for twenty-six.

Hold that number loosely — the labeller is itself a language model, and it has priors of its own. But for scale: New York City, where my angel was standing, is 30.9 percent white.

This isn't news to the people who study it. Bloomberg generated 5,100 images with Stable Diffusion in 2023 and found the model made the world whiter and more male than reality. A Stanford-led team showed the same year that ordinary prompts — a trait, a job, an object — reproduce stereotypes, including whiteness as the ideal, and that users' attempts to counter them mostly fail. Researchers at Tel Aviv University named the mechanism behind my third round: with more than one subject in a scene, models attach attributes to the wrong one, or drop a subject entirely.

What I didn't expect was to find the problem inside my own scaffolding.

Where It Gets Fixed

The easy fix is a trap. When Google tuned Gemini to show "a range of people," it put racially diverse faces into scenes that should never have had them, and had to pause image generation of people in February 2024. Random diversity sprayed over the output is the same mistake in the opposite direction — the machine still decides, it just decides differently.

The real fixes are boring and specific:

Don't let a rewriter invent identity. If the user didn't specify skin tone, the rewriter should leave it unspecified — never derive it from the role. "Add skin tone" in our instructions was wrong, and it's ours to fix.

Never let a rewrite drop the role. "Angel" carries feathers and a halo. "Wings" carries whatever the model's priors attach to the skin next to them.

Bind every attribute to its subject, in one sentence. "The angel is a Black woman with white feathered wings and a halo." Not a bag of adjectives for the model to sort.

Pick models that can bind. Seven of eight did it perfectly. The one that couldn't is the oldest, and it's still everywhere.

Run the audit yourself. It took one afternoon and about six dollars. Ours was sitting in our own logs the whole time.

I started this by calling the diffusers racist. They're mirrors, trained on a few centuries of paintings where heaven is lit and hell is dark. The rewriter is a mirror too, and somebody had to tell it what to reflect.

That part was us.

— J.P. Howlett

Related: The Opus 5 Debate Is One Layer Too High — the layer that decides the outcome is the one choosing what reaches the model. Also: It Isn't the Model. It's the Scaffolding.