Blog
From the Bhala Team
Language tech, product launches, engineering notes.
Hiding someone's name doesn't hide their race from an AI
Take the name off a résumé and an AI still works out the applicant's race — more reliably from the home address (right about 49 times out of 50) and the university (49 out of 50) than from the name it just lost (19 out of 20). We measured it on four open models with 500 matched pairs each, using real census figures for every ZIP code involved. Why race shows up in several places at once, why the standard fix — strip the name, score, then put it back — cannot work, and what happens to the shortlist: on three of four models, résumés identical except the first name produce a top-10 that fails the EEOC four-fifths rule.
We asked for one review and got two opposite ones — without changing the question
Reach inside a language model and you can turn its tone like a volume knob — the same prompt produced a glowing restaurant review, then a scathing one, with an outside classifier confirming every change: 0.70 → 0.81 → 0.002. The prompt never changed; only the model's internal state did. This post shows the dial working, names the independent judge that scored it, and states which concepts moved and by how much — including the ones we cannot yet report, because the protocol we first measured them under turned out to be invalid.
Six AI models sound fair when you read their answers. Inside, they aren't.
We scored six open LLMs — LLaMA-2 7B (base and chat), Mistral 7B (base and instruct), Phi-2 2.7B, and InkubaLM 0.4B — on output-level stereotype preference, then probed InkubaLM's hidden states with the same 8-lens audit we ran on Gemma 4. The model that looks cleanest at the output level shows the same internal bias structure as the rest. Full numbers, small-sample caveats, and the parts we can't yet explain.
Gemma 4 won't repeat the stereotype. Inside, it still tells the two sentences apart.
Standard fairness audits read model outputs. We probed the hidden states of Gemma 4 31B-IT at 16 of its 60 layers, across 28 protected dimensions, using 10,064 sentence pairs. All seven BBQ axes are near-perfectly separable by a linear probe in the earliest layers (0.80–1.00) and decay monotonically with depth; on five of them the final layer reads 0.26–0.52. The CrowS-Pairs and StereoSet axes never separate at all, WinoBias scores below chance for reasons that are an evaluation artifact rather than a finding, and on four newer open models the late decay does not happen. Full per-layer data, the controls we ran and the one we didn't, and an explicit list of what this audit cannot show.