Bhala research / programmable embeddings

Named operators on meaning — measured, not asserted.

Bhala's embedding space is built so you can name a concept — a bias, a protected trait, a tone — and act on it: detect it, subtract it, and verify exactly what changed. Our strongest result is that removing one concept disturbs the others 4–6.5× less than standard methods. This page lays out the operators that hold up under testing, the ones that don't, and the evidence for both.

01 — New capabilities

Naming a concept, then acting on it

Current AI models are black boxes: you feed in text and get back a vector or prediction, but you cannot name the change you want and apply it as an operator. You cannot extract the bias component, subtract it, and verify it is gone. You cannot tell the model “apply formal register” without retraining.

Bhala's embedding space changes the primitive. The representation is structured so that named operators act on it: compose two meanings, invert a transformation, subtract a demographic direction and verify the result. These operators are learned components with their own training objectives — not arithmetic post-hoc on a frozen space.

Steerable models

Control outputs at inference by composing named operators — sentiment, formality, register — without retraining or RLHF.

Auditable AI decisions

Every prediction comes with a verifiable record of which operators were applied. Regulators get evidence, not post-hoc explanations.

Cross-lingual transfer without parallel data

Operators learned in one language transfer algebraically to others. A compact model trained on Zulu beats GPT-4o on Swahili intent classification, zero-shot.

Guaranteed bias removal

Bias is a direction in the embedding space. Subtracting it produces a neutralized vector, verifiable by construction across 28 demographic axes — not statistical post-hoc filtering.

02 — What the operators do

What “programmable” means concretely

Three primitive operations on the embedding space, each one a learned map with its own training objective and held-out evaluation:

compose

Given vectors h_a and h_b, produce a vector h_c that represents their composition. Trained against discourse-adjacent sentence pairs.

invert

Given h_c and one operand, recover the other. This works for strong, well-separated concepts, but does not close reliably in the general case — a limit we measure and report rather than hide.

decompose

Given a composed vector with no hint, recover both operands. The hardest case — and an honest negative: blind decomposition does not reliably recover the operands, so we don't claim it. Composition is directional, not fully reversible.

Set operations on concept directions

Concept directions combine like sets — union, intersection, and difference — which is what makes intersectional cases (say, race × gender) tractable. Set intersection as a detection operation checks out at 0.81 AUROC. We frame these as practical set operations — useful and checkable, not a formal proof of the axioms of set theory.

Counterfactual constructor

A linear constructor takes (T_protected, T_domain) and produces T_protected_in_domain — the bias direction synthesized for a specific decision context. Held-out cosine 0.91 across four unseen regulated domains: criminal sentencing, pain management, promotion review, public benefits.

03 — Empirical evidence

What we measure

Every result is reproducible from public datasets in under 90 seconds on a laptop GPU. We do not publish numbers we cannot regenerate on request.

Result	Value	Test conditions
Low-collateral editing (remove one concept, others hold)	4–6.5×	Less cross-concept movement than diff-of-means and LEACE · 3 seeds · 2 independent models · variance ±0.0005
Set-intersection detection (race × gender)	0.81	AUROC · concept directions combine like sets (union · intersection · difference) for intersectional cases
Bias-axis correction (BBQ + StereoSet + CrowS-Pairs + WinoBias)	28 / 28	15,966 sentence pairs · zero failures via centroid algebraic identity
MASSIVE Swahili intent classification	73.2%	Zero-shot · zero target-language data · above GPT-4o (70.6%)
Cross-family transfer (Korean / Hindi / Amharic)	72.5 / 69.7 / 66.5%	Linear probe on frozen encoder · 38–43× over random · strongest published frozen + linear + zero-target-language result we know of
Injongo (8 Bantu languages)	SOTA 4 / 8	vs AfroXLMR-76L (fine-tuned per language). Bhala is zero-shot — no in-language training
Sentiment steering (operator algebra)	100% in-family	Verified by independent classifier · single operator vector · cross-language transfer 77% to English (zero-shot)

Methodology and per-axis breakdowns: full benchmarks page.

04 — What this unlocks

Problems that now have clean solutions

Fairness audits that pass legal scrutiny

EEOC 4/5ths-rule, EU AI Act high-risk, NYC LL144 hiring audits all need decision-level disparate-impact ratios. The counterfactual constructor synthesizes the bias direction for any domain it has not seen — computable in the embedding space, not approximated from outputs.

Multi-step reasoning on unseen compositions

Current models hallucinate or refuse on compositions they have not seen. Bhala's operators support decomposition (recover the parts) and inversion (find the missing operand) — structural reasoning that generalizes by construction.

Edge AI and sovereign deployment

On-CPU inference, sub-100ms latency, fully offline. Regulated industries, air-gapped networks, and sovereign deployments that cannot use cloud-hosted 100B+ models can now get compositional-generalization-grade embeddings.

Cross-lingual NLP without parallel data

Operators estimated in one language transfer to others without retraining. The same primitive that enables Swahili MASSIVE performance applies to low-resource languages across language families — opening deployment in markets where parallel corpora do not exist.

05 — Academic context

For the technically curious

If you know word2vec: king − man + woman ≈ queen. That's word arithmetic — approximate, inconsistent, breaks after one step. Bhala does the same for sentences and meanings, but as learned operators with their own training objectives and held-out tests, not loose arithmetic on a frozen space. Some of these operators hold up cleanly; others don't, and we report which.

The idea has deep roots in a four-decade debate about whether neural networks can support structure-sensitive operations. We are testing how far a learned embedding space can go toward that ideal. Our generalization results on that question are under internal audit, and until the audit is done we make no claim — in either direction. One thing we already know and report: full round-trip closure, blindly decomposing a composition back into its parts, does not hold. Composition is directional, not exactly reversible.

Bhala's architecture is a purpose-built encoder trained under joint algebraic constraints. Composition and detection are learned components with their own loss terms. What that buys in practice is the result that matters in the field: you can remove one concept and leave the others measurably in place — 4–6.5× more cleanly than standard methods, stable across seeds and across independent models.

The evidence table in Section 03 shows what holds and what doesn't. Every result is reproducible from public datasets on a laptop GPU.

Read further

Full benchmark methodology, per-axis breakdowns, and reproduction code on the benchmarks page. We're happy to walk research collaborators or investors through the architecture in detail.

Benchmarks Contact