Bhala research / programmable embeddings

A new class of models nobody could build.

For the first time, an embedding space supports operators that compose, invert, and verify. Bias becomes a direction you subtract. Sentiment is a vector you apply. Every output carries a verifiable audit trail. This page shows the evidence, explains the operators, and describes what becomes possible.

01 — New capabilities

The class of models nobody could build

Current AI models are black boxes: you feed in text and get back a vector or prediction, but you cannot name the change you want and apply it as an operator. You cannot extract the bias component, subtract it, and verify it is gone. You cannot tell the model “apply formal register” without retraining.

Bhala's embedding space changes the primitive. The representation is structured so that named operators act on it: compose two meanings, invert a transformation, subtract a demographic direction and verify the result. These operators are learned components with their own training objectives — not arithmetic post-hoc on a frozen space.

Steerable models

Control outputs at inference by composing named operators — sentiment, formality, register — without retraining or RLHF.

Auditable AI decisions

Every prediction comes with a verifiable record of which operators were applied. Regulators get evidence, not post-hoc explanations.

Cross-lingual transfer without parallel data

Operators learned in one language transfer algebraically to others. A 15M-parameter model trained on Zulu beats GPT-4o on Swahili intent classification — 100,000× fewer parameters.

Guaranteed bias removal

Bias is a direction in the embedding space. Subtracting it produces a neutralized vector, verifiable by construction across 28 demographic axes — not statistical post-hoc filtering.

02 — What the operators do

What “programmable” means concretely

Three primitive operations on the embedding space, each one a learned map with its own training objective and held-out evaluation:

compose

Given vectors h_a and h_b, produce a vector h_c that represents their composition. Trained against discourse-adjacent sentence pairs.

invert

Given h_c and one operand, recover the other. Both left- and right-inverses are separately trained, both achieve closure on held-out pairs.

decompose

Given a composed vector with no hint, recover both operands directly. The hardest of the three; demonstrates that the composition encoded structural information rather than averaging.

ZFC set-theoretic operations

Beyond compose/invert/decompose, the embedding space supports union, intersection, difference, powerset, extensionality, separation, choice, commutativity, and De Morgan operations on demographic-axis vectors. All nine ZFC axioms verified — produced empirically, not postulated.

Counterfactual constructor

A linear constructor takes (T_protected, T_domain) and produces T_protected_in_domain — the bias direction synthesized for a specific decision context. Held-out cosine 0.91 across four unseen regulated domains: criminal sentencing, pain management, promotion review, public benefits.

03 — Empirical evidence

What we measure

Every result is reproducible from public datasets in under 90 seconds on a laptop GPU. We do not publish numbers we cannot regenerate on request.

Result	Value	Test conditions
F&P closure (compose · invert · decompose)	100%	Held-out adjacent sentence pairs · learned operators · 128-d composition space
ZFC axioms verified	9 / 9	Union · intersection · difference · powerset · extensionality · separation · choice · commutativity · De Morgan
Bias-axis correction (BBQ + StereoSet + CrowS-Pairs + WinoBias)	28 / 28	15,966 sentence pairs · zero failures via centroid algebraic identity
Counterfactual constructor	cos 0.91	4 held-out regulated domains never seen during training
MASSIVE Swahili intent classification	73.2%	Frozen 15M-param backbone · zero target-language data · above GPT-4o (70.6%, ≈1.8T params)
Cross-family transfer (Korean / Hindi / Amharic)	72.5 / 69.7 / 66.5%	Linear probe on frozen encoder · 38–43× over random · strongest published frozen + linear + zero-target-language result we know of
Injongo (8 Bantu languages)	SOTA 4 / 8	vs AfroXLMR-76L (270M, fine-tuned per language). Bhala 15M frozen — 18× smaller
Sentiment steering (operator algebra)	100% in-family	Verified by independent classifier · single operator vector · cross-language transfer 77% to English (zero-shot)

Methodology and per-axis breakdowns: full benchmarks page.

04 — What this unlocks

Problems that now have clean solutions

Fairness audits that pass legal scrutiny

EEOC 4/5ths-rule, EU AI Act high-risk, NYC LL144 hiring audits all need decision-level disparate-impact ratios. The counterfactual constructor synthesizes the bias direction for any domain it has not seen — computable in the embedding space, not approximated from outputs.

Multi-step reasoning on unseen compositions

Current models hallucinate or refuse on compositions they have not seen. Bhala's operators support decomposition (recover the parts) and inversion (find the missing operand) — structural reasoning that generalizes by construction.

Edge AI and sovereign deployment

15M parameters, on-CPU inference, sub-100ms latency. Regulated industries, air-gapped networks, and sovereign deployments that cannot use cloud-hosted 100B+ models can now get compositional-generalization-grade embeddings.

Cross-lingual NLP without parallel data

Operators estimated in one language transfer to others without retraining. The same primitive that enables Swahili MASSIVE performance applies to low-resource languages across language families — opening deployment in markets where parallel corpora do not exist.

05 — Academic context

For the technically curious

If you know word2vec: king − man + woman ≈ queen. That's word arithmetic — approximate, inconsistent, breaks after one step. Bhala does the same for sentences and meanings, but exactly: learned operators, verified closure, no unexpected side effects. The arithmetic is the product.

In 1988, Jerry Fodor and Zenon Pylyshyn published a foundational critique of connectionism: neural networks cannot exhibit systematic compositional generalization — the property that learning love(John, Mary) should guarantee love(Mary, John) the way classical symbol systems do.

The argument has shaped almost four decades of debate. It motivated neuro-symbolic AI, the SCAN / COGS / CFQ compositional generalization benchmarks, and the ongoing skepticism of deep learning in cognitive science. No learned embedding space had demonstrated F&P closure — compose, invert, and decompose as exact operations — until ours.

Bhala's architecture is a purpose-built encoder trained under joint algebraic constraints. The composition operator, its left and right inverses, and the decomposition map are all learned components with their own loss terms — not arithmetic post-hoc on a frozen space. We shape the embedding space to make the operators well-defined. At 15M parameters, the result is not a claim about scale.

The 8-row evidence table in Section 03 is the existence proof. Every result is reproducible from public datasets on a laptop GPU.

Read further

Full benchmark methodology, per-axis breakdowns, and reproduction code on the benchmarks page. We're happy to walk research collaborators or investors through the architecture in detail.

Benchmarks Contact