Blog

Beyond Bias: Why the Next Frontier in AI Evaluation Must Be Human Rights Law

The Measurement Crisis in AI

For years, the AI community has wrestled with a significant measurement crisis. In the pursuit of developing ever more powerful systems, there has been a substantial mismatch between what we claim we are measuring in AI and what we are actually measuring in practice. We often fall back on imprecise, unscientific descriptions, treating sophisticated models as “black boxes” or “magic solutions” rather than scientific artifacts requiring rigorous measurement.

This crisis is encapsulated by the state of current AI evaluation: while standard benchmarks exist for technical performance, logical reasoning, or certain narrow ethical topics like social bias, they consistently fail to grapple with the actual societal impact of Large Language Models (LLMs) when they are deployed in the real world.

When LLMs Become Arbiters of Rights

LLMs are no longer confined to low-stakes tasks. They are increasingly used in high-stakes processes that directly determine opportunity and access for millions:

Job Recruitment and Resume Screening
Deciding Access to Housing, Healthcare, and Education
Assisting in Judicial and Legal Processes
Drafting Human Rights Reports and Communications

When these systems become arbiters of rights, it is essential that they do not entrench existing biases, contribute to increased inequality, or lead to new forms of human rights violations. The bare minimum expectation is that AI models, especially those used by state actors or companies operating within a state’s jurisdiction, do not act in ways that contravene internationally recognized legal standards.

The Inadequacy of Existing Benchmarks

Existing AI benchmarks are simply insufficient for this monumental task because they lack the necessary construct validity—the measure must be true to the real-world domain it claims to evaluate.

Benchmark Focus	Why It Fails to Cover Human Rights
Technical Performance (e.g., speed, efficiency)	Measures how well the model runs, not what it understands.
Cognitive Tasks (e.g., math, logical reasoning)	Asks Can the model solve a problem? Not Should it, under law?
Narrow Ethics (e.g., social bias)	Important, but lacks the force of codified international law. It measures specific discriminatory patterns, not general legal obligations.

A common, but flawed, approach is using standardized tests (like the US Bar exam) to evaluate AI competence. As experts point out, this is a failure of construct validity: a lawyer’s day-to-day work is not taking an exam. Worse, LLMs may have simply memorized previous test questions, leading to saturated results that don’t reflect true reasoning ability.

The Human Rights Benchmark Project is the first initiative designed to fill this gap. It shifts the evaluation paradigm from abstract ethical principles to measurable legal compliance based on the rigorous framework of International Human Rights Law (IHRL).

We are not just testing “ethics”; we are testing an LLM’s comprehension of its fundamental duties. Our benchmark asks: Does the LLM have an internalized representation of international human rights law that aligns with expert understanding?

This focus is crucial because IHRL defines explicit, non-negotiable obligations for states—the obligation to Respect, Protect, and Fulfill human rights. If an LLM cannot distinguish between these foundational legal concepts, it possesses a key deficit that makes it unsafe for deployment in high-stakes contexts.

The Scope of Accountability

To ensure the benchmark is robust and truly reflective of the human rights domain, its design is informed by a comprehensive Taxonomy that covers the full “axes of variation” within human rights situations.

Analytic Categories: We test the model’s ability to identify legal failure modes, such as whether a state has violated its obligation to Respect (refrain from interfering with rights), Protect (prevent others from interfering with rights), or Fulfill (take positive measures to realize rights).
Descriptive Categories: Scenarios are meticulously categorized by the Actors involved (state, company, NGO) and the Rights Holders affected (women, children, indigenous peoples, etc.). This structure allows us to test for latent bias, for example, by swapping out the rights holder group in a modular scenario.

By focusing on the human rights worker’s task of Monitoring and Reporting, we ensure the evaluation tasks are true to the real-world application of human rights law. The result is a scientifically rigorous, expert-validated dataset that is the essential first step toward holding LLMs accountable not just for their technical brilliance, but for their fundamental legal and ethical responsibilities.

We’re creating a global community that brings together individuals passionate about inclusive, human rights-based AI.

Join our AI & Equality community of students, academics, data scientists and AI practitioners who believe in responsible and fair AI.