Can You Trust Your AI to Be Unbiased? By: Rob Viglione

A recent research paper suggests it's a little more complicated. Unfortunately, bias isn't just a bug — it's a persistent feature without proper cryptographic guardrails.

Can you trust your AI to be unbiased? A recent research paper suggests it’s a little more complicated.

Unfortunately, bias isn’t just a bug — it’s a persistent feature without proper cryptographic guardrails. A September 2024 study from Imperial College London shows how zero-knowledge proofs (ZKPs) can help companies verify that their machine learning (ML) models treat all demographic groups equally while still keeping model details and user data private.

We recently covered how ZKPs are being used to detect fraud and anomalies in financial transactions. But in this case, ZKPs can be applied to verify the fairness property of ML models.

When discussing "fairness," we're entering a complicated area. There are several mathematical definitions of fairness, and the preferred definition shifts with the political landscape. For instance, consider the US government's approach to fairness over the past two administrations.

The previous administration was focused on diversity, equity and inclusion. They used demographic parity as a key measure of fairness, aiming to ensure the output probability of a specific prediction is the same across different groups.

But as we integrate more ML models into critical systems like college admissions, home loans and future job prospects, we could use a little more reassurance that AI is treating us fairly.

We need to be sure that any attestations of fairness keep the underlying ML models and training data confidential. They need to protect intellectual property and users’ privacy while providing enough access for users to know that their model is not discriminatory. Not an easy task.

Enter, zero-knowledge proofs.

ZKML (zero knowledge machine learning) is how we use zero-knowledge proofs to verify that an ML model is what it says on the box. ZKML combines zero-knowledge cryptography with machine learning to create systems that can verify AI properties without exposing the underlying models or data. We can also take that concept and use ZKPs to identify ML models that treat everyone equally and fairly.

Recently, we covered how ZKPs were becoming more efficient to perform at scale. Previously, using ZKPs to prove AI fairness was extremely limited because it could only focus on one phase of the ML pipeline. This made it possible for dishonest model providers to construct data sets that would satisfy the fairness requirements, even if the model failed to do so. The ZKPs would also introduce unrealistic computational demands and long wait times to produce proofs of fairness.

But in recent months, ZK frameworks have become more efficient to scale ZKPs to be able to perform synthesis tasks like quickly generating diverse pieces of content or merging large amounts of data. This makes it possible to integrate ZKPs to detect fraud or anomalies in financial transactions, which is a critical step toward large-scale adoption.

So how do we measure whether an AI is fair? Let's break down three of the most common group fairness definitions:

* Demographic parity

* Equality of opportunity

* Predictive equality

As we mentioned, diversity, equity and inclusion departments often use demographic parity as a measurement to attempt to reflect the demographics of a population in a company's workforce. It's not the ideal fairness metric for ML models because it's used to measure the probability of a specific prediction. For example, we wouldn't necessarily expect that every group will have the same outcomes.

Equality of opportunity is easy for most people to understand. It gives every group the same chance to have a positive outcome, assuming they are equally qualified. For instance, it is not optimizing for outcomes — only that every demographic should have the same opportunity to get a job or a home loan.

Likewise, predictive equality measures if an ML model makes predictions with the same accuracy across various demographics, so no one is penalized simply for being part of a group. So in both cases, the ML model is not putting its thumb on the scale for equity reasons but only to ensure that groups are not being systematically discriminated against in any way. And that is an eminently sensible fix.

Over the past year, the US government and other countries have issued statements and mandates around AI fairness and protecting the public from ML bias. Now, with a new administration in the US, there will likely be a different approach to AI fairness, shifting the focus back to equality of opportunity and away from equity.

As political landscapes change, so do the definitions of fairness in AI, moving between those focused on equity and those focused on opportunity. We are proponents of ML models that treat everyone equally without needing to put a thumb on the scale. And ZKPs can serve as an airtight way to verify that ML models are doing this without revealing private data.

While ZKPs have faced plenty of scalability challenges over the years, the technology is finally becoming more affordable for mainstream use cases. We can use ZKPs to verify training data integrity, protect privacy, and ensure the models we’re using are what they say they are.