Final Project

Mutual information, Fairness and Robustness

Based on FR-Train (Y. Roh et al., 2020): A mutal information based approach to train fair and robust models

What is Fairness?

(Li, Qi, Liu, et al., 2022)

Figure: Relationship between different aspects of AI trustworthiness

An Interesting Example: Gender Classification

Figure: Overall accuracy

http://gendershades.org/

Breakdown

Figure: Accuracy by face color and gender

Fairness

  • Hiring/ Admisson
  • Risk management
  • Face recognition for law enforcement

Under-represented groups are more likely to be misclassified or experience systematic disadvantage.

Goal of Fairness: Eliminate or mitigate the effects of biases.

Metric for Fairness

Definiton: Disparate Impact (DI):

The ratio of the probability of a positive outcome for one group to the probability of a positive outcome for the other group.

Ex: If our model predict the probability of getting a loan for a white person is 0.8, and for a black person is 0.4, then the DI is 0.5.

Robustness

The definition of contamination varies in different contexts. Here we consider a specific case of label flipping.

Name Gender Age Give Loan
John Male 25 No
Kate Female 22 Yes
Brain Male 20 No
... ... ... ...

Robustness

Let be the label of a sample, and be the flipped label. Then is the -replacement of if

In such contaminated condition, we want our model to still perform relatively well.

Observation: Pursuing fairness can compromise robustness. (H. Xu et al, 2021)

Recap of Definition of Mutual Information:

Given two random variables and , the mutual information between and is defined as

Measure of dependence between two random variables. If , then and are independent

Synthetic dataset and problem

(Y. Roh et al., 2020)

  • Two non-sensitive features and .
  • One binary sensitive feature .
  • One binary label .
  • Some data points are flipped.

FR-Train architecture

(Y. Roh et al., 2020)

  • Similar to generative adversarial networks (GANs) (Goodfellow et al., 2014).
  • Generator to provide label predictions, fairness discriminator use predicted label to predict sensitive attribute.
  • Intuition: If the discriminator can perform well, then the model is not fair.

FR-Train architecture (cont.)

  • The robust discriminator use give feedback to generator using loss and a reweighting process.
  • It require a clean dataset to train the robust discriminator. (Sometimes unavailable).

Details of robust discriminator

  • It's more like a "privacy" mechanism.
  • Robust discriminator is to predict the probability of a sample coming from the training dataset.
  • Generator need to mask the the individual information of the sample.
  • Theory guarantee not provided.

Cross entropy and mutual information

Theorem 1 (Y. Roh et al., 2021)

Suppose is a discrete random variable and could be continuous or discrete. The mutual information can be shown to be equivalent to an optimzation problem.

The term we are maximizing is equivalent to the cross entropy of the discriminator with a flip of sign. Thus by minimizing discriminator loss, we can estimate the mutual information .

Discriminator loss Mutual information Fairness

Structure of our experiments

(i) Can discriminator losses converge to mutual information efficiently?

(ii) Can we use mutual information as loss for the generator directly?

(iii) Can mutual information guarantee good fairness performance even when the dataset is unbalanced?

Exp 1: Original Model - Convergence of discriminator losses

KNN estimator for mutual information (Kraskov et al., 2004)
knncmi (Mesner et al., 2019) package in python

Exp 1: Original model - DI and accuracy vs. iteration

Fluctuation of DI and accuracy is due to competition nature between generator and discriminator. Can we use mutual information as loss for the generator directly?

Exp 2: Mutual information as loss for the generator

How to estimate mutual information while also make it differentiable?

  • Naive approach to estimate

Figure: Sigmoid functions

Exp 2: Naive solution - DI and accuracy vs. iteration

Exp 2: Comparison of performance

Exp 3: Problem with unbalanced datasets

  • Under extreme cases, small mutual information doesn't necessarily mean good fairness performance.

Figure: Mutual information vs. DI for 1000 randomly generate samples

Exp 3: Dataset generation

Figure: Visualization of datasets under different parameters,
: proportion of positive group, : knob to control the proportion of sensitive group

Exp 3: Performance on unbalanced datasets

Work in progress / Future work

  • Normalizing mutual information to mitigate the effect of unbalanced dataset.
  • Benchmarking on unbalanced dataset (with other fairness methods).
  • Theoretical guarantee for the robust discriminator.

References

[1] Bo Li, Peng Qi, Bo Liu, Shuai Di, Jingen Liu, Jiquan Pei, Jinfeng Yi, and Bowen Zhou. Trustworthy ai: From principles to practices, 2022.
[2] Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. PMLR, 18–24 Jul 2021.
[3] Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. FR-train: A mutual information- based approach to fair and robust training. PMLR, 13–18 Jul 2020.
[4] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014.
[5] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA, 2006.

[6] Octavio César Mesner and Cosma Rohilla Shalizi. Conditional mutual information estimation for mixed discrete and continuous variables with nearest neighbors, 2019.
[7] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Phys. Rev. E, 69:066138, Jun 2004.

Thank you!

Good evening everyone. Today I'm going to present this paper by Yuji Roh et al. (2020). FR-train is a mutual information based approach to train fair and robust models. I'm also going to present some of my experiments. Hopefully you may find them intersting.

So for those who are not familiar with fairness and robustness. What is fairness and robustness? In a broader sense, both fairness and robustness are related to trustworthiness of AI, which is a hot topic in recent years. And here is a graph showing the relationship between different aspects of AI trustworthiness. by Li et al. (2022) As we all know, nowadays AI and machine learning models are very powerful. And we wish to deploy them in many different scenarios. However, we also want to make sure the AI is trustable. This includes many aspects as we can see in this graph. Technically we want AI to be reliable, where robustness is one of the key aspects. We also want AI to follow human values, which is related to fairness.

The idea of this comes from connection between differential privacy and robust statistics

To better understand why fairness is important, let's look at this example. Here are three State of the Art face recognition models. And the task here is to predict the gender of a person base on the face image. We can see that the overall accuracy of these models are really high. However can we just say these models are good enough and put them into production?

The answer is no. Let's breakdown the performance of these models by face color and gender and see what we happen. Predict well on male and light female. But worse on darker female. The gap between the accuracy is significant. So we can't say these models fair towards different groups.

Such observations can also be found in ... Where under-represented groups are more likely to be misclassified or experience systematic disadvantage. So what we want to achieve is to eliminate or mitigate this kind of inherent biases.

First I want to introduce the metric we use for fairness in this project. Let's say we have a sensitive attribute S, and a binary label Y. We define ... We put the smaller probability in the numerator to make sure the value is between 0 and 1.

Ok that's for fairness. Now let's talk about robustness. Overall, robustness is about how well the model can perform under different conditions. like what is there's outliers in the dataset, or what if the dataset is contaminated. The definition of contamination varies in different contexts. In the paper, the authors consider the following setting where some of the labels are artificially flipped.

Here we show a formal definition of such label flipping using epsilon replacement notation. We can say our observed Y is a epsilon replacement of the true label Y if the number of flipped labels is less than epsilon times the total number of samples. In such contaminated condition, we want our model to still perform relatively well. And it's important to study fairness and robustness together. Because pursuing fairness can compromise robustness. According to this paper by Han Xu et al. (2021).

You may ask: How are these things related to our course information theory. So first let's see how mutual information is related to fairness. A bit of recap of the deinifition of mutual information. I'm not going into the details. The idea is that mutual information is a measure of dependence between two random variables. If the mutual information is 0, then the two random variables are independent.

Now we are ready to get into the details of our model FR-train. Let's take a look at the sythetic data we are using and the problem we want to solve.

The arichitecture of FR-train is made of a generator to make predictions. And two discriminators, one for fairness and one for robustness. During training the genrator and the two discriminators compete with each other. really similar to GANs. Let's first look at the fairness discriminator. which is basically the first row of the architecture. The fairness discriminator takes the logit as input and predict the sensitive attribute. The intuition is that if the discriminator can perform well, then the model is not fair..

The robust discriminator is a bit different. It achieves robustness by both reweighting samples and give the loss as feedback to the generator. This kind of combine two different ideas of robustness training. One is just to get rid of the contaminated samples. The other is make the model capable of handling contaminated samples. Also it requires a clean dataset to train the robust discriminator. Which is not really realistic in practice. That's probabiliy one of the major drawbacks of this model.

But anyways We will take a look at the detail of this discriminator.

Ok so that wrapped up the architecure of FR-train. You may ask again. Where is the mutal information we just talk about and how do we minimize the mi? That's the second connection the paper made with information theory. It's about the relationship between cross entropy and mutual information. Here's the key theorem of the paper.

Based on what we just learnt. Here is the summary of the key idea of the orginall paper.