Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
AI Fairness 360 (AIF360) is an open-source library, released under the Apache-2.0 license, for detecting and mitigating bias in machine-learning models, available for Python and R. In research it is used to compute group-fairness metrics on a dataset or a model’s predictions and to apply bias-mitigation algorithms before, during or after model training. To start, run pip install aif360 (Python 3.10 to 3.13 are listed as supported), load one of the bundled datasets, and call a metric such as disparate_impact_ratio; the sections below walk through each step.
What AI Fairness 360 is used for in research
The toolkit was introduced by Bellamy and colleagues in the 2018 paper “AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias” (arXiv:1810.01943). The paper describes a Python toolkit with three kinds of component: fairness metrics, explanations of those metrics, and mitigation algorithms, backed by built-in testing. The repository README adds an interactive web demonstration and a folder of Jupyter notebooks that show the library applied to example datasets.
Researchers reach for it in three situations. The first is an audit: a classifier already exists and the question is whether its selection rates or error rates differ between groups defined by a protected attribute such as sex or race. The second is a comparison: several mitigation methods are run on the same data and their effect on a fairness metric and on accuracy is reported side by side. The third is teaching and replication: the bundled loaders for public benchmark datasets let a reader reproduce a published fairness analysis without assembling the data by hand. The concepts behind those questions are defined on CASRAI’s pages for AI fairness and AI bias, and the wider governance context is in the guide to responsible AI. AIF360 measures and mitigates; it does not decide which fairness definition suits a study. That choice stays with the researcher and has to be justified in the methods section.
The library groups its metrics into several families, all listed in the README: group-fairness metrics computed from selection rates and error rates, sample distortion metrics, the Generalized Entropy Index, Differential Fairness and Bias Amplification, and Bias Scan with Multi-Dimensional Subset Scan. The mitigation algorithms named in the README include Reweighing, Optimized Preprocessing, Disparate Impact Remover, Learning Fair Representations, Adversarial Debiasing, Prejudice Remover Regularizer, Reject Option Classification, Calibrated Equalized Odds Postprocessing, Exponentiated Gradient Reduction and Grid Search Reduction, among others.
How to install or access it
The official source is the Trusted-AI/AIF360 repository on GitHub, with documentation at aif360.readthedocs.io. According to the README, the Python package supports Python 3.10 to 3.13 on macOS, Ubuntu and Windows. The installation commands it lists are:
- Basic install:
pip install aif360 - With all optional dependencies:
pip install 'aif360[all]' - A fresh conda environment on Python 3.11 first:
conda create --name aif360 python=3.11, then activate it and run the pip command - R:
install.packages("aif360")
If you install manually from the source repository rather than from PyPI, the README says the datasets must be downloaded into folders laid out as described in aif360/data/README.md inside the repository. For a no-install look at the toolkit, the README links an interactive demo at aif360.res.ibm.com. The notebooks in the repository’s examples directory show complete workflows.
Step-by-step task: measure a metric and apply a mitigation step
The library has two interfaces. The original one wraps data in AIF360’s own dataset classes under aif360.datasets, aif360.metrics and aif360.algorithms. The newer aif360.sklearn interface follows scikit-learn conventions and works with pandas objects, and the Getting Started page of the documentation is written for it. The steps below use that interface.
- Install the package. Run
pip install 'aif360[all]'in a clean environment on a supported Python version. - Load a bundled dataset. The
aif360.sklearn.datasetsmodule provides loaders includingfetch_adult,fetch_german,fetch_bank,fetch_compas,fetch_lawschool_gpaandfetch_meps. The Getting Started page usesfrom aif360.sklearn.datasets import fetch_compasand shows thatfetch_compas().X.head()returns a DataFrame in which the protected attributes (sex and race) are stored in the index. - Fit your own classifier and obtain predictions. AIF360 does not replace your model. Any scikit-learn-style estimator works, and the metric functions take the true labels and your predicted labels as
y_trueandy_pred. - Compute a fairness metric. The documentation’s example uses the disparate impact ratio:
from aif360.sklearn.metrics import disparate_impact_ratio disparate_impact_ratio(y_true, y_pred, prot_attr='race', priv_group='White', pos_label=1)Here
prot_attrnames the protected attribute,priv_groupis the label of the privileged group, andpos_labelis the label of the favourable outcome. - Apply a mitigation stage and measure again. The Getting Started page shows one call per stage, covered in the next section. Rerun step 4 on the new predictions and report both values.
The three mitigation stages and their scikit-learn calls
Mitigation methods are grouped by where they act in the modelling pipeline. Pre-processing changes the training data or its representation, in-processing changes the learning algorithm, and post-processing changes the predictions. The Getting Started page gives one example for each:
- Pre-processing:
from aif360.sklearn.preprocessing import LearnedFairRepresentations, thenXt = LearnedFairRepresentations().fit_transform(X, y). - In-processing:
from aif360.sklearn.inprocessing import AdversarialDebiasing, theny_pred = AdversarialDebiasing().fit(X_train, y_train).predict(X_test). - Post-processing:
PostProcessingMeta(LogisticRegression(), RejectOptionClassifierCV('sex', scoring='disparate_impact')).fit(X_train, y_train).predict(X_test), withLogisticRegressionimported fromsklearn.linear_modeland the other two names fromaif360.sklearn.postprocessing.
The data table below the main text maps every module of the aif360.sklearn package to the classes or functions the documentation lists for it, so you can see which stage offers which method before choosing one.
Output and how to read it
disparate_impact_ratio returns a single float. The documentation defines it as the ratio of selection rates: the probability of the positive prediction for the unprivileged group divided by the same probability for the privileged group. A value of 1.0 therefore means both groups are selected at the same rate, a value below 1.0 means the unprivileged group is selected less often, and a value above 1.0 means it is selected more often.
Illustrative arithmetic (invented numbers, not from any dataset). Suppose a classifier predicts the positive label for 30 of 100 applicants in the unprivileged group and 50 of 100 in the privileged group. The selection rates are 0.30 and 0.50, so the ratio is 0.30 / 0.50 = 0.60.
Two details in the documentation affect interpretation. If you pass only y_true, the function returns the ratio of base rates, which is the disparate impact of the original dataset rather than of a model. If you pass both y_true and y_pred, only y_pred is used. This distinction matters when a paper reports bias “in the data” and “in the model”: the same function answers both questions depending on the arguments, so state which one you computed.
Other metrics in aif360.sklearn.metrics are listed with their own descriptions: statistical_parity_difference is the difference in selection rates, equal_opportunity_difference and average_odds_difference are relaxed versions of equality of opportunity and equality of odds, and generalized_entropy_index measures inequality over a population. Check each function’s page for its exact formula and argument order before reporting a value.
A ratio is not a legal test
The disparate impact ratio resembles the four-fifths (80%) rule, and it is tempting to treat 0.8 as a pass mark. Watkins, McKenna and Chen argue in “The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness” (arXiv:2202.09519, February 2022) that responsible-AI work turned the four-fifths rule, one part of disparate impact discrimination law, into a “disparate impact metric” that carries a new ethical nuance and potential for harm that the original rule lacked. In a research report, describe the number as a ratio of selection rates, say which group is privileged and why, and avoid claiming legal compliance or non-compliance from it.
Troubleshooting
- Installation fails or imports break. Check the Python version first. The README lists Python 3.10 to 3.13; its own conda example creates an environment on Python 3.11, which is a safe starting point.
- A dataset loader cannot find its files. With a manual source install, the README says the dataset files must be downloaded into the folders described in
aif360/data/README.md. - The ratio looks inverted.
priv_groupdefaults to 1. If your attribute is coded differently (for example as strings such as ‘White’), setpriv_groupexplicitly. - The result is 0 with a warning.
zero_divisiondefaults to'warn', which returns 0 and raises a warning when a group has no positive predictions. Setting it to 0 or 1 chooses the value silently, so report which setting you used. - Several protected attributes. If
prot_attris None, all protected attributes found iny_trueare used. Name the attribute you intend to analyse instead of relying on the default.
Free alternatives
AIF360 is not the only open-source fairness toolkit. The table lists what each project’s own pages state; it is not a ranking.
| Toolkit | What its own description says | License stated | Install command shown |
|---|---|---|---|
| AI Fairness 360 | Extensible library to detect and mitigate bias in machine-learning models; Python and R | Apache-2.0 | pip install aif360 |
| Fairlearn | Python package for assessing and improving the fairness of machine-learning models, with assessment metrics and mitigation algorithms | MIT | pip install fairlearn or conda install -c conda-forge fairlearn |
| Aequitas Flow | Open-source Python framework for end-to-end fair ML experimentation and benchmarking; extends the Aequitas auditing tools | MIT | pip install aequitas |
Whichever toolkit you use, record its exact version and the exact function you called, so that a reader can tell which definition of each metric produced your numbers.
Reporting AIF360 results in a paper
A reproducible fairness result needs more than the metric value. Record the package version (pip show aif360 prints it), the Python version, the dataset and its split, the protected attribute and the privileged group with the label coding used, the positive label, any random seed in the mitigation algorithm, and the metric before and after mitigation. If you use the toolkit, the README gives the citation: Bellamy et al., arXiv:1810.01943. Report the model’s accuracy next to the fairness metric, so that readers can see what a mitigation step changed on both.
For the conceptual side of the write-up, the guide to what artificial intelligence is and the entry on responsible AI give shared vocabulary for methods sections.
Frequently asked questions
What is AI Fairness 360?
AI Fairness 360 is an open-source, Apache-2.0-licensed toolkit for detecting and mitigating unwanted bias in machine-learning models. It provides fairness metrics, explanations of the metrics and bias-mitigation algorithms, and is available for Python and R, according to its GitHub README and the 2018 paper by Bellamy et al.
How do I install AIF360?
Run pip install aif360, or pip install 'aif360[all]' for all optional dependencies. The README lists Python 3.10 to 3.13 on macOS, Ubuntu and Windows, and install.packages("aif360") for R.
Which fairness metrics does AIF360 include?
The README lists group-fairness metrics based on selection rates and error rates, sample distortion metrics, the Generalized Entropy Index, Differential Fairness and Bias Amplification, and Bias Scan with Multi-Dimensional Subset Scan.
Does a disparate impact ratio of 0.8 mean a model is legally fair?
No. The ratio is a ratio of selection rates computed from predictions. Watkins, McKenna and Chen (arXiv:2202.09519) argue that the four-fifths rule is not disparate impact, so a threshold taken from it is not a legal test.
Is there a free alternative to AIF360?
Yes. Fairlearn is an MIT-licensed Python package with assessment metrics and mitigation algorithms, and Aequitas Flow is an open-source Python toolkit for fair-ML experimentation. See the alternatives table above.
Sources
- Trusted-AI/AIF360 repository README, https://github.com/Trusted-AI/AIF360, read 2026-11-15.
- AIF360 documentation, Getting Started, https://aif360.readthedocs.io/en/stable/Getting%20Started.html, read 2026-11-15.
- AIF360 documentation, aif360.sklearn module reference, https://aif360.readthedocs.io/en/stable/modules/sklearn.html, read 2026-11-15.
- AIF360 documentation, aif360.sklearn.metrics.disparate_impact_ratio, https://aif360.readthedocs.io/en/stable/modules/generated/aif360.sklearn.metrics.disparate_impact_ratio.html, read 2026-11-15.
- Bellamy et al., AI Fairness 360, arXiv:1810.01943, https://arxiv.org/abs/1810.01943, read 2026-11-15.
- Watkins, McKenna and Chen, The four-fifths rule is not disparate impact, arXiv:2202.09519, https://arxiv.org/abs/2202.09519, read 2026-11-15.
- Fairlearn repository, https://github.com/fairlearn/fairlearn, and installation guide, https://fairlearn.org/main/user_guide/installation_and_version_guide/installation_guide.html, read 2026-11-15.
- Jesus et al., Aequitas Flow, arXiv:2405.05809, https://arxiv.org/abs/2405.05809, read 2026-11-15.
Data: Components documented in each module of the AIF360 scikit-learn-compatible API (aif360.sklearn)
Count of classes and functions listed for each aif360.sklearn module in the AIF360 documentation module reference, with the names listed for each module (a selection for the metrics module). Compiled by counting the entries in the module tables of the stable documentation.
| Module | Documented items (count) | Names listed in the documentation |
|---|---|---|
| aif360.sklearn.metrics | 33 | statistical_parity_difference, disparate_impact_ratio, equal_opportunity_difference, average_odds_difference, generalized_entropy_index, consistency_score |
| aif360.sklearn.preprocessing | 4 | FairAdapt, LearnedFairRepresentations, Reweighing, ReweighingMeta |
| aif360.sklearn.inprocessing | 5 | AdversarialDebiasing, ExponentiatedGradientReduction, GridSearchReduction, SenSeI, SenSR |
| aif360.sklearn.postprocessing | 3 | CalibratedEqualizedOdds, RejectOptionClassifier, PostProcessingMeta |
| aif360.sklearn.detectors | 3 | MDSS_bias_scan, FACTS_bias_scan, FACTS |
| aif360.sklearn.datasets | 8 | fetch_adult, fetch_german, fetch_bank, fetch_compas, fetch_lawschool_gpa, fetch_meps, standardize_dataset, NumericConversionWarning |
Sources
- AIF360 documentation: aif360.sklearn module reference (accessed 2026-11-15)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







