Detecting trade-offs between fairness, stability, and accuracy for responsible machine learning model selection
DOI:
https://doi.org/10.30837/2522-9818.2025.1.005Keywords:
responsible artificial intelligence; algorithmic fairness; model stability; experimental analysis.Abstract
The subject of the study in the article is the process of machine learning model selection performed by data scientists to build models in critical areas. The purpose of the work: 1) create a software library for measuring the accuracy, stability, and fairness of models; 2) conduct experiments to identify trade-offs between fairness, stability, and accuracy; 3) propose a responsible model selection process to improve the safety of using machine learning models. The article provides for the following tasks: to measure the fairness and stability of machine learning models and to investigate their relationship with the accuracy of models. The following methods are introduced: empirical evaluation, the theory of decomposition of model error into bias and variance, the theory of algorithmic fairness, and methods for quantitative assessment of uncertainty. Results achieved: 1) low predictive variability is proposed as a desirable property to ensure safety and equality of variability between different social groups as a new metric of fairness of machine learning models; 2) it is demonstrated how stability analysis helps specialists overcome the challenges of model multiplicity and choose reliable, stable and fair models; 3) an open-source software framework for community use is created that integrates stability measurements into model development processes. Conclusions. This work proposes a new paradigm of group fairness that combines the issues of correctness/quality and randomness/stability from the research agenda of responsible artificial intelligence. The application of the proposed approaches helps in the responsible selection of machine learning models under conditions of model multiplicity, demonstrating that, although there may be many models with comparable accuracy, there is only one (or a few) "best" model that is reliable, fair and stable, as it should be.
References
Список літератури
Man is to computer programmer as woman is to homemaker? debiasing word embeddings / T. Bolukbasi et al. Advances in neural information processing systems. 2016. No. 29. P. 4356–4364. DOI: https://doi.org/10.48550/arXiv.1607.06520
Buolamwini J., Gebru T. Gender shades: Intersectional accuracy disparities in commercial gender classification. Conference on fairness, accountability and transparency. 2018. P. 77–91. URL: https://proceedings.mlr.press/v81/buolamwini18a.html
Caliskan A., Bryson J. J., Narayanan A. Semantics derived automatically from language corpora contain human-like biases. Science. 2017. Vol. 356. P. 183–186. DOI: https://doi.org/10.1126/science.aal4230
Sweeney L. Discrimination in online ad delivery. Communications of the ACM. 2013. Vol. 56. No. 5. P. 44–54. DOI: https://doi.org/10.1145/2447976.2447990
Calders T., Verwer S. Three Naive Bayes Approaches for Discrimination-Free Classification. Data Mining and Knowledge Discovery. 2010. Vol. 21. No. 2. P. 277–292. DOI: https://doi.org/10.1007/s10618-010-0190-x
Chouldechova A. Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data. 2017. Vol. 5. No. 2. P. 153–163. DOI: https://doi.org/10.1089/big.2016.0047
Preserving statistical validity in adaptive data analysis / C. Dwork et al. Proceedings of the forty-seventh annual ACM symposium on Theory of computing. 2015. P. 117–126. DOI: https://doi.org/10.1145/2746539.2746580
Fairness through awareness / C. Dwork et al. Proceedings of the 3rd innovations in theoretical computer science conference. 2012. P. 214–226. DOI: https://doi.org/10.1145/2090236.2090255
Certifying and Removing Disparate Impact / M. Feldman et al. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015. P. 259–268. DOI: https://doi.org/10.1145/2783258.2783311
Kamishima T., Akaho S., Sakuma J. Fairness-aware Learning through Regularization Approach. IEEE 11th International Conference on Data Mining Workshops. 2011. P. 643–650. DOI: https://doi.org/10.1109/icdmw.2011.83
Kleinberg J. M., Mullainathan S., Raghavan M. Inherent Trade-Offs in the Fair Determination of Risk Scores. Innovations in Theoretical Computer Science Conference. 2017. Vol. 67. P. 43:1–43:23. DOI: https://doi.org/10.4230/LIPIcs.ITCS.2017.43
Black E., Raghavan M., Barocas S. Model Multiplicity: Opportunities, Concerns, and Solutions. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022. P. 850–863. DOI: https://doi.org/10.1145/3531146.3533149
Underspecification presents challenges for credibility in modern machine learning / A. D’Amour et al. The Journal of Machine Learning Research. 2022. Vol. 23. No. 1. P. 10237–10297. DOI: https://doi.org/10.48550/arXiv.2011.03395
Marx C., Calmon F., Ustun B. Predictive multiplicity in classification. International Conference on Machine Learning. 2020. P. 6765–6774. URL: https://proceedings.mlr.press/v119/marx20a.html
Creel K., Hellman D. The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic decision making systems. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021. 816 р. DOI: https://doi.org/10.1145/3442188.3445942
Arbitrariness and social prediction: The confounding role of variance in fair classification / A. F. Cooper et al. Proceedings of the AAAI Conference on Artificial Intelligence. 2024. Vol. 38. No. 20. P. 22004-22012. DOI: https://doi.org/10.1609/aaai.v38i20.30203
Arbitrariness Lies Beyond the Fairness-Accuracy Frontier / C. Long et al. arXiv preprint arXiv:2306.09425. 2023. DOI: https://doi.org/10.48550/arXiv.2306.09425
Shvets A. Dive into design patterns. Refactoring Guru. 2018. P. 22–29.
Domingos P. A unified bias-variance decomposition. Proceedings of 17th international conference on machine learning. 2000. P. 231–238. URL: https://www.scirp.org/reference/referencespapers?referenceid=2848771
Efron B., Tibshirani R. J. An introduction to the bootstrap. CRC press. 1994.
Darling M. C., Stracuzzi D. J. Toward Uncertainty Quantification for Supervised Classification. 2018. OSTI.GOV. DOI: https://doi.org/10.2172/1527311
Model Stability with Continuous Data Updates / H. Liu et al. arXiv preprint arXiv:2201.05692. 2022. DOI: https://doi.org/10.48550/arXiv.2201.05692
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias / R. K. E. Bellamy et al. IBM Journal of Research and Development. 2019. Vol. 63. No. 4/5. P. 4:1–4:15. DOI: https://doi.org/10.1147/jrd.2019.2942287
Certifying and removing disparate impact / M. Feldman et al. Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 2015. P. 259–268. DOI: https://doi.org/10.1145/2783258.2783311
Preserving statistical validity in adaptive data analysis / C. Dwork et al. Proceedings of the forty-seventh annual ACM symposium on Theory of computing. 2015. P. 117–126. DOI: https://doi.org/10.1145/2746539.2746580
A survey on datasets for fairness-aware machine learning / T. Le Quy et al. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2022. Vol. 12. No. 3. DOI: https://doi.org/10.1002/widm.1452
References
Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., (2016), "Man is to computer programmer as woman is to homemaker? debiasing word embeddings", Advances in neural information processing systems, 29, Р. 4356-4364. DOI: https://doi.org/10.48550/arXiv.1607.06520
Buolamwini, J. and Gebru, T., (2018), "Gender shades: Intersectional accuracy disparities in commercial gender classification", In Conference on fairness, accountability and transparency Р. 77-91, PMLR. available at: https://proceedings.mlr.press/v81/buolamwini18a.html
Caliskan, A., Bryson, J.J. and Narayanan, A., (2017), "Semantics derived automatically from language corpora contain human-like biases", Science, 356(6334), Р.183-186. DOI: https://doi.org/10.1126/science.aal4230
Sweeney, L., (2013), "Discrimination in online ad delivery", Communications of the ACM, 56(5), Р.44-54. DOI: https://doi.org/10.1145/2447976.2447990
Calders, T., and Verwer, S. (2010), "Three Naive Bayes Approaches for Discrimination-Free Classification", Data Min. Knowl. Discov., 21(2). Р. 277–292. DOI: https://doi.org/10.1007/s10618-010-0190-x
Chouldechova, A. (2017), "Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments", Big Data, 5(2). Р. 153–163. DOI: https://doi.org/10.1089/big.2016.0047
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., and Roth, A. L. (2015), "Preserving statistical validity in adaptive data analysis", In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, Р. 117–126. DOI: https://doi.org/10.1145/2746539.2746580
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. S. (2012), "Fairness through awareness", In Goldwasser, S., ed., Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 810, 2012, Р. 214–226, ACM. DOI: https://doi.org/10.1145/2090236.2090255
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015), "Certifying and Removing Disparate Impact", In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, Р. 259–268. New York, NY, USA: Association for Computing Machinery, ISBN 9781450336642. DOI: https://doi.org/10.1145/2783258.2783311
Kamishima, T., Akaho, S., and Sakuma, J. (2011), "Fairness-aware Learning through Regularization Approach", In 2011 IEEE 11th International Conference on Data Mining Workshops, Р. 643–650. DOI: https://doi.org/10.1109/icdmw.2011.83
Kleinberg, J. M., Mullainathan, S., and Raghavan, M. (2017), "Inherent Trade-Offs in the Fair Determination of Risk Scores", In Papadimitriou, C. H., ed., 8th Innovations in Theoretical Computer Science Conference, ITCS 2017, January 9-11, 2017, Berkeley, CA, USA, Volume 67 of LIPIcs, Р. 43:1–43:23, Schloss Dagstuhl Leibniz-Zentrum fur Informatik. DOI: https://doi.org/10.4230/LIPIcs.ITCS.2017.43
Black, E., Raghavan, M., and Barocas, S. (2022), "Model Multiplicity: Opportunities, Concerns, and Solutions", In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, Р. 850–863. New York, NY, USA: Association for Computing Machinery, ISBN 9781450393522. DOI: https://doi.org/10.1145/3531146.3533149
D’Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. 2022, "Underspecification presents challenges for credibility in modern machine learning", The Journal of Machine Learning Research, 23(1). Р. 10237–10297. DOI: https://doi.org/10.48550/arXiv.2011.03395
Marx, C., Calmon, F., and Ustun, B. (2020), "Predictive multiplicity in classification", In International Conference on Machine Learning, Р. 6765–6774, PMLR. available at: https://proceedings.mlr.press/v119/marx20a.html
Creel, K. and Hellman, D. (2021), "The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic decision making Systems", In Elish, M. C.; Isaac, W.; and Zemel, R. S., eds., FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 816 р. ACM. DOI: https://doi.org/10.1145/3442188.3445942
Cooper, A. F., Lee, K., Choksi, M. Z., Barocas, S., De Sa, C., Grimmelmann, J., Kleinberg, J., Sen, S., and Zhang, B. (2024), "Arbitrariness and social prediction: The confounding role of variance in fair classification", In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, No. 20, Р. 22004-22012. DOI: https://doi.org/10.1609/aaai.v38i20.30203
Long, C. X., Hsu, H., Alghamdi, W., and Calmon, F. P. (2023), "Arbitrariness Lies Beyond the Fairness-Accuracy Frontier", arXiv preprint arXiv:2306.09425. DOI: https://doi.org/10.48550/arXiv.2306.09425
Shvets, A. (2018), Dive into design patterns, Refactoring, Guru, Р.22-29.
Domingos, P. (2000), "A unified bias-variance decomposition", In Proceedings of 17th international conference on machine learning. Р. 231–238, Morgan Kaufmann Stanford. available at: https://www.scirp.org/reference/referencespapers?referenceid=2848771
Efron, B. and Tibshirani, R. J. (1994), "An introduction to the bootstrap", CRC press.
Darling, M. C. and Stracuzzi, D. J. (2018), "Toward Uncertainty Quantification for Supervised Classification". OSTI.GOV. DOI: https://doi.org/10.2172/1527311
Liu, H., Patwardhan, S., Grasch, P., Agarwal, S., et al. (2022), "Model Stability with Continuous Data Updates", arXiv preprint arXiv:2201.05692. DOI: https://doi.org/10.48550/arXiv.2201.05692
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K. N., Richards, J. T., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., and Zhang, Y. (2019), "AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias", IBM J. Res. Dev., 63(4/5). Р. 4:1–4:15. DOI: https://doi.org/10.1147/jrd.2019.2942287
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015), "Certifying and removing disparate impact", In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, Р. 259–268. DOI: https://doi.org/10.1145/2783258.2783311
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., and Roth, A. L. (2015), "Preserving statistical validity in adaptive data analysis", In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, Р. 117–126. DOI: https://doi.org/10.1145/2746539.2746580
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., and Ntoutsi, E. (2022), "A survey on datasets for fairness-aware machine learning", Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(3): e1452. DOI: https://doi.org/10.1002/widm.1452
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Our journal abides by the Creative Commons copyright rights and permissions for open access journals.
Authors who publish with this journal agree to the following terms:
Authors hold the copyright without restrictions and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-commercial and non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
Authors are permitted and encouraged to post their published work online (e.g., in institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.
