Questioning the assumptions behind fairness solutions

In addition to their benefits, optimization systems can have negative economic, moral, social, and political effects on populations as well as their environments. Frameworks like fairness have been proposed to aid service providers in addressing subsequent bias and discrimination during data collection and algorithm design. However, recent reports of neglect, unresponsiveness, and malevolence cast doubt on whether service providers can effectively implement fairness solutions. These reports invite us to revisit assumptions made about the service providers in fairness solutions. Namely, that service providers have (i) the incentives or (ii) the means to mitigate optimization externalities. Moreover, the environmental impact of these systems suggests that we need (iii) novel frameworks that consider systems other than algorithmic decision-making and recommender systems, and (iv) solutions that go beyond removing related algorithmic biases. Going forward, we propose Protective Optimization Technologies that enable optimization subjects to defend against negative consequences of optimization systems.

POTs: Protective Optimization Technologies

In spite of their many advantages, optimization systems often neglect the economic, ethical, moral, social, and political impact they have on populations and their environments. In this paper we argue that the frameworks through which the discontents of optimization systems have been approached so far cover a narrow subset of these problems by (i) assuming that the system provider has the incentives and means to mitigate the imbalances optimization causes, (ii) disregarding problems that go beyond discrimination due to disparate treatment or impact in algorithmic decision making, and (iii) developing solutions focused on removing algorithmic biases related to discrimination. In response we introduce Protective Optimization Technologies: solutions that enable optimization subjects to defend from unwanted consequences. We provide a framework that formalizes the design space of POTs and show how it differs from other design paradigms in the literature. We show how the framework can capture strategies developed in the wild against real optimization systems, and how it can be used to design, implement, and evaluate a POT that enables individuals and collectives to protect themselves from unbalances in a credit scoring application related to loan allocation.