Algorithms used in law enforcement and welfare enforce prejudice while displacing blame onto machines that responsible parties manage to convince the public as being neutral.
Exhibit A. SAFER in Ontario.
Black prisoners are assigned harsher living conditions in Ontario jails—thanks to AI ⋆ The Breach
Exhibit B. COMPAS.
Reporting from Pro Publica in 2016:
”Scores like this — known as risk assessments — are increasingly common in courtrooms across the nation.”
Exhibit C.
The Dutch childcare benefit scandal.
”Thousands of lives were ruined by a disgraceful process which included a xenophobic algorithm based on racial profiling.”
Dutch childcare benefit scandal an urgent wake-up call to ban racist algorithms
Exhibit D.
Discriminatory algorithm used by the French social security agence.
”The criteria that increase one’s risk score include parameters which discriminate against vulnerable households, including being on a low income, being unemployed, living in a disadvantaged neighbourhood, […]”
France: Discriminatory algorithm used by the social security agency must be stopped
Exhibit E.
Surveillance and discrimination in Denmark’s welfare state.
”Denmark’s transition to a digital welfare system now poses human rights risks and violations for social security benefits recipients.”
Exhibit F.
Use of Entity Resolution in India: Shining a light on how new forms of automation can deny people access to welfare
”[…] algorithmic systems in welfare can be error prone, which can result in mass exclusions of people from their entitlements […]”
Exhibit G.
Poverty and discrimination in Serbia’s welfare state
”Bogdan was one of possibly thousands of people who lost social assistance after the Social Card law entered into force in March 2022 and introduced automation into the process of determining [eligibility for social assistance].”
Trapped by Automation: Poverty and discrimination in Serbia's welfare state
Exhibit H.
Sweden’s suspicion machine.
”[The] Social Insurance Agency (Försäkringskassan) has silently conducted large-scale experiments with algorithms that score hundreds of thousands of people on benefits, supposedly predicting whether they will commit fraud.”