I don't think anybody's ever notified that they were sentenced to an extra two years because their recidivism score had been high, or notified that this beat cop happened to be in their neighborhood checking people's pockets for pot because of a predictive policing algorithm. That's just not how it works.
Cathy O'NeilMy fantasy is that there is a new regulatory body that is in charge of algorithmic auditing.
Cathy O'NeilMost people don't have any association in their minds with what they do and with ethics. They think they somehow moved past the questions of morality or values or ethics, and that's something that I've never imagined to be true.
Cathy O'Neil