I don't think anybody's ever notified that they were sentenced to an extra two years because their recidivism score had been high, or notified that this beat cop happened to be in their neighborhood checking people's pockets for pot because of a predictive policing algorithm. That's just not how it works.

I think big data companies only like good news. So I think they're just hoping that they don't get sued, essentially.

We can't just throw something out there and assume it works just because it has math in it.

It's a standard thing you hear from startup people - that their product is somehow improving the world. And if you follow the reasoning, you will get somewhere, and I'll tell you where you get: You'll get to the description of what happens to the winners under the system that they're building.

Because of my experience in Occupy, instead of asking the question, "Who will benefit from this system I'm implementing with the data?" I started to ask the question, "What will happen to the most vulnerable?" Or "Who is going to lose under this system? How will this affect the worst-off person?" Which is a very different question from "How does this improve certain people's lives?"

My fantasy is that there is a new regulatory body that is in charge of algorithmic auditing.

I would argue that one of the major problems with our blind trust in algorithms is that we can propagate discriminatory patterns without acknowledging any kind of intent.