With recidivism algorithms, for example, I worry about racist outcomes. With personality tests [for hiring], I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm [used in New York City to score teachers], I worry literally that it's not meaningful. That it's almost a random number generator.

Evidence of harm is hard to come by.

I would argue that one of the major problems with our blind trust in algorithms is that we can propagate discriminatory patterns without acknowledging any kind of intent.

We can't just throw something out there and assume it works just because it has math in it.

People are starting to be very skeptical of the Facebook algorithm and all kinds of data surveillance.

It's a standard thing you hear from startup people - that their product is somehow improving the world. And if you follow the reasoning, you will get somewhere, and I'll tell you where you get: You'll get to the description of what happens to the winners under the system that they're building.

An insurance company might say, "Tell us more about yourself so your premiums can go down." When they say that, they're addressing the winners, not the losers.