Obviously the more transparency we have as auditors, the more we can get, but the main goal is to understand important characteristics about a black box algorithm without necessarily having to understand every single granular detail of the algorithm.

With recidivism algorithms, for example, I worry about racist outcomes. With personality tests [for hiring], I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm [used in New York City to score teachers], I worry literally that it's not meaningful. That it's almost a random number generator.

It's a standard thing you hear from startup people - that their product is somehow improving the world. And if you follow the reasoning, you will get somewhere, and I'll tell you where you get: You'll get to the description of what happens to the winners under the system that they're building.

You'll never be able to really measure anything, right? Including teachers.

Because of my experience in Occupy, instead of asking the question, "Who will benefit from this system I'm implementing with the data?" I started to ask the question, "What will happen to the most vulnerable?" Or "Who is going to lose under this system? How will this affect the worst-off person?" Which is a very different question from "How does this improve certain people's lives?"

Micro-targeting is the ability for a campaign to profile you, to know much more about you than you know about it, and then to choose exactly what to show you.

People felt like they were friends with Google, and they believed in the "Do No Evil" thing that Google said. They trusted Google more than they trusted the government, and I never understood that.