There are lots of different ways that algorithms can go wrong, and what we have now is a system in which we assume because it's shiny new technology with a mathematical aura that it's perfect and it doesn't require further vetting. Of course, we never have that assumption with other kinds of technology.
Cathy O'NeilWhen people are not given an option by some secret scoring system, it's very hard to complain, so they often don't even know that they've been victimized.
Cathy O'NeilWe've learned our lesson with finance because they made a huge goddamn explosion that almost shut down the world. But the thing I realized is that there might never be an explosion on the scale of the financial crisis happening with big data.
Cathy O'Neil