So much of our society as a whole is gearing us to maximize our salary or bonus. Basically, we just think in terms of money. Or, if not money, then, if you're in academia, it's prestige. It's a different kind of currency. And there's this unmeasured dimension of all jobs, which is whether it's improving the world.
Cathy O'NeilIt's a standard thing you hear from startup people - that their product is somehow improving the world. And if you follow the reasoning, you will get somewhere, and I'll tell you where you get: You'll get to the description of what happens to the winners under the system that they're building.
Cathy O'NeilI know how models are built, because I build them myself, so I know that I'm embedding my values into every single algorithm I create and I am projecting my agenda onto those algorithms.
Cathy O'NeilMicro-targeting is the ability for a campaign to profile you, to know much more about you than you know about it, and then to choose exactly what to show you.
Cathy O'NeilI would argue that one of the major problems with our blind trust in algorithms is that we can propagate discriminatory patterns without acknowledging any kind of intent.
Cathy O'NeilWith recidivism algorithms, for example, I worry about racist outcomes. With personality tests [for hiring], I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm [used in New York City to score teachers], I worry literally that it's not meaningful. That it's almost a random number generator.
Cathy O'Neil