WEAPONS OF MATH DESTRUCTION

How ill-conceived algorithms now micromanage America’s economy, from advertising to prisons.

“Welcome to the dark side of Big Data,” writes math guru O’Neil (Doing Data Science: Straight Talk from the Frontline, 2013, etc.), a blogger (mathbabe.org) and former quantitative analyst at the hedge fund D.E. Shaw. In this simultaneously illuminating and disturbing account, she describes the many ways in which widely used mathematic models—based on “prejudice, misunderstanding, and bias”—tend to punish the poor and reward the rich. The most harmful such models, which she calls “Weapons of Math Destruction,” often have devastating effects on people when they are going to college, borrowing money, getting sentenced to prison, or finding and holding a job. For example: credit scores are used to evaluate potential hires (assuming bad scores correlate with bad job performance, which is often not true); for-profit colleges use data to target and prey on vulnerable strivers, often plunging them into debt; auto insurance companies judge applicants by their consumer patterns rather than their driving records; crime predictive software often leads police to focus on nuisance crimes in impoverished neighborhoods. As the author notes, the harmful effects are apparent “when a poor minority teenager gets stopped, roughed up, and put on warning by the local police, or when a gas station attendant who lives in a poor zip code gets hit with a higher insurance bill.” She notes the same mathematical models “place the comfortable classes of society in their own marketing silos,” jetting them off to vacations in Aruba, wait-listing them at Wharton, and generally making their lives “smarter and easier.” The author writes with passion—a few years ago she became disillusioned over her hedge fund modeling and joined the Occupy movement—but with the authority of a former Barnard professor who is outraged at the increasingly wrongheaded use of mathematics. She convincingly argues for both more responsible modeling and federal regulation.

An unusually lucid and readable look at the daunting algorithms that govern so many aspects of our lives.