Weapons of Math Destruction Summary and Analysis

Summary: Introduction and Chapters 1–2

Introduction

Weapons of Math Destruction begins with the author’s childhood, following her through an early fascination with prime numbers to an eventual PhD in mathematics. O’Neil describes her subsequent dismay when mathematical models, wielded with little oversight, fueled the speculation that produced the 2008 financial crisis. She notes that similar models—often opaque to all but a select few—govern many aspects of society, dispensing judgments that can be hard or impossible to override.

To illustrate, O’Neil takes the reader through a roughly contemporary example from a different field: education. Struggling with poor student performance, the Washington, D.C., school system introduced a scoring system for teachers, the “worst” of whom were fired. Low scores led the school district to fire some highly skilled and well-liked educators, such as fifth-grade teacher Sarah Wysocki. The algorithm that generates the scores, O’Neil notes, is proprietary, so its details are not revealed. Once it has assigned a score, there is no way to appeal the verdict. Worse, these qualities are typical of a large and growing class of models used in finance, criminal justice, and the job market.

Chapter 1: Bomb Parts: What Is a Model?

O’Neil begins this chapter with an account of how statistical data came to dominate baseball strategy. She briefly describes the widespread mathematical modeling now known as “moneyball,” after Michael Lewis’s 2003 book of the same name. O’Neil then draws the reader’s attention to some specific traits of the modeling that goes on in baseball. It is transparent, since player statistics are publicly available, and it is rigorous, because it is based on close analysis of data that are directly relevant to winning games. Finally, it is constantly updated with new data from the games played every season.

Next, O’Neil gives a less formal example of modeling: deciding what to cook for dinner. This model considers things like nutrition, the household budget, and each family member’s food preferences. It is not especially rigorous, and since it relies on O’Neil’s personal recollections, it is not transparent either. However, it works well within the limited scale of the household.

The third model introduced in this chapter is used to predict recidivism rates. Versions of this model, O’Neil notes, correlate race with recidivism in a way that has led to aggravated sentencing for some Black convicted felons. More recent models do not include race directly; however, they do ask many questions whose answers correlate with race and socioeconomic status. O’Neil asks the reader to consider whether the use of such models represents the elimination or the codification of prejudice.

Finally, O’Neil brings the examples together to develop her criteria for a Weapon of Math Destruction (WMD). She observes that many influential institutions, from governments to banks, apply models that are either unavailable for public examination or prohibitively complex for nonexperts to assess. These same models are often deployed at large scale and are potentially damaging in the decisions they generate. Collectively, these three traits—“Opacity, Scale, and Damage”—provide the basic definition of WMDs.

Chapter 2: Shell Shocked: My Journey of Disillusionment

Here, O’Neil provides the more detailed autobiographical background behind her critique of Big Data practices. She shares her experiences working at a hedge fund as a “quant,” or quantitative analyst, where her colleagues were eager to accumulate “dumb money” from less active and less sophisticated investors. O’Neil narrates the unease, then the panic, as the global financial crisis of 2008 came into view. She explains the role of mortgage-backed securities in generating that crisis, then tells of leaving to find work in the risk management business that boomed after 2008. Finding that she was being asked to blithely sign off on companies’ risky behavior, she left again to work in e-commerce before beginning the research that became Weapons of Math Destruction.

Analysis: Introduction and Chapters 1–2

Although Weapons of Math Destruction is written for the general public and contains little technical jargon, a few terms bear defining because they are central to the author’s argument. O’Neil herself gives an extensive explanation of what constitutes a model, providing examples from sports, household management, and criminal sentencing (Chapter 1) as well as finance (Chapter 2). The gist is that any simplified representation of reality for decision-making purposes can be called a model.

The models that occupy most of O’Neil’s book, however, are formal ones based on the collection and analysis of large-scale data (“Big Data”). These models use advanced statistical tools to determine how the different variables affect each other and to identify the factors with the biggest influence. O’Neil’s complaint is not that such models exist or that they are used to make important decisions but that they are misused in ways that harm many people and perpetuate unfairness. Her concept of WMDs (“Weapons of Math Destruction”) is a way of singling out the models most likely to cause such harm because they are widely applied without adequate accountability.

Elsewhere, O’Neil refers to “the age of the algorithm.” An algorithm in general is just a formal set of rules for making a decision or finding an answer; it does not need to be carried out by a computer. (Long division, for instance, is an algorithm even when done by hand.) In practice, however, people often use “algorithm” to refer to the much more complex processes that decide which videos appear in a person’s YouTube recommendations or whether a person should be approved for a loan. O’Neil is concerned with algorithms whose workings are opaque, in the sense that they are either not made available to the public or are too complex for most laypeople to understand.