THE BOOKLET OF STATISTICAL VICES Julian L. Simon William Bennett, author of the best-selling The Book of Virtues, also has written The Index of Leading Cultural Indica- tors, widely distributed as a short pamphlet. It should have been called The Booklet of Statistical Vices, because it is an awesome and educational display of misleading statistical contrivances. (Recently it has also been issued in longer form.) "[W]e have experienced substantial social regression...over the last 30 years," Bennett's introduction says. Scary. But what's the evidence? "Number of Crimes Committed" is the first topic. Bennett presents one chart showing "Millions of Violent Crimes" and another showing "Millions of Total Crimes." Both embody a clas- sic obfuscation - no adjustment for the growth of the country's population since 1960. Therefore crimes per capita - the rele- vant measure - have not grown so fast as Bennett's charts show. (Bennett notes this in his discussion, but that's like small print in a contract. It's the dramatically rising curves in the graphs that hit the reader over the head.) Even more crucial is the type of data used, as John Stossell showed in a recent ABC documentary. Bennett uses the FBI series, which refer only to reported crimes. The rate of reporting has increased greatly in recent decades, thereby biasing the FBI data trends. Since 1973 the U. S. Bureau of Justice Statistics has there- fore conducted "victimization surveys." The results are mind- boggling : no apparent growth in violent crime (maybe a decline), with a large decline in non-violent larceny, burglary, theft, and auto theft. But Bennett presents only the FBI series that support his thesis. Bennett's second topic is "Median Prison Sentence for All Serious Crimes." His graph shows a decline from 25 days in 1954 to 5 days in 1974, and then a modest rise to 1990, suggesting that the society has gotten soft on punishing crime. Can five days in prison really be the "median sentence"? Of course not. Reading the fine print you find out that the numbers do not refer in any way to the "median sentence" that is in the title, but rather are a computation called "expected punishment". This includes the chances of being arrested, of going to trial, of being convicted, and getting any prison sentence, as well as the sentence itself. This concept is interesting and probably useful, but has almost no connection with what Bennett claims to discuss. This abuses statistics akin to the non-sequitur fallacy in rhetoric. Another common statistical misuse is wrongly combining apples and oranges - the "fallacy of composition." Does "serious crimes" in the "median sentence" graph have the same composition of crimes over the decades? Certainly not. Homicide and rape and theft have changed at vastly different rates, so lumping them together produces confusion at best. Why does Bennett start this graph in 1954 whereas the first set of graphs (and most of Bennett's others) start in 1960? Answer: Starting this one at 1960 would make a less dramatic chart. Similarly, starting the crime graphs earlier than 1960 also would seem less dramatic. Taken together, these graphs suggest that the presentation of crime data is rigged to get the worrisome effect that the author wants to achieve. Bennett's third topic is "Juvenile Violent Crime Arrest Rates". This graph Bennett at least puts on a "per 100,000" basis. But he does not tell us what the "100,000" refers to. Juveniles? Population? One cannot know what mischief may lurk behind the undefined number. Vagueness of definition is one of the most useful practices for statistical doubletalk. The "Juvenile ..." graph contains all the fallacies in the earlier chart of total violent crimes, plus a new one: The vertical axis in the graph does not start at zero. This is a real meat-ax of a crude statistical trick, grade-school stuff. If the plot had started at zero the rise in the curve would have been much less eye-catching. Next we notice that the subject is arrests rather than crimes, though the casual reader is not likely to notice the shift. Is it possible that juveniles have been getting arrested more frequently for given crimes than in the past? We don't know. But if so, arrests would give a misleading impression of crime. Shifting definitions is a most useful contrivance for portraying a false statistical picture. The fourth topic is "Children Relying on AFDC". Again Bennett uses the now-familiar device of showing total numbers of children without adjusting for population increase. And again Bennett mistitles the graph as "Relying on", when the data refer to receiving AFDC. Who knows how many children get AFDC who do not rely on it at all? Indeed, how many children received AFDC half a century ago? How many children receive AFDC-type payments in Somalia? Mighty few, even though the children were and are more needy in those cases than in the U.S. now. The explanation, of course, is that the graph probably shows more generous government programs rather than greater need or reliance. (Indeed, this is suggested by the rapid rise in AFDC receipt shown between 1960 and 1975, with constancy since then.) So: First four topics in the short version - five graphs, all misleading. This is a bravura display of statistical obfuscation, providing a wonderful case study for my elementary statistics course. Julian L. Simon teaches business administration at the University of Maryland. In the 1960s he pioneered what has become the revolutionary "new statistics" - the resampling simulation method. 301-951-0922 fax 301-951-8468 110 Primrose St., Chevy Chase, Md. page 1 /article4 bennett/June 13, 1994