WHY JOHNNIES (AND MAYBE YOU) HATE MATH AND STATISTICS Julian L. Simon Many folks who are not good at math believe it means that they are not "smart". And math-clever people often regard the math-allergic as stupid. "Among mathematicians and statisticians who teach introductory statistics, there is a tendency to view students who are not skillful in mathematics as unintelligent", says the instructor's guide to the best-selling introductory statistics text package (McCabe and McCabe, 1989, p. 2). Indeed, a book on The Psychology of Learning Mathematics says: Surely the main ability required for mathematics [is] the ability to form and manipulate abstract ideas; and surely this ability coincided closely with what we mean by intelligence[.](Skemp, p. 15) This is an unfair charge against many capable human beings, and a nasty, destructive conceit of the math-clever. Many of the math-allergic hate math for the best of reasons: Their teachers demand that they perform tasks by rote that they do not understand. And they do not understand because the tasks simply cannot be understood intuitively. People are forced to manipulate x's and y's whose meanings are difficult to remember from one moment to the next, for mathematical reasons that even their high-school and university teachers have forgotten - if the teachers themselves ever understood the reasons. But is all this conventional abstract complexity necessary? Surprisingly, no; much of the work that needs doing can be done in less-complex ways that are just as good theoretically and more effective in practice. Some would respond that it is better to handle mathematical tasks with the usual mumbo-jumbo in order to learn to think better. No one has ever shown this claim to be true, however. And in fact this sort of abstract training may damage one's mental capacities in some crucial ways. In brief, we have here what may be the greatest intellectual fraud of all time: Folks are being sold a false bill of goods when they are told that it is good for them to learn mathematics the way it is customarily taught in courses in statistics and calculus. Of course this is a radical assertion, but read on to see if this claim will stand up to your scrutiny. Even as I am promising to skin off the emperor's clothes, the question may be forming in your mind: If this assertion is true, why do we hear and believe that the difficult higher mathematics taught in school and college is necessary to learn? Here is a temporary short answer: Instructors of mathematics tell you that math must be done the conventional formulaic way because they themselves love the mysterious machinery of equations and proofs, and because they enjoy the puzzle-solving nature of deductive proofs - but also because it is to their personal advantage to have you believe that they are the high priests of the intellect. They convince you to collabo- rate in your own confusion and humiliation because you share with the mathematicians the belief that being clever and quick with symbols is the sign of a capable and respectworthy person. You are a member of the cult of being smart. You admire your captors and torturers as prisoners too often admire their keepers. You accede to your hoodwinking because you, too, share the belief that doing something the clever way is better than doing it in a way that is more effective but seems less clever. Now let's get down to specifics. I will address the two subjects that are studied by the most students at colleges and universities - statistics, and calculus. I'll use statistics as my main case study because this is the field I've been trying to plow for more than a quarter of a century, and it may well be the most egregious fraud of all. Let's keep in mind the most basic difficulty with mathematics: Remembering what the abstract symbols stand for, so that when you manipulate the symbols you can understand what you are doing. There are other difficulties with equational mathematics, too, but it should be pretty clear that if you do not understand the meaning of the symbols you are working with, you can't possibly surmount any other difficulties. GEOMETRY, AND THE INTEGRAL CALCULUS Do you remember from high school geometry the formula for the hypotenuse (the slant line) of a right triangle? Bertrand Russell tells us that "No formula in the whole of mathematics has had such a distinguished history" (1925/1985,p. 96). For those of you who do not remember the formula, how about just measuring the hypotenuse? Interestingly, Russell says that "It is true that the 'proof' proved nothing, and that the only way to prove it is by experiment. It is also the case that the proposition is not quite true - it is only approximately true" (p. 96) because of the curvature of the earth. So what do we gain by learning to "prove" the formula? Is the formulaic answer more "exact" than a measurement? If the two legs truly are perpendicular, and if they are measured perfectly, than the formula will give a better answer. But these two conditions are never met in real life because there always is some error, and hence the formulaic answer will never be exactly exact. Whether it or a measurement will produce a better answer will differ from situation to situation. The formulaic answer is exact only after we move from real life into the mathmatician's never-never land of abstracted idealized magnitudes. Well, you say, what about when you cannot easily measure it, as is sometimes the case when measuring across water, or as was the case in ancient Egyptian farming, Bertrand Russell tells us? (p. 96). You could then make a model of the actual situation by drawing a picture with the two known legs to scale on paper, and measure the hypotenuse there - which is exactly what you do in the Navy to calculate distances at sea. Now how about the circumference of a circle? You probably remember that formula, but if you don't but you do you have the habit of making a simulation model - no problem; draw the figure on a piece of cardboard, cut it out, and measure it around with a tape measure. For a less regular shape you could use a sticky tape measure that will follow the in's and out's, or one of those neat pen-like devices where you trace the line with a little wheel-counter in the tip. That does the job even where no formula is available. In high school one learns to calculate the areas of straight-line forms with formulae whose proofs the student learns in geometry. But if the form includes curved lines, matters become more difficult. How about the area of a circle? If the formula does not come to you, the practical mathematician may turn to an ancient method that has been available to us since at least Archimedes the Greek: Draw the figure on a piece of graph paper, and count the number of squares within the figure. When you squeeze and elongate the circle into an ellipse, the formulas for the circumference and area no longer are part of that stock of knowledge that you (perhaps) brought with you from high school. Does this mean you are stuck? Not at all; the same cut-the-cardboard and count-the-squares methods that worked for the circle will work nicely here, too. Again, these formulaic methods seem attractive because they are thought to be "exact". But consider this: The formulaic calculation for the area of a circle is not exact; it is only an approximation, because any value of pi that is used - even a value with 100 digits - is only an approximation. And the methods that mathematicians have used to approximate pi for more than two thousand years since Archimedes have been variations on the method of count-the squares. (The Archimedean method that ruled for 2000 years puts one polygon - say, a hexagon - inside a given circle and circumscribes another hexagon around it, calculates the areas of the two hexagons, splits the difference between the two areas, and makes the result the basis of pi. Subsequent methods that counted the areas of thin rectangles within a hemisphere are even closer to square-counting in spirit. For more on the history of pi, see Beckmann, 1982.) When you squeeze and elongate the circle into an ellipse, the formula for the area no longer is part of that stock of knowledge that you (perhaps) brought with you from high school. Does this mean you cannot find its circumference? Not at all; the same count-the-squares method that worked for the circle will work nicely here, too. And if the curved figure is complex, even the skilled mathematician may not be able to find an appropriate formula with the integral calculus. In that case, the practical mathematician may turn to the count-the-squares method. That method would work fine if you own a saw mill and you must compute the area of the cross-section of a very irregular large tree. If the graph-paper method works for complex figures, why not use it for simple figures as well? It surely works, and there is no danger of using the wrong formula. Furthermore, if a young person is taught to calculate areas the graph-paper way, and then applies it to (say) a rectangle, the young person may well learn the formula by him/herself. The young person may then gain not only the skill to calculate areas, but also the habit of discovering for him/herself workable solutions to problems - and maybe a deeper knowledge of the deductive formulaic rule, too, because its nature was self-discovered. But even if the proof of the formula is not discovered - what then? To answer that, ask yourself if you have carried through life from school days and into your present knowledge the nature of the proof of the formula for the area of a parallelogram or even that of a rectangle. Is that knowledge necessary for you - other than to help your kids learn the same thing you learned, so that they can teach their kids, so that their kids can teach their kids, so ... ? Circumference, area - now the volume of a solid. Yes, the volume of a cube is probably easy to calculate. But if you don't know how, or if the solid is irregular or curved, turn your memory to (supposedly) Archimedes, who went flying out of his bathtub and into the street stark naked shouting "Eureka", "I've found it", when the king wanted him to determine if a gold ornament had been debased with a cheap metal. (If you know the weight of an object, you know how much volume an amount of gold of that weight displaces.) To get the volume of solid in which you are interested, fill the bathtub to the level at which it overflows a drip hole, then insert the solid and catch the water that overflows. The overflow is a measure of the volume. (Of course the solid must not float, an easy thing to arrange in most cases.) Does this mean that youths should not study the rules of geometry and their deductive proofs, and instead devote the time to such subjects as history, computer science, and languages? I won't answer that question other than to say that anyone who contemplates a future of being a professional mathematician, or even an engineer, almost surely can benefit from the conventional training. But I would ask you to ask yourself: Who would you prefer working with you in most practical enterprises - a person who has learned and forgotten the rules of geometry, or the person who has the habit of finding a practical method of modeling and measuring the problem at hand? Here we may find evidence of that curious phenomenon known as "trained incapacity". It would be interesting to actually collect data from a randomly-drawn sample of artisans who have never studied geometry and a randomly-drawn sample of scientific academics who have studied that subject, and pose to both groups this problem: "Consider this curved shape (say an ellipse). Would you please give me your best estimate of its circumference?" I'll bet that the artisans perform better. Martin Gardner, for decades the mathematical puzzle editor of Scientific American, suggests that all mathematics is inherently difficult to teach. A teacher of mathematics, no matter how much he loves his subject and how strong his desire to communicate, is perpetually faced with one overwhelming difficulty: How can he keep his students awake? (Gardner, 1977, p. x) This difficulty vanishes when the teaching does not include formulas. Mathematical physicist John Barrow invented a revealing scenario about proof-based mathematics. He imagined what might happen if we were to receive a response from Martians to an Earth-transmitted extra-terrestrial messages. Those messages depend heavily upon mathematics, on the assumption that that will be the easiest for the Martians to decode. Barrow writes first about the excitement: There is great excitement at NASA today. Years of patient listening have finally borne fruit. Contact has been found. Soon the initial euphoria turns to ecstasy as computer scientists discover that they are eavesdropping not upon random chit-chat but a systemat- ic broadcast of some advanced civilisation's mathemati- cal information bank. The first files to be decoded list all the contents of the detailed archives to come. Terrestrial mathematicians are staggered: at first they see listings of results that they know, then hundreds of new ones including all the great unsolved problems of human mathematics.... Soon, the computer files of the extraterrestrials' mathematical textbooks begin to arrive on earth for decoding and are translated and compiled into English to await study by the most distinguished representatives of the International Mathematical Congress. Mathematicians and journalists all over the world wait expectantly for the first reactions to this treasure chest of ideas. Then he writes about the next peculiar reaction: But odd things happened: the mathematicians' first press conference was postponed, then it was cancelled without explanation. Disappointed participants were seen leaving, expressionless, making no comment; the whole atmosphere of euphoria seemed to have evaporated. After some days still no official statement had been made but rumours had begun to circulate around the mathematical world. The extraterrestrials' mathematics was not like ours at all. In fact, it was horrible. They saw mathematics as another branch of science in which all the facts were established by observation or experiment (Barrow, 1992, pp. 178-179). The key element in Barrow's story is the disappointment. Terrestrial mathematicians are not excited by a method that simply offers answers or solutions. To be acceptable, the method must also meet the aesthetic test of deductive proof. CALCULUS MADE EASY Now the differential calculus. To put you in the proper mood, take note that the World Book Encyclopedia publised at the time of my youth had just a single paragraph on calculus, saying that it is "the highest branch of mathematics, studied only in colleges and universities after a thorough preparation in algebra, geometry and trignometry". In other words, it is too difficult a subject for the encyclopedia to teach you. Does that impress and scare you? If (when) the following discussion of the calculus - which inevitably uses the algebraic notation that is such a barrier to understanding - loses your interest, please skip the rest of it and go on to the following subject, statistics; the discussion there is entirely free of algebraic notation. Consider a famous book called Calculus Made Easy: Being a Very-Simplest Introduction to Those Beautiful Methods of Reckoning Which Are Generally Called by the Terrifying Names of the Differential Calculus and the Integral Calculus, written at the turn of the twentieth century by Silvanus P. Thompson (1910/1946, third edition) F.R.S. The "F.R.S" means "Fellow of the Royal Society", one of the highest honors a scientist can achieve in Great Britain, so we can be sure that Thompson was no slouch at his profession. In the book's prologue we read: Being myself a remarkably stupid fellow, I have had to unteach myself the difficulties, and now beg to present to my fellow fools the parts that are not hard. Master these thoroughly, and the rest will follow. What one fool can do, another can. (italics added) Calculus Made Easy has been damned by every professional mathematician I have asked about it. So far as I know, it is not used in any calculus courses anywhere. Nevertheless, almost a century after its first publication, it still sells briskly in paperback even in college bookstores. It teaches a system of approximation that makes quite clear the central idea of calculus - the idea that is extraordinarily difficult to comprehend using the mathematician's elegant method of limits. And by "difficult to comprehend" I mean that the invention of the calculus required the powers of Isaac Newton and Gottfried Leibniz - the greatest mathematical minds in Europe for centuries. And even they did not fully understand their own invention, as we shall see. Thompson starts his book as follows: The preliminary terror, which chokes off most fifth- form [eleventh or twelfth grade in the U. S.] boys from even attempting to learn how to calculate [use calculus], can be abolished once for all by simply stating what is the meaning - in common-sense terms - of the two principal symbols that are used in calculating. These dreadful symbols are: 1) d which merely means "a little bit of" (p.3). Thus dx mdeans a litle bit of x". (Thompson's second "dreadful symbol" is "merely a long S", the integral sign, which we can leave aside here.) Then Thompson goes on: "All through the calculus we are dealing with quantities that are growing, and with rates of growth (p. 8)... right through the differential calculus we are hunting, hunting, hunting for a curious thing: a mere ratio, namely the proportion which dy bears to dx when both of them are growing" (p. 10). The quantity dy is a "little bit" of some "variable" y whose change depends in some way upon the extent of change in the variable x. For those of you who have had an encounter (perhaps painful) with the calculus, as well as for those of you who are entirely innocent of the subject, it may be helpful to provide a bit of history (which students are ordinarily deprived of, to their great detriment in understanding what the calculus is all about). But if the next few pages and their geometry and algebra get too tedious for you, skip them and move on to the rest of the story which is entirely free of all such paraphernalia. Isaac Newton's professor and predecessor at Cambridge University, Isaac Barrow, drew a graph like Figure 1, which portrays the flight of a cannonball. Say that we want to know the speed of the ball at point P (a typical simple problem that calculus answers for us). One can approximate the speed by drawing a small triangle with horizontal and vertical legs centered below P, and counting the (perhaps fractional) number of units of increase in the vertical axis portraying variable y, per each unit of increase in the horizontal axis portraying the variable x. The result is a perfectly acceptable approximation of the typical result that we seek from the calculus. When we have done this WE HAVE ACHIEVED WITH A CRUDE EARLY APPROACH TO THE PROBLEM WHAT THE CALCULUS ACHIEVES. THE REST IS ELABORATION. Figure 1 This method has some defects, however. Unless the triangle is very small, it may yield too crude an approximation, especially if the curve is changing direction at that point. And carrying out this method means that we must be able to draw a neat, sharp graph. Barrow refined this method and got within an ace of the modern calculus. He, like most others, chose as a first example a parabola (the cannon-ball path, usually drawn as a U-shaped figure that plots the equation y = x2). He drew a line tangent to the curve at P, and extended it to the baseline. He could then use geometry (trignometry, if you like) to calculate the ratio of the increase in y to a unit increase in x, which may then be written as a ratio of the "dreadful symbols", dx/dy. But perhaps we are not good at drafting, and can't accurately sketch in the curve between the points that you plot. And if there are more than two variables involved or if things otherwise get complicated - for example, if you want to know the rate at which water is rising in a cone-shaped vessel - a graph may not be workable. And you may desire a general rule for making the computation, and not just the specific number at a given point. For those reasons, as well as for esthetic reasons, mathematicians wanted a more powerful method. Thompson explains to us the nature of that more powerful method, but manages to avoid entirely the major mystery of the usual way of approaching the modern method. You could compute the tangent at different points along a parabola, and you might notice that at every point the rate is 2x. Good try. But that inductive method will not satisfy mathematicians, and it will not work where you cannot construct the graph. Now let's see how Thompson proceeds. In his initial chapter entitled "On Different Degrees of Smallness", Thompson writes: [W]e have to deal with small quantities [that are] of various degrees of smallness. We shall have also to learn under what circumstances we may consider small quantities to be so minute that we may omit them from consideration. Everything depends upon relative minuteness (p.3). Thompson begins work with the standard example of a parabola, and the equation y = x2. Like others before him, he considers what happens when you pick a point on the parabola and make a small addition to it on the x axis. He tells us to add a "little bit" of x and the corresponding little bit of y to the equation, to get y + dy = (x + dx)2. Then he actually does the algebra of squaring to get y + dy = (x + dx)2 = x2 + 2x*dx + (dx)2, where the asterisk means to multiply 2x by (dx); "dx" can be an ambiguous symbol. Thompson then subtracts the equal quantities y and x2 from the left and right sides of the equation to get dy = 2x*dx + (dx)2. Now comes the crucial part of Thompson's method: He points out to us that (dx)2 is only a little bit of a little bit, which means that it must be insignificantly small (he has early taught us how to decide when things are insignificantly small), and therefore we can throw it away - just ignore it from here on. After throwing awat the little bit of a little bit, we can re- write the last previous equation as dy = 2x*dx. The act of throwing away the little bit of a little bit leaves us with one more tiny step to get to the "derivative" we seek. We tidy up algebraically by rewriting as dy/dx = 2 which is what we are looking for, and exactly what you obtain with the conventional method. Thompson uses a method of approximation, but he arrives at the same exact formula as Newton and Leibniz. If you have followed what has just been done, you now understand the heart of the differential calculus. All the rest is application of this very same method to more complicated algebraic constructions. If you read Thompson's book you will grasp what calculus is all about - what it is useful for, and the nature of its power. After you understand Thompson's explanation you are better prepared to understand the conventional approach - if you still need to or want to. Now contrast the way the subject is taught - the way it was in the text I tried to learn calculus from by correspondence when I was in the Navy, and later again in graduate school. (I flunked most of the lessons I sent from my ship to the University of Wisconsin, and you might say that my attack on the conventional method is just because I myself failed. Well, if so, why not? What's wrong with learning to sympathise with the difficulties of others by experiencing the same difficulty yourself? One of the drawbacks of having this subject taught by professional mathematicians is that they have not suffered the same difficulties as the rest of us.) Instead of throwing away the term (dx)2 as Thompson did in the equation we had above, y + dy = x2 + 2x*dx + (dx)2. the standard method first cancels the y and the x2, and then divides by dx to get dy/dx = 2x*dx + dx. Here is where all the fancy reasoning about limits enters - in discussing what happens as dx becomes smaller and smaller. The endpoint (more or less) is that dx can be made to disappear logically, either by saying "It is plain" that this happens, or with exceedingly tight mathematical logic about "neighborhoods" around the points x and y2. So we get to dy/dx = 2dx which is exactly where Thompson got to. You may object that throwing away the little bit of a little bit, as Thompson does, is "merely" an approximation. But in fact (and despite the assertion of many texts that the standard method is "exact") the standard method of "limits" is "merely" a super- rigorous approximation, not in essential quality different from Thompson's method of approximation. The difference is that Thompson deals straightforwardly with relatively small quantities whereas the standard method twists itself into a pretzel dealing with nearly infinitely small quantities. (When I say "nearly" I mean small to any standard of comparison, but never so small as to be non-existent. How's that for neat mathematical logic?) Consider this analogy: The houses in a new development are arranged along a curve as in Figure 2. The curve has the shape y = x2. Consider x to be the house number, and the price will equal y. Very unusually for houses (but demanded by the calcu- lus), all houses are infinitely thin. There are houses at all places along the curve from 100 to 200 except at number 150, where no house exists. The question is: What is the rate of change of price at house number 150? Figure 2 Peculiar as it may seem, the calculus will answer this question about the rate at house 150 even though house 150 does not exist. That is because the rate of change of price at house 150 is the limit of the process, and it is calculated on the basis of the (almost) infinitely thin houses next to it. This shows you that even the standard method of deriving the calculus is only an approximation rather than an "exact" method. If it were exact, it could not calculate the rate at house 150 because that house does not exist. The house-price example is related to the famous paradox of the Greek philosopher Zeno: If a turtle covers half the distance to its goal in each successive period, does this mean that it will never reach its goal? The standard method of the calculus explains the paradox because the calculus is founded on the same concept as the paradox. As the Grolier Encyclopedia (1973) asserts: The fundamental concept of calculus, which distingishes it from other branches of mathematics and is the source from which all its theory and appplications are developed, is the theory of limits of functions of variables (vol. 4, p. 452). Thompson's method never has to employ the concept of the limit or grapple with paradoxes like that of Zeno. The text I was assigned in graduate school typically typically teaches you none of the history above. First the author, Donald Richmond, goes into great detail about the nature of the concept of a "limit", upon which the refined modern method depends, and then he strips down the explanation of finding the derivative to the shortest possible space. To make the matter even more obscure, his first (and central) graph shows decreases in x and y instead of increases, in order to make his algebra a bit briefer later on (Richmond, 1950, p. 70). That undoubtedly seems like a trivial change to Richmond. But it is no accident that the famous mathematician-philosopher Alfred Whitehead de- fined calculus as the "systematic consideration of the rates of increase of functions" (1911/1948?, p. 164, italics added), a definition quoted widely (e. g., Allen, 1956). In practice we are more often interested in an increase than in a decrease. And increase is easier to understand algebraically. So shifting to the examination of decrease is costly to the student in ease of understanding. But author Richmond makes clear that understanding by the average non-mathematician is not his central goal. He uses a different notation (in D's) instead of dx/dy, and after mention- ing the dx/dy notation he says: It should be emphasized... that convenient as the differential notation is for remembering results... the D notation lends itself better to the construction of satisfactory proofs and was therefore adopted at the outset (p. 100). And we remember from Barrow's example of the Martians that logical proof all-important to mathematicians. Here is another illustration of the unnecessary but brain- breaking complexity in the standard method: The Encyclopedia Brittanica (1946 edition) begins work with the notation [trian- gle] x and [triangle] y and writes them as a ratio. When the author finishes the derivation he shifts to the dy/dx notation but abjures you never to think of that notation as a ratio - because the meaning has shifted as the limit is approached. In other words, [triangle] y /[triangle] x approaches dy/dx as a limit, but somewhere along the line it becomes something differ- ent than what it was - no longer being a ratio. How? When? Why? Get someone else to answer those questions for you. Those questions don't arise for Thompson at all, because he has no need whatsoever for the concept of a limit. Question: why don't high school and college kids get to learn calculus the Thompson way? Answer: Thompson's system has an unremediable fatal flaw: It is ugly in the eyes of the world- class mathematicians who set the standards for the way mathematics is taught all down the line; the run-of-the-mill college and high school teachers, and ultimately their students, are subject to this hegemony of the aesthetic tastes of the great. Thompson simply avoids the deductive devices that enthrall mathematicians with their beauty and elegance. It does not enhance the "construction of satisfactory proofs" that is the goal of professional mathematicians such as Richmond, quoted above. One more item before we leave the calculus: According to Whitehead (who should know), Newton and Leibniz did not fully understand the method they created. The entire business of limits, the heart of the modern explanation, is very deep philosophically, almost mysterious. [W]e are tempted to say that for ideal accuracy an infinitely small period is required. The older mathematicians, in particular Leibniz, were not only tempted, but yielded to the temptation (p. 167). Leibniz held that, mysterious as it may sound, there were actually existing such things as infinitely small quantities, and of course infinitely small numbers corresponding to them. Newton's language and ideas were more on the modern lines; but he did not succeed in explaining the matter with such explicitness so as to be evidently doing more than explain Leibniz's ideas in rather indirect language. pp. 168-169). Whitehead tells us that "The real explanation of the subject was first given by Weierstrass and the Berlin School of mathematicians aobut the middle of the nineteenth century. But between Leibniz and Weierstrass a copious literature, both mathematical and philosophical, had grown up round these myserious infinitely small quantities which mathematics had discovered and philosophy proceeded to explain...the general effect...was to generate a large amount of bad philosophy... The relics of this verbiage may still be found in the explanations of many elementary mathematical text-books on the Differential Calculus" (pp. 169-170). Of course when Whitehead goes on to explain the calculus himself, he gives us what he considers to be a clearer explanation of the Weierstrass approach - rather than something like Thompson's approach. (Interestingly, Thompson's book was published almost contemporareously with Whitehead's book.) Given that the standard route to the "modern method" is so difficult that even its inventors Newton and Leibniz did not understand it thoroughly, does it make sense to press it upon non-mathematically-minded young people when a better method of stating it is available? DO YOU NEED BEAUTY, OR DO YOU NEED PRACTICAL TOOLS? If you ask them about Thompson, mathematicians talk about gaining "insight" with the conventional method. And they will mention the holy mathematical trinity of "elegance", "rigor", and "sophistication". But Albert Einstein said that in mathematical physics "matters of elegance ought to be left to the tailor and to the cobbler" (1916, p. i). What the mathematicians will not admit is that what (maybe) gives insight to them, and pleases them with its elegance and sophistication, only gives the non-mathematician a fast headache and a long-continuing case of brain fog. Here is the mind set of the mathematician, as expressed by one of the great mathematicians of the century: A mathematician, like a painter or a poet, is a maker of patterns...The mathematician's patterns, like the painter's or the poet's, must be beautiful (C. H. Hardy, A Mathematician's Apology, 1967, pp. 84-85, italics in original) Here is a more detailed statement of the same mind set: [T]he mathematician... may be driven to creative activity, as is the poet or painter, by pride in his reasoning faculty, the spirit of exploration, and the desire to express himself ... It offers intellectual delight and the exaltation of resolving the mysteries of the universe... Mathematicians enjoy the excitement of the quest, the thrill of discovery, the sense of adventure, the satisfaction of mastering difficulties, the pride of achievement or, if one wishes, the exaltation of the ego, and the intoxication of success... As man's greatest and most successful intellectual experiment, mathematics demonstrates manifestly how powerful our rational faculty is. It is the finest expression of man's intellectual strength. (Kline, 1985, pp. 551-3) Consider this analogy: A group of people is given the problem of escaping from a formal box-hedge maze on an English country estate. A maze is a classic mathematical problem. The mathematician would have you use your deductive problem-solving powers to find that path through the maze that leads to the exit with the fewest steps, or in the shortest time. And when set the problem, most students will dutifully set out to do what the mathematical teachers expect, and accept their teachers' judgments about their success or failure. But another person might do differently: S/he whips out a machete and slashes ports through the hedges directly from the starting point to the finish. Or perhaps s/he gets a ladder and climbs over. Or swings over on a rope from a high tree. Or simply runs back out the entry point and then around the maze to the end point. One can guess the grade s/he would receive from a mathemati- cian for such behavior: "F" for flunk! Why? Because the maze was not "supposed" to be dealt with that way. But who decided what was "supposed" to be done? And was that even stated at the start? More likely the proper behavior was only implicitly specified, and people understood what was wanted because we are so well socialized to work by certain rules, usually deductively. This analogy reflects on the general value of training in deductive thinking. Ever since the Greek geometers, and probably before, it has been assumed that the possession of the sorts of skills that they taught implied that a person is a "better" thinker. But whom would you prefer to have with you if cast away on a deserted desert island, or working in a new factory you are setting up - the person who is good at the deductive mathematics of box hedges, or the person who finds another and quicker way to get to the finish point? I do not dismiss the value of the deductive thinker; s/he may be very helpful in setting up effi- cient production schemes in a factory. Rather, it is simply to say that the non-formal thinking is not necessarily inferior to the former. Indeed, by a curious irony, those who preach creativity in business and elsewhere are fond of the phrase "thinking that gets outside of the box," which refers to a famous deductive puzzle- problem of connecting a set of dots that trace a box; the solution is to extend a line beyond the apparent rectangular borders - "outside of the box", in violation of an implicit rule for behavior. THE NEW STATISTICS AND PROBABILITY So far I've talked mainly in generalities. I have not proven my main point with detailed evidence that you can assess for yourself. For that evidence, let's now talk about statistics and probability - arguably the most important tool of good thinking that a person can possess. In business, in government, in investments, in your personal life, you need the ability to sensibly answer practical questions concerning the uncertainties in decision-making. And it doesn't matter how you get those answers - whether you work with equations or simpler tools - as long as the answers are sound. Though of course it is rare for them to say so publicly, many mathematical statisticians are fully aware that the emperor is naked, and that the conventional introduction to statistics is a disaster. As two of them (one a former president of the American Statistical Association) wrote, "The great ideas of statistics are lost in a sea of algebra" (Wallis and Roberts, 1956, viii, xi). Another well-known statistician, Bradley Efron, writes (with Tibshirani, page xiv, 1993): "The traditional road to statistical knowledge is blocked, for most, by a formidable wall of mathematics". Robert Hogg argues that the formal equational approach is unsound not only because it is difficult, but also because it points the student away from deep understanding of scientific statistical problems. Statistics is often presented as a branch of mathematics, and good statistics is often equated with mathematical rigor or purity, rather than with careful thinking (1991). The authors of the most-respected introductory text say: [W]hen we started writing, we tried to teach the conventional notation... But it soon became clear that the algebra was getting in the way. For students with limited technical ability, mastering the notation demands so much effort that nothing is left over for the ideas. To make the point by analogy, it is as if most the undergraduates on the campus were required to take a course in Chinese history--and the history department insisted on teaching them in Chinese. (Freedman et al., 1991, from the introduction to the first edition) Indeed, it is well-accepted in the statistical profession that the attempt to teach the subject conventionally at an introductory level is a crashing failure. There is much written testimony to this effect by thoughtful critics of statistics education. Here are some examples: 1. Garfield (1991): "A review of the professional literature over the past thirty years reveals a consistent dissatisfaction with the way introductory statistics courses are taught" (p. 1). Garfield asserts (referring to her dissertation, 1981, and to work by Wise) that "It is a well known fact that many students have negative attitudes and anxiety about taking statistics courses" (p. 1). "Students enrolled in an introductory statistics course have criticized the course as being boring and unexciting... Instructors have also expressed concern that after completing the course many students are not able to solve statistical problems... (1981, quoting Duchastel, 1974). 2. Dallal (1990, p. 266): "[T]he field of statistics is littered with students who are frustrated by their courses, finish with no useful skills, and are turned off to the subject for life". 3. Hey (1983, p. xii): For more years than I care to recall, I have been teaching introductory statistics and econometrics to economics students. As many teachers and students are all too aware, this can be a painful experience for all concerned. Many will be familiar with the apparently never-ending quest for ways of reducing the pain - by redesigning courses and by using different texts or writing new ones. But the changes all too often turn out to be purely cosmetic, with the fundamental problem left unchanged. 4. Barlow (1989, Preface) Many science students acquire a distinctly negative attitude towards the subject of statistics...As a student I was no different from any other in this respect. 5. Hogg: "[S]tudents frequently view statistics as the worst course taken in college." He explains that "many of us are lousy teachers, and our efforts to improve are feeble" (1991, p. 342). 6. Vaisrub (1990) about her attempt to teach medical students conventional statistical methods: "I gazed into the sea of glazed eyes and forlorn faces, shocked by the looks of naked fear my appearance at the lectern prompted" . 7. Freedman et al. noting that most students of probability and statistics simply memorize the rules: "Blindly plugging into statistical formulas has caused a lot of confusion." (1991, p. xv) 8. Ruberg (1992): It seems that many people are deeply afraid of statistics. [They say] `Statistics was my worst subject' or `All those formulas'...I wish they had a deeper understanding of the statistical method...rather than the general confusion about which formulas are most appropriate for a particular data set. 9. Based on their review of the literature, Garfield and Ahlgren say that "students appear to have difficulties developing correct intuitions about fundamental ideas of probability", and they proceed to offer reasons why this is so (1988, p. 45). These sorts of negative comments are not commonly heard about other subjects and other groups of students; both the nature and the volume of the criticism with respect to statistics are unusual. I have been teaching economics, business, and demography for three decades without hearing such complaints in those fields. Indeed, in the last decade or so, the statistics discipline's graybeards have decided that probabilistic (inferential) statistics is just too tough a nut to crack, and have concluded that students should be taught mainly descriptive statistics -- tables and graphs -- rather than how to draw inferences probabilistically. But probability and inferential statistics are the heart of the matter. A statistics course without inferential statistics is like Hamlet without the Prince appearing. ENTER THE SOLUTION: RESAMPLING The new resampling approach to inferential statistics, using simulation instead of formulae, mitigates the problem. A physical process necessarily precedes any statistical procedure. Resampling methods stick close to the underlying physical process by simulating it, requiring less abstraction than classical methods. The abstruse structure of classical mathematical formulas cum tables of values based on restrictive assumptions concerning data distributions tend to separate the user from the actual data or physical process under consideration; this is a major source of statistical error. Resampling, and especially the specific device known as the "bootstrap", has most commonly been used by statisticians in situations where classical methods are inappropriate. But resampling has a more important role as the tool of first resort in everyday practice of statistical inference. Its greatest advantage is that there is a greater chance that a correct resampling test will be used than a correct classical test. It is easiest to understand the issue if we digress with some history. About 1615, Italian gamblers brought to Galileo Galilei a problem in the game of three dice. The theorists of the day had figured as equal the chances of getting totals of 9 and 10 (also 11 and 12), because there are the same number of ways (six) of making those points -- for example, a nine can be 126, 135, 144, 234, 225, and 333. But players had found that in practice 10 is made more often than 9. How come? Galileo then invented the device of the "sample space" of possible outcomes. He colored three dice white, gray, and black, and systematically listed every possible permutation. The previous theorists - including Gottfried Leibniz - had instead lumped together into a single category the various possible ways of getting (say) a 3, 3, and 4 to make 10. That is, they listed combinations rather than permutations, and various combinations contain different numbers of permutations. Galileo's analysis confirmed the gamblers' empirical results. Ten does come up more frequently than 9, because there are 27 permutations that add to 10 whereas there are only 25 permutations that add to 9. The use of repeated trials to learn what the gamblers wanted to know illustrates the power of the resampling method -- which we can simply call "simulation" or "experimentation" here. And with sufficient repetitions, one can arrive at as accurate an answer as desired. Not only is the resampling method adequate, but in the case of three dice it was a better method than deductive logic, because it gave the more correct answer. Though the only logic needed was enumeration of the possibilities, it was too difficult for the doctors of the day. The powers of a Galileo were necessary to produce the correct logic. Even after Galileo's achievement, the powers of Blaise Pascal and Pierre Fermat were needed to correctly analyze with the multiplication rule another such problem - the chance of at least one ace in four dice throws. (This problem, presented by the Chevalier de la Mere, is considered the origin of probability theory.) For lesser mathematical minds, the analysis was too difficult (though now it is an elementary problem). Yet ordinary players were able to discern the correct relative probabilities, even though the differences in probabilities are slight in both the Galileo and Pascal-Fermat problems. Simulation's effectiveness is its best argument. One might rejoin that the situation is different after Galileo, Pascal, Fermat and their descendants have invented analytic methods to handle such problems correctly. Why not use already existing analytic methods rather than resampling? The existence of a correct algorithm does not imply that it will be used appropriately, however. And a wrongly-chosen algorithm is far worse than no algorithm at all -- as the Chevalier's pocketbook attested. In our own day, decades of experience have proven that "pluginski" -- the memorization of formulas that one cannot possibly understand intuitively -- may enable one to survive examinations, but does not provide usable scientific tools. Resampling's advantage over deduction is evident in the now- famous problem of the three doors as popularized in Parade: The player faces three containers, one containing a prize and two empty. After the player chooses, s/he is shown by the "host" that one of the other two containers is empty. The player is now given the option of switching from her/his original choice to the other closed container. Should s/he do so? Answer: Switching doubles the chances of winning. The thousands of responses columnist Marilyn vos Savant received prove that logical mathematical deduction fails badly in this case. Most people -- laypersons and statisticians alike -- arrive at the wrong answer. Simulation, however -- and hands-on simulation with physical symbols, rather than computer simulation -- is a surefire way of getting and displaying the correct solution. Table 1 shows such a simple simulation with a random-number table. Column 1 repre- sents the box you choose, and column 2 where the prize is. Using the information in columns 1 and 2, the "host" opens a box - indicated in column 3 - and reveals it to be empty. Lastly, column 4 scores whether the "switch" or "remain" strategy would be preferable. A count of the number of winning cases for "switch" and the "remain" gives the result sought. Table 1 This process is transparent. Not only is the best choice obvious, but as you do a few random-number trials you are likely to understand why switching is better. No other mode of explanation or solution teaches this intuition so well. With many other problems in probability and statistics, too, simulation provides not only sound answers but also insight into why the process works as it does. In contrast, formulas produce obfuscation and confusion for most non-mathematicians. SEE RESAMPLING RUN Resampling is best understood by seeing it being learned. The instructor walks into a new class and immediately asks: "What are the chances that if I have four children that three of those children will be girls?" Someone says "Put a bunch of kids into a hat and pick out four at a time". After the laughs, Teach says, "Nice idea, but it might be a bit tough to do...Other suggestions?" Someone else says, "Have four kids and see what you get." Teach praises this idea because it points toward learning from experiment, one of the key methods of science. Then s/he adds, "But let's say you have four children once. Will that be enough to give you an acceptable answer?" So they discuss how big a sample is needed, which brings out the important principle of variability in the samples you draw. Teach then notes that to have (say) a hundred families, it could take quite some time, plus some energy and money, so it doesn't seem to be practical at the moment. "Another suggestion?" Someone suggests taking a survey of families with four children. Teach applauds this idea, too, because it focuses on getting an answer by going out and looking at the world. But what if a faster answer is needed? A student wonders if it is possible to "do something that is like having kids. Put an equal number of red and black balls in a pot, and pull four of them out. That would be like a family." This kicks off discussion about how many balls are needed, and how they should be drawn, which brings out some of the main concepts in probability -- sampling with or without replacement, independence, and the like. Then somebody wonders whether the chance of having a girl the first time you have a child is the same as the chance of a red ball from an urn with even numbers of red and black balls, an important question indeed. This leads to discussion of whether 50-50 is a good approximation. This brings up the question of the purpose of the estimate, and the instructor suggests that a clothing manufacturer wants to know how many sets of matched girls' dresses to make. For that purpose, 50-50 is okay, the class says. Coins are easier to use than balls, all concur. Someone wonders whether four coins flipped once give the same answer as one coin flipped four times. Eventually all agree that trying it both ways is the best way to answer the question. Teach commissions one student to orchestrate the rest of the class in a coin-flipping exercise. Then the question arises: Is one sample of (say) thirty coin-flip "families" enough? So the exercise is repeated several times, and the class is impressed with the variability from one sample of thirty to the next. Once again the focus is upon variability, perhaps the most important idea inherent in prob-stats. WHY RESAMPLING EASILY GETS CORRECT ANSWERS Resampling is a much simpler intellectual task than the formulaic method because simulation obviates the need to calculate the number of points in the entire sample space. In all but the most elementary problems where simple permutations and combinations suffice, the calculations require advanced training and delicate judgment. Resampling avoids the complex abstraction of sample-space calculations by substituting the particular information about how elements in the sample are generated randomly in a specific event, as learned from the actual circumstances; the analytic method does not use this information. In the case of the gamblers prior to Galileo, resampling used the (assumed) facts that three fair dice are thrown with an equal chance of any outcome, and they took advantage of experience with many such events performed one at a time; in contrast, Galileo made no use of the actual stochastic element of the situation, and gained no information from a sample of such trials, but rather replaced all possible sequences by exhaustive computation. Contrary to what many mathematicians formerly claimed, the analytic method for obtaining solutions - using permutation and combination formulas, for example - is not theoretically superior to resampling. Resampling is not "just" a stochastic-simulation approximation to the formulaic method. It is a quite different route to the same endpoint, using different intellectual processes and utilizing different sorts of inputs; both resampling and formulaic calculation are shortcuts to estimation of the sample space and its partitions. As a partial analogy, resampling is like fixing a particular fault in an automobile with the aid of a how-to-do-it manual's checklist, the formulaic method like writing a book about the engineering principles of the auto; the engineer-author may be no better at fixing cars than the hobbyist is at writing engineering books, but the hobbyist can learn quicker than can the author all that he needs to know for a particular repair. The much lesser degree of intellectual difficulty is the source of the central advantage of resampling. It improves the probability that the user will arrive at a sound solution to a problem - the ultimate criterion for all except for pure mathematicians. A common objection is that resampling is not "exact" because the results are "only" a sample. Ironically, the basis of all statistics is sample data drawn from actual populations. Statisticians have only recently managed to win most of their battles against those bureaucrats and social scientists who, out of ignorance of statistics, believed that only a complete census of a country's population, or examination of every volume in a library, could give satisfactory information about unemployment rates or book sizes. Indeed, samples are sometimes even more accurate than censuses. Yet many of those same statisticians have been skittish about simulated samples of data points taken from the sample space - drawn far more randomly than the data themselves, even at best. They tend to want a complete "census" of the sample space, even when sampling is more likely to arrive at a correct answer because it is intellectually simpler (as with the gamblers and Galileo.) Resampling offends mathematical statisticians because it does not meet the main criteria of what has always been considered truth in mathematics. It butts up against the fundamental attitude of the mathematics profession toward non- proof-based "Martian" methods. [SECTION OUT?] SOME ORIGINS OF RESAMPLING [It was a case of too much book-learning, too little understanding. The students had swallowed but not digested a bundle of statistical ideas which now misled them, taught by professors who valued fancy mathematics even if useless or wrong. It was the spring of 1967 at the University of Illinois, in my course in research methods with four PhD students. I required each student to start and finish an empirical research project. Now the students were presenting their work in class. Each used wildly wrong statistical tests to analyze their data. All had had several courses in statistics. But when the time came to apply even the simplest statistical ideas and tests in their research projects, they were lost. Their courses had plainly failed to equip them with basic usable statistical tools. So I wondered: How could we teach the students to distill the meaning from their data? Given that the students' data had a random element, could not the data and the events that underlie the data be "modeled" with coins or cards or random numbers, doing away with any need for complicated formulas? Next class I asked the students the probability of getting three girls in a four-child family. After they recognized that they did not know the correct formula, I demanded an answer anyway. After suggesting some interesting other ideas - empirical experiments and surveys - one student suggested flipping coins. With that the class was off to the races. Soon the students were inventing ingenious ways to get answers -- and sound answers -- to even subtle questions in statistics and probability by flipping coins and using random numbers. Meyer Dwass in 1957, and J. H. Chung and D. A. S. Fraser in 1958, had suggested a resampling version of Fisher's permutation test for deciding whether two sample means differ from each other (which, in ignorance of their work, I reinvented.) Now I suggested handling all (or at least most) problems with resampling. I offered a wide illustrative variety of methods to cover the field, and taught a systematic procedure for the wide range of standard problems. Could even children learn this powerful way of dealing with the world's uncertainty? Max Beberman, the guru of the "new math" who then headed the mathematics department in the University of Illinois High School, quickly agreed that the method had promise, and suggested teaching the method to a class of volunteer juniors and seniors. The kids had a ball. In six class hours they were able to discover solutions and generate correct numerical answers for all the major problems ordinarily found in a semester-long university introductory statistics class. And they loved it. Students in an introductory university class complained that coins and random number tables get tiresome. So in 1973 a computer language was developed to do with the machine what one's hands do with cards or dice - RESAMPLING STATS. Kenneth Travers recruited several PhD students at the U. of I's College of Education for theses on experiments teaching and comparing resampling and conventional methods, and in 1976 the successful results were published in American Mathematical Monthly. A turning point came in the late 1970s when a major sub-part of the general resampling method -- called the "bootstrap" -- was developed extensively by Bradley Efron and Stanford colleagues, and now has swept the field of mathematical statistics and provided legitimacy. By 1991 a National Research Council publication referred to "statisticians' infatuation with resam- pling methodology", saying that "Resampling techniques and semi- parametric models freed investigators from the shackles of normal theory and linear models". And in Science News, the then- Executive Director of the American Statistical Association, Barbara Bailar, said: [Resampling] has had a tremendous influence on the route statistics is taking," Bailar says. "It's a very powerful technique -- a major contribution to the theory and practice of statistics. We're going to see more and more applications of it." PROOF THAT RESAMPLING WORKS Every day of the week people argue that their approach to education is better than the other fellow's. Few provide solid evidence for their claims, however. To show that this is not one of those unproven assertions, here is some evidence: As far back as the 1970s, controlled experiments (Simon, Atkinson, and Shevokas, 1976) found better classroom results for resampling methods than for conventional methods, even without the use of computers. Students handle more problems correctly, and like statistics much better, with resampling than with conventional methods. The experiments comparing the resampling method against conventional methods also showed that students enjoy learning statistics and probability this way. Recent surveys of student judgments of courses using the resampling method - including both introductory classes and graduate students in statistics, and taught at places ranging from Frederick Junior College to Stanford University Graduate School - show that students approve the method. They say they learn from it, would recommend a course taught with the resampling method, find it interesting, and use what they learned in years after they finish to course. The current students, like the students in the 1970s experiments, do not show the panic about this subject often shown by other students. This contrasts sharply with the less positive reactions of students learning by conventional methods, even when the same teachers teach both methods in the experiment. These results should constitute a prima facie case for at least trying out resampling in a wide variety of educational settings. But more empirical study would be welcome. The statistical utility of resampling and other methods is an empirical issue, and the test population should be non- statisticians. WHAT WILL HAPPEN WITH RESAMPLING STATISTICS? As would be expected, even those statisticians who are critical of the present substance and methods of teaching statistics, and who recognize how the algebraic devices lead to confusion in statistical practice, are ambivalent about it. For example, a recent president of the American Statistical Association called for research that would provide better methods - but would not even acknowledge its existence when shown evidence that the resampling method is indeed provably better. (Zellner, correspondence of November 12, 1990). The trouble in statistics teaching is in the product, and not the packaging and advertising. The theory that's usually taught to elementary [statistics] students is a watered-down version of a very complicated theory that was developed in order to avoid a great deal of numerical calculation... It's really quite a hard theory, and should be taught second, not first. (Efron quoted by Peterson, 1991, p. 56) Sooner or later the conventional enterprise smashes up against an impenetrable wall -- the body of complex algebra and tables that only a rare expert understands right down to the foundations. For example, almost no one teacher can write the formula for the "Normal" distribution that is at the heart of most statistical tests, let alone the students. Even fewer understand its meaning. Yet without such understanding, there can be only rote learning. One must note a certain schizophrenia. The very statisticians who assert that the problem is the "wall of algebra" proceed to use closed-form formulas heavily - even in their discussions of resampling which boasts that it renders much (if not all) of the formulaic approach nugatory (see for example Efron and Tibshirani, 1993; Hall, 1992; Westfall and Young, 1993). IN THOMPSON'S TITLE: "EPILOGUE AND APOLOGUE" The final words of Calculus Made Easy say it well: EPILOGUE AND APOLOGUE It may be confidently assumed that when this tractate Calculus Made Easy falls into the hands of the professional mathematicians, they will (if not too lazy) rise up as one man, and damn it as being a thoroughly bad book. Of that there can be, from their point of view, no possible manner of doubt whatever. It commits several most grievous and deplorable errors. First, it shows how ridiculously easy most of the operations of the calculus really are. Secondly, it gives away so many trade secrets. By showing you that what one fool can do, other fools can do also, it lets you see that these mathematical swells, who pride themselves on having mastered such an awfully difficult subject as the calculus, have no such great reason to be puffed up. They like you to think how terribly difficult it is, and don't want that superstition to be rudely dissipated. Thirdly, among the dreadful things they will say about "So Easy" is this: that there is an utter failure on the part of the author to demonstrate with rigid and satisfactory completeness the validity of sundry methods which he has presented in simple fashion, and has even dared to use in solving problems! But why should he not? You don't forbid the use of a watch to every person who does not know how to make one? You don't object to the musician playing on a violin that he has not himself constructed. You don't teach the rules of syntax to children until they have already become fluent in the use of speech. It would be equally absurd to require rigid demonstrations to be expounded to beginners in the calculus. One other thing will the professed mathematicians say about this thoroughly bad and vicious book: that the reason why it is so easy is because the author has left out all the things that are really difficult. And the ghastly fact about this accusation is that--it is true! That is, indeed, why the book has been written--written for the legion of innocents who have hitherto been deterred from acquiring the elements of the calculus by the stupid way in which its teaching is almost always presented. Any subject can be made repulsive by presenting it bristling with difficulties. The aim of this book is to enable beginners to learn its language, to acquire familiarity with its endearing simplicities, and to grasp its powerful methods of solving problems, without being compelled to toil through the intricate out-of-the-way (and mostly irrelevant) mathematical gymnastics so dear to the unpractical mathematician. There are amongst young engineers a number on whose ears the adage that what one fool can do, another can, may fall with a familiar sound. They are earnestly requested not to give the author away, nor to tell the mathematicians what a fool he really is (1910/1946, pp. 236-237). What Thompson says about the calculus and his method also is true about statistics and the resampling method. The author is well aware of the quixotic nature of the campaign described in this article. It might even be called the most foolish crusade ever undertaken. It is up against the largest constituency in the world: all those who think they are clever. And it has no constituency of its own. Why then undertake it? One might as well as why birds fly and fish swim. Some readers will chide me for the aggressive attitude I display here. They may even tell me that I am my own worst enemy, because the fighting words written here will anger those whom I should try to persuade with a positive approach. But for more than a quarter of a century I have been trying to persuade those persons with the most conciliatory words and thoughts that I could dream up, and I have been quite unsuccessful. The reason is that they see beyond the words to the reality - which is a threat to their situation. And no amount of statesmanship can get around that. This controversy is not about communication; it is about intellectual and occupational turf. REFERENCES Allen, R. G. D., Mathematical Analysis For Economists (London: Macmillan & Co. Ltd; New York: St. Martin's Press, 1956). Barlow, Roger, Statistics (New York: Wiley, 1989). Barrow, John D., Pi in the Sky: Counting, Thinking and Being (New York: Oxford UP, 1992) Beckmann, Petr, A History of Pi (Boulder, Colo: The Golem Press, 5th edition, 1982). Dallal, Gerard E., "Statistical Computing Packages: Dare We Abandon Their Teaching to Others?", in The American Statistician, November 1990, Vol. 44, No. 4, p. 265-6. Efron, Bradley and Tibshirani, Robert J., An Introduction to the Bootstrap (New York: Chapman and Hall, 1993). Einstein, Albert, Relativity (New York: Crown Press, 1916/1952). Freedman, David, Robert Pisani, Roger Purves, and Ani Adhikari, Instructor's Manual for Statistics (second edition) (New York: Norton, 1991). Gardner, Martin, Mathematical Carnival (New York: Vintage Books, 1977). Garfield, B. Joan, "Reforming the Introductory Statistics Course," paper presented at the American Educational Research Association Annual Meeting, Chicago, 1991. Garfield, Joan, and Andrew Ahlgren, "Difficulties in Learning Basic Concepts in Probability and Statistics: Implications for Research," Journal for Research in Mathematics Education, 1988, vol. 19, no. 1, 44-63. Hall, Peter, The Bootstrap and Edgeworth Expansion (New York: Springer-Verlag New York, Inc., 1992). Hardy, C. H., A Mathematician's Apology (New York: Cambridge U. P, 1967). Hey, John Dn D., Data in Doubt: An Introduction to Bayesian Statistical Inference for Economists (Oxford: Martin Robertson, 1983). Hogg, Robert V., "Statistical Education: Improvements are Badly Needed", The American Statistician, vol. 45, Nov. 1991, 342-343. Hogg, Robert V., "Statisticians Gather to Discuss Statisti- cal Education," Amstat News, November 1990. Kline, Morris, Mathematics for the Nonmathematician (New York: Dover Publications, Inc., 1985). McCabe, George P., and Linda Doyle McCabe, Instructors Guide with Solutions for Introduction to the Practice of Statistics (New York: W. H. Freeman, 1989). Peterson, Ivars, "Pick a Sample," Science News, July 27, 1991. Richmond, Donald E., Fundamentals of the Calculus (New York: McGraw-Hill Book Company, Inc., 1950). Ruberg, Stephen J., Biopharmaceutical Report, Vol 1, Summer, 1992. Russell, Bertrand, The ABC of Relaivity (New York: Mentor, 1925/1985). Simon, Julian L., Atkinson, David T., and Shevokas, Carolyn, "Probability and Statistics: Experimental Results of a Radically Different Teaching Method", American Mathematical Monthly, vol. 83, no. 9, Nov. 1976, pp. 733-739. Skemp, Richard, R., The Psychology of Learning Mathematics (New York: Penguin, 1971). Thompson, Silvanus P., Calculus Made Easy: Being a Very- Simplest Introduction to Those Beautiful Methods of Reckoning Which Are Generally Called by the Terrifying Names of the Differential Calculus and the Integral Calculus (New York: Macmillan, 1910/1946 third edition). Vaisrub, Naomie, Chance, Winter, 1990, p. 53. Wallis, W. Allen, and Harry V. Roberts, Statistics: A New Approach (Chicago: Free Press, 1956). Westfall, Peter H., and S. Stanley Young, Resampling-Based Multiple Testing (New York: Wiley, 1993). Whitehead, Alfred North, An Introduction to Mathematics (London, Oxford, and New York: Oxford University Press, 1969. First published in the Home University Library, 1911). Zellner, Arnold, correspondence of November 12, 1990.