Analytics Press
News

June 2, 2003  

Sorry, Wrong Number: Separating Fact from Fiction in the Information Age

"Sorry, Wrong Number," an article by the author of the book Turning Numbers into Knowledge, now appears in the June 2003 issue of IEEE Spectrum magazine (circulation 385,000). This article gives four examples of widely cited statistics that are either misleading or wrong, and describes simple techniques for separating fact from fiction in the information age

Getting the numbers right really matters. Virtually all business and policy decisions today are based on quantitative data, and no good can come of incorrect information becoming widely accepted. Annoyance, inconvenience, or even disaster could follow from important decisions based on faulty data.

Here are some simple techniques to avoid some of the most common pitfalls, summarized from the recommendations in the article and in the book Turning Numbers into Knowledge:

--Don't believe everything you read. Even renowned institutions can publish nonsense, and respected authorities are sometimes way off the mark. Just because the authors are famous or associated with a reputable firm or magazine doesn't mean that they are always right.

--Do your homework. Numbers frequently become disembodied—separated from the original source, detached from any caveats, or averaged or manipulated in inappropriate ways. These transformations can breed what the statistician and author Joel Best at the University of Delaware calls "mutant statistics": data twisted into new and incorrect forms.

--When in doubt, talk to experts in the field. People become experts because of a passionate interest. They really care about their topic and want other people to care, too. So go ahead and call them up. Your interest will make them happy, and you just might learn something significant.

--Rely on peer-reviewed research. Independent peer review ensures that research results accurately reflect current scientific knowledge. Be skeptical of results that are not peer reviewed. Some self-proclaimed experts announce their results to the media but do not publish in peer-reviewed journals. Usually, these announcements are funded by particular organizations with an ax to grind and have little to do with science.

--Dig into the footnotes. Lots of footnotes should not necessarily convey authority. You first need to check what they say!

--Follow the money.  Interest groups often fund research favorable to their causes.  Find where the funding comes from and you'll be better able to assess the credibility of the researchers.

Your best defense against being fooled is to think for yourself and do your homework. Never make important decisions based on "common knowledge" unless you can verify its accuracy independently. By thinking critically, you can be part of the solution rather than part of the problem.

Turning Numbers into Knowledge, authored by Jonathan Koomey, Ph.D., is a lively and entertaining guide to making informed professional and personal choices.  The book’s insights help people beat information overload, hone their decision-making skills, and achieve success in this information-glutted world.

For more details on the book Turning Numbers into Knowledge, go to www.numbersintoknowledge.com. Turning Numbers into Knowledge is distributed by the Independent Publishers Group (IPG), which can be reached at www.ipgbook.com.

IEEE SPECTRUM magazine, the flagship publication of the Institute of Electrical and Electronics Engineers (IEEE), explores the development, applications and implications of new technologies. It anticipates trends in engineering, science, and technology, and provides a forum for understanding, discussion and leadership in these areas (www.spectrum.ieee.org/ieeemedia/index.html).

##

Jonathan Koomey is a Staff Scientist at Lawrence Berkeley National Laboratory, in Berkeley, California. He holds M.S. and Ph.D. degrees from the Energy and Resources Group at the University of California at Berkeley, and an A.B. in History of Science from Harvard University.

Koomey is the author or co-author of seven books and more than one hundred and thirty articles and reports. Koomey’s latest book is Turning Numbers into Knowledge:  Mastering the Art of Problem Solving, ISBN 0-9706019-0-5, www.numbersintoknowledge.com.

Koomey has appeared on Nova/Frontline, BBC radio, CNBC, All Things Considered, Marketplace, Tech Nation, On the Media, the California Report, Tech TV, KPIX TV (SF), CNET radio, and KQED radio.  He has been quoted in the New York Times, the Wall Street Journal, Barron's, The Washington Post, Science, Science News, American Scientist, Dow Jones News Wires, USA Today, SF Chronicle, Oakland Tribune, Interactive Week, Business 2.0, Salon.com, and Network Magazine.