27/9/2011

The importance of numeracy for data journalists

 

In 1995, journalists relayed a report by the Committee on Safety of Medicines explaining that women who took the contraceptive pill had a 100 percent risk increase of lethal thrombosis. What did not make headlines was that the risk increase went from 1 in 7,000 to 2 in 7,000. 

Nevertheless, this convinced many to stop the pill and the number of abortions rose by 13,000 (source). Had all Brits reneged on the pill, approximately 15 lives a year would have been saved. Ironically, abortion is also linked to thrombosis, possibly in a stronger way (see pdf article).

It is up to every woman to make a choice about her sex life, but one needs clear information to make an informed decision. I guess few journalists had this trade-off in mind when reporting about thrombosis risks.

German scholar Prof. Gerd Gigerenzer made risk communication his main area of research. He evangelizes crowds about the dangers of communicating risks relatively (your risk of thrombosis will go up 100 percent) instead of absolutely (you have an additional 1 in 7,000 chance of developing thrombosis) among other mind-blowing research. His 2002 book, Calculated Risks, will help you understand and communicate risks effectively.

See how you fare in understanding numbers correctly by taking this simple test. It is adapted from a test (pdf) by medicine researchers Lisa Schwartz, Steven Woloshin and Gilbert Welch. 

 

 

Don’t worry if you did not succeed in the test. Most journalists have never received any training with numbers. This, however, starts to be recognized as a problem, says Prof. Gigerenzer. Going from conference to conference, he notices that journalists are aware of the topics he covers in his talks. The main reason for this positive evolution, though, seems to be that they had previously heard his talk.

Things are moving slowly. Calculated Risks was published almost 10 years ago. Since then, says Prof. Gigerenzer, very little has changed. A few textbooks make use of natural frequencies instead of statistics, a few medical schools have changed the way they communicate risks, but they remain a minority.

New tools brought by the data journalism craze will not help with data interpretation. “Tools don’t give insights”, says Prof. Gigerenzer. He quotes the advent of magnetic resonance imaging as a case in point. Looking at the scans on their computer screens, some get the impression that they see a person’s brain thinking, even though the tool only measures levels of oxygen in blood vessels.

Without a sound understanding of the data at hand, tools might merely add a layer of misunderstanding. We need to convince journalism schools to teach math in a purpose-oriented fashion. As Gigerenzer said in a 2010 conference, students are taught trigonometry but how to understand risks properly is overlooked. Decision-makers in J-schools and media companies need to realize that data in itself without better numeracy skills will not lead to better journalism. To tell true facts to their audience and to build trust, journalists need to gain the skills to understand and interpret data.

Asked about the perverse incentives of news and sensational risk reporting (big numbers sell more), Prof. Gigerenzer says that “we need a different kind of journalists, with more ethics”. In other words, we need to shift from a focus on circulation and page views numbers to a measure of trust (e.g. declarative trust in the brand). That’s where the value lies, and that’s where we ought to go.

Reading list

  • Calculated Risks, by Gerd Gigerenzer. The most comprehensive piece on how to communicate risks effectively and how to understand what a percentage really means. If you want to be able to debunk PR mystifications, that’s the place to start.
  • Numbers Rule Your World, by Kaiser Fung. The easier-to-read version of Gigerenzer’s book. Through 6 in-depth examples, it introduces you to similar concepts but remains more shallow.
  • Proofiness, by Charles Seife. Explains in an enjoyable way how numbers can be fabricated – and how you can deconstruct them.
  • Freakonomics Series, by Steven Levitt and Stephen Dubner. The funniest read in the list. While it won’t give you precise keys on how to deal with a certain figure, it is a good introduction on how to think with numbers.

Comments