Getting the full story: How machine learning and data journalism can support democracy


Fake news. Echo chambers. Alternative facts. Post-truth. These words are increasingly popular in discussions of the role of journalism in the digital age. They are a worrying sign of a deepening distrust in news outlets and media online. This distrust, and the ways in which people navigate news content in the chaos of voices that is the Internet, must be addressed in order to foster more informed democratic debate among citizens today.


An ocean of information yet we’re stuck in a stream

Are these problems new? Not at all. They’ve been around since Gutenberg. However, the profound changes in how we get news online has made it worse. We are witnessing a shift towards social media as the primary source of news, particularly among younger demographics. This creates an ideological echo chamber where people believe they get a well-rounded view of issues when they are in fact reinforcing their biases by sharing news with friends holding similar political views.

The rise of social media as a distribution channel has also exacerbated the issue of fake news, as sites that do not uphold high journalistic standards are seen alongside - often equals to - their more rigorous counterparts. Click-bait headlines are shared more than investigative journalism due to their sensationalist appeal, reader confirmation bias and the efficacy of targeted advertising. The big players are weighing in on the problem, but the solutions proposed are imperfect and inherently problematic. You can’t easily tell someone what to trust. You can empower them to easily find relevant information and compare perspectives.


Democracy is drowning in data

A free and independent press may be vital to democracy, but what good is freedom and independence for these institutions when the web has put them on a level playing field with individuals who do not uphold the same standards of journalism? Investigative journalism is difficult and costly. Why pay a journalist to investigate when a short-form article with a gripping headline gets more hits? Why trust a journalist if anyone can write a story and publish it online? The sheer volume of blog posts online, let alone the 24-hour news cycle, results in total information overload.

Christopher Hitchens once remarked, “I became a journalist because I did not want to rely on newspapers for information.” I became a data journalist because I did not want to rely on search engines for information.

Results in search engines are personalised, based on your browsing history, and peppered with targeted ads. Searching for information on a news story and getting a list of hundreds of thousands of pages is not particularly informative at a glance. The time required to sift through the information is prohibitive. The information itself is questionable. Traditional search engines are great for a lot of things. For getting a well-rounded view of news, we can do much much better.

Machine learning and data visualisation can help

Over three years ago, I embarked on a journey with two friends and fellow Oxford alumni to change how we get news online and address many of these issues. It was a largely academic concern then. Political events have recently brought this issue into the mainstream. Two weeks ago, we launched a Kickstarter for Nupinion: The Smart News Toolbox For Digital Citizens.

Nupinion aims to mitigate fake news, combat ideological echo chambers caused by social media, and solve the problem of information overload in digital news. Nupinion is not a partisan project. We want to build a tool that can be used by anyone in the world to find news from anywhere easily. We dream (perhaps naively) of a world where everyone challenges their beliefs on a daily basis. We’re hoping people will help us build it so that we can remain ad-free and independent from special interests.

Nupinion aggregates news articles, meta-tags them and runs natural language processing algorithms on the content to meaningfully organise news and evaluate political bias. It summarises, translates and groups relevant news by topic. It simplifies information discovery of multiple perspectives, providing an easy way to explore differences in media coverage on any issue. By showing right-wing, left-wing and centrist positions on the same issue, it challenges the reader to recognise bias in media coverage and confront their ideological echo chamber.

Users seamlessly shift from a global, bird’s eye view of a topic to highly granular information about an article or actor. Each article is contextualised within the wider news ecosystem using interactive visualisations of geographic distribution, credibility, and political view. We have a public beta available running on limited sources with limited features. We greatly appreciate any feedback as we continue to improve the platform before full release.

Digital citizens deserve better data journalism

The goal of Nupinion is to support a healthy democratic system by improving the breadth of facts and opinions on a news story that anyone can access easily. We genuinely believe that well designed data journalism can help. People want to reduce information overload, minimise time spent doing research, lower the cost of media analytics tools, overcome language barriers and improve credibility-checking mechanisms to verify information. They also want easier ways to find different views on important social and political issues.

Nupinion aims to address all of the above and continuously improve the way we visualise news content to make insights from big data more intuitive, accessible and engaging. By making it easier for citizens to inform themselves and compare perspectives, we believe we support the democratic process and address the information needs of digital citizens. We hope that you will join us on this journey and help us build data journalism tools to achieve this.

Find out more here.