25/10/2017

OONI data

 

Investigate internet censorship with OONI data.

By Maria Xynou, OONI

It’s easy to notice when popular platforms that we commonly use - like Google, WhatsApp, or Facebook - are blocked. The not-so-easy part is noticing the censorship of all those other, less popular platforms, such as the sites of minority groups. Due to their sensitive nature, minority group sites are probably more likely to be blocked. But who’s monitoring the accessibility of such sites on a daily basis?

Many cases of internet censorship around the world can go unnoticed. This is even more true when it’s not clear if a site or service has intentionally been blocked or not. The fact that you cannot access a website may not necessarily mean that your Internet Service Provider (ISP) is blocking access to it. Maybe the site owner is blocking all IP addresses originating from your country (in compliance with laws and regulations), or maybe the site itself is being hosted on an unreliable server. There are many reasons why you may not be able to access a website. But ultimately, this means that governments and/or ISPs can potentially seek plausible deniability, particularly when it’s not obvious that a site has been blocked (if you don’t see a blockpage, for example).

Knowing whether an internet resource has intentionally been blocked is important. It’s the first step in understanding whether information controls are being implemented in the digital world. This level of transparency is essential because it can support public debate on the legality and ethics around internet censorship, which obviously vary from country to country.      

We no longer need to solely trust our local governments and ISPs to limit internet censorship to that which is “legally proportionate”. Now, we can all measure networks and collect data that shows what is blocked, how, when, and by whom. This is possible with a free software tool, called OONI Probe.

OONI

Back in 2011, some curious hackers started writing free software tests designed to measure the blocking of websites. They figured that everyone could (and should) benefit from these tests. And so they created the architecture for a software system that would enable others to automatically run tests, collect data from all corners of the planet, and openly publish such data. A few months later, the Open Observatory of Network Interference (OONI) was born.

Today, OONI is a community-driven free software project that aims to empower decentralized efforts in increasing transparency of internet censorship around the world. Over the last 5-6 years, tens of thousands of people from around the world have been measuring their networks for censorship through the use of OONI’s software, called OONI Probe. This includes multiple free software tests designed to measure:

​Anyone can run OONI Probe on Android, iOS, macOS, Linux, or on a Raspberry Pi. As soon as you run a test, the network measurement data will automatically be collected and published (unless you opt-out) on OONI Explorer and on the OONI API.

Explore OONI data

We publish all network measurement data that we collect to increase transparency of internet censorship around the world. We also publish data so that you can verify our methodology and the censorship findings. More importantly, we publish data because it can serve as evidence of internet censorship, which is useful when challenging those in power.

OONI Explorer

Image: OONI Explorer.

Millions of network measurements have been collected from more than 200 countries since 2012. All data is automatically published on OONI Explorer everyday. OONI Explorer is, therefore, probably the largest publicly available resource on internet censorship.

By filtering the measurements in OONI Explorer, you can uncover multiple cases of internet censorship worldwide. The blocking of a news site run by the Hazara ethnic minority in Pakistan is provided as an example below.

Image: OONI Explorer.

OONI Explorer also provides the raw network measurement data, which can serve as evidence of the blocking.

Many more examples of internet censorship can be drawn from OONI Explorer, including the blocking of news outlets and human rights sites in Iran, media sites in Egypt, LGBT sites in Indonesia, and the blocking of WhatsApp in Ethiopia and Brazil. OONI data also shows that censorship events worldwide emerge during political events. Examples include the blocking of social media during Uganda’s 2016 general elections and the blocking of opposition sites during Ethiopia’s wave of political protests. Other cases of politically motivated censorship include the blocking of sites covering Malaysia’s 1MDB scandal and, more recently, the blocking of sites related to the Catalan independence referendum in Spain.

OONI API

Image: OONI API.

We also publish data on the OONI API. Dynamically updated everyday, the OONI API enables data scientists, data journalists and researchers to download the json files and analyze all data collected from OONI Probes around the world. We provide documentation on how to query the data. As an example, by running the following command you will view all sites blocked in Italy.

`curl "https://api.ooni.io/api/v1/measurements?probe_cc=IT&confirmed=true"`

Thanks to OONI’s global community, a wealth of data is being collected on a daily basis, providing evidence of information controls worldwide. But this data needs to become more actionable. We encourage researchers to use OONI data to explore questions and contribute to knowledge around how the internet works (or doesn’t work). We encourage human rights advocates to use OONI data as part of their campaigns. We encourage journalists to use OONI data for investigations and evidence-based reporting. And we encourage everyone to help build a world where information controls no longer take place in the dark.

About the author

Maria Xynou manages OONI's partnerships and writes research reports on internet censorship around the world. Previously, Maria worked with other digital rights organizations, such as Tactical Tech and India's Centre for Internet and Society.

Explore OONI here.

Comments