2/10/2017

Assess broadband performance with M-Lab’s public data

 

Over 400 million Internet users in Europe want fast and reliable broadband access for their day to day work needs or for their commercial, social, and cultural life. Internet service providers (ISPs) offer special products for broadband internet that promise ultrafast speed. However, consumers often have few resources to assess whether promises about performance are being fulfilled or how their internet experience is shaped by their ISP.

Recently these promises were challenged in the United Kingdom. In July 2017, the BBC published a news report that claimed that customers of Virgin Media at times only received 3% of the internet speed that was promised to them. For many customers such a loss of performance has serious, real world implications. At best, this means that videos do not load properly, but in the worst cases, it means that services stop working entirely, and websites fail to load. Usually, we only hear about these serious issues when customers complain in large numbers. The problem is that there is not enough public data to monitor broadband performance of users across Europe. One initiative trying to fill this gap is M-Lab.

About M-Lab

Measurement Lab (M-Lab) is a consortium of research, industry and public interest partners dedicated to providing an open and verifiable measurement platform for global network performance. With servers around the world and a suite of open source network tests, M-Lab helps people get an accurate picture of their internet experience. These tests describe important qualities of internet access, such as speed, reliability, and delay.

M-Lab hosts one of the largest open internet performance databases on the planet, which consists of multiple petabytes of data stretching back to 2009. Everyday, over a half a million tests are run by users via websites and applications, or with configured tools, for example software or hardware devices that conduct tests regularly. With this user-contributed data anyone can analyze internet performance for different segments of the network, population, or country. You can also run a test yourself on the M-Lab site, compare your ISP’s performance over time, or compare your results with another city.

Image: How M-Lab performance test works.

A case study to analyze broadband performance

There are many reasons why broadband performance, and hence the user experience, is limited, and it is not always easy to say what exactly causes Netflix or Skype to perform poorly. In general, however, congestion on the ISP networks or within the home network is the phenomenon that limits user experience the most. Congestion normally happens when too much data is requested by too many users at the same time. This may lead to traffic being queued or discarded at one point on the path, bottlenecks where supply cannot meet demand. The degradation of performance due to congestion in a network will be experienced as lower download speeds, among other metrics such as higher round trip times or packet retransmission rates (more about these values). Typically, performance fluctuations due to congestion can be seen across different networks. Some applications such as voice over IP or online gaming, however, are very sensitive to these fluctuations.

ISPs control their network management practices, and make the determinations of how to allocate capacity to meet user demand. By analyzing performance data, you get insights into where and when congestion happens, and how this might affect the user experience. Here are two examples.

In the BBC news report, it was claimed that, in some areas of the UK, Virgin Media’s network was over-utilized and users’ traffic could not be delivered quickly and smoothly anymore. If a lot of Virgin Media customers have faced problems with internet services and applications, or more generally a poor user experience, this should be visible in the performance data. With M-Lab, you can explore the performance data and see whether this degradation was measured.

Graph A: From M-Lab data visualization - Median Download Speed over the course of a day from Virgin Media customers in Europe during one year from May 2016 to 2017. The Center for Applied Internet Data Analysis offers a database for names and numbers of ISPs, in case labels are unclear.

This graph was produced with M-Lab’s visualization system. It shows the median download speed that Virgin Media customers experienced during one year from May 2016 to 2017. Over the course of a day the actual speed fluctuates in part depending on the relation between capacity and demands from users. That means, if the provisioned capacity of a path is not enough to meet the current demand, the increased congestion in the network will lead to lower speed for most users. While during early morning the download speed approaches 30 Megabits per second (Mbps), the performance in the evening approached 14 Mbps. This roughly 50% difference of actual speed over the course of a day explains why many users had bad experiences using the internet in the evening during peak times. This is when most home users turn on their computer to watch videos, stream music or simply ‘surf’ the web, resulting in increased demand on Virgin Media. As a reference point, Netflix suggests that to watch video at the best quality settings you will need a minimum of 25 Mbps.

Graph B: From M-Lab data visualization - Median Download Speed over the course of a day from Vodafone customers in Europe during one year from May 2016 to 2017. The Center for Applied Internet Data Analysis offers a database for names and numbers of ISPs, in case labels are unclear.

In contrast with the Virgin Media graph, other networks demonstrate more steady performance over the same time period. The above graph shows the median download speed that Vodafone customers in Spain experienced in the same time period as the first graph. Only small performance fluctuations are visible, which indicates that the capacity management was better adjusted to customer demand. These two cases are examples of how broadband performance can be analyzed with open data. The same principle of measurement can be applied to other ISPs as well.

There are many more data driven stories

The Virgin Media case is not unique. There are other ISP networks where issues of congestion can be seen. We compared Virgin Media with some other ISPs from different European countries. While ISPs in France (Free) and Belgium (Belgacom) relate to the Virgin Media patterns, other ISPs from Italy (Fastweb), Spain (Vodafone), and Greece (Forthnet) show no signs of congestion.

A poor broadband performance causes real world problems for many European internet users. Open data, contributed by people like you and published via M-Lab, plays a critical role in providing evidence about important qualities of internet access. M-Lab has documented the sometimes severe effects of under-provisioned capacity around the world, most prominently in interconnection disputes in the United States, however, many stories remain untold. If you want to help people to better understand internet performance and to finally help everyone to get the best internet, take a dive into the open performance data on M-Lab’s platform.

Explore M-Lab here.

Image: Brian Holland.

Comments