Twitter as a data source: An overview of tools for journalists


Journalists may wish to use data from social media platforms in order to provide greater insight and context to a news story. For example, journalists may wish to examine the contagion of hashtags and whether they are capable of achieving political or social change. Moreover, newsrooms may also wish to tap into social media posts during unfolding crisis events. For example, to find out who tweeted about a crisis event first, and to empirically examine the impact of social media.

Furthermore, Twitter users and accounts such as WikiLeaks may operate outside the constraints of traditional journalism, and therefore it becomes important to have tools and mechanisms in place in order to examine these kinds of influential users. For example, it was found that those who were backing Marine Le Pen on Twitter could have been users who had an affinity to Donald Trump

There remains a number of different methods for analysing social media data. Take text analytics, for example, which can include using sentiment analysis to place bulk social media posts into categories of a particular feeling, such as positive, negative, or neutral. Or machine learning, which can automatically assign social media posts to a number of different topics.

Image credit: Multiple Tweets Plain by mkhmarketing. This work is licensed under a CC BY 2.0 license.

There are other methods such as social network analysis, which examines online communities and the relationships between them. A number of qualitative methodologies also exist, such as content analysis and thematic analysis, which can be used to manually label social media posts. From a journalistic perspective, network analysis may be of importance initially via tools such as NodeXL. This is because it can quickly provide an overview of influential Twitter users alongside a topic overview.

From an industry standpoint, there has been much focus on gaining insight into users’ personalities, through services such as IBM Watson’s Personality Insights service. This uses linguistic analytics to derive intrinsic personality insights, such as emotions like anxiety, self-consciousness, and depression. This information can then be used by marketers to target certain products; for example, anti-anxiety medication to users who are more anxious.

Popularity of Twitter

Twitter is not the most popular platform in terms of monthly active users, being ranked at eighth in the overall list (see figure below), while Facebook and WhatsApp are the top two. However, many of the platforms with the highest number of monthly active users do not make their data available on a similar scale to Twitter.

Image: Number (in millions) of monthly active users across social media platforms. Created using data powered by statista.

It can be argued that there is no other social media platform with an infrastructure like Twitter. Twitter is unique in the sense that it allows any user to be able to follow another user, and it provides almost 100% of its data through APIs. With such a large number of monthly active users, Twitter is likely to remain popular for social media and industry research. From a journalistic perspective, many journalists and citizens across the world access Twitter, which has the potential to drive hashtags to trend, and can consequently be used to give stories a global insight. 

An overview of tools for 2017

Tool OS Get it Platforms*
Audiense Web-based https://buy.audiense.com/trial/new (offers 14 day trial) Twitter
Boston University Twitter Collection and Analysis Toolkit (BU-TCAT) Web-based http://www.bu.edu/com/research/bu-tcat Twitter
Chorus  Windows (Desktop advisable)

http://chorusanalytics.co.uk/chorus/request_download.php (free)

COSMOS Project  Windows; MAC OS X http://socialdatalab.net/software (free) Twitter
DiscoverText Web-based http://discovertext.com (3 day trial) Twitter; Facebook; 
Blogs; Forums; Online news platforms
Echosec Web-based https://www.echosec.net Instagram; Twitter; 
Foursquare; Panoramio;
AIS Shipping; Sina Weibo; Flickr; YouTube; VK
Followthehashtag Web-based http://www.followthehashtag.com Twitter
IBM Bluemix Web-based https://www.ibm.com/cloud-computing/bluemix Twitter
Mozdeh Windows (Desktop advisable) http://mozdeh.wlv.ac.uk/installation.html Twitter
Netlytic Web-based https://netlytic.org Twitter; Facebook; YouTube; 
Instagram; RSS Feed
NodeXL Windows http://nodexl.codeplex.com Twitter; YouTube; Flicker
NVivo Windows; Mac http://www.qsrinternational.com/product Twitter
Pulsar Social Web-based http://www.pulsarplatform.com Twitter; Facebook topic data;
Online blogs
SocioViz Web-based http://socioviz.net Twitter
Social Elephants  Web-based https://socialelephants.com/en/ Twitter; Facebook; Instagram; YouTube
Trendsmap Web-based https://www.trendsmap.com Twitter
Twitonomy Web-based http://www.twitonomy.com Twitter
Twitter Arching Google Spreadsheet (TAGS) Web-based https://tags.hawksey.info Twitter
Visibrain Web-based http://www.visibrain.com Twitter
Webometric Analyst Windows http://lexiurl.wlv.ac.uk Twitter (with image extraction capabilities); YouTube; Flickr;
Mendeley; Other web resources

*It is always best to check with the developers of tools as there may be additional platforms that they can access. Moreover, some tools provide users with the ability of importing data into the applications from external sources.

A number of the tools provided in the table above have been tested and used by me over a number of years, and the vast majority of these chiefly handle data from Twitter. It would be nice to have academic and social listening tools to retrieve data from other social media platforms, such as Facebook, Instagram, and Amazon, and also dark social media platforms such as WhatsApp. However, this may not be possible because these applications are not likely to provide all of their data to developers as Twitter does. Moreover, there may be ethical implications of accessing data from dark social media platforms.

Other applications are available but these require programming knowledge and have not being rigorously tested as part of this post. These include:

Moreover, there are a number of advanced data analysis and statistical applications which can be used to analyse social media data, such as:

These packages should be researched when deciding which application is to be used for a project. I’d also like to mention The Digital Methods Initiatives' list of tools, and Ryerson University’s list of tools from its Social Media Lab.

This post has been modified from Wasim’s post on social media research tools for academic research on the London School of Political Sciences Impact blog. Image: Esther Vargas.