From unstructured data to intelligence

Over the last 15-20 years, significantly improved computing power and advances in algorithms and artificial intelligence (AI) have enabled us to process more data than was previously digestible.

Whether we are talking about social media monitoring platforms, or the many different AI-based analytic approaches, these technology and data science advancements are impressive. They help us to make sense of huge amounts of data quickly and efficiently. But they need more to constitute a useful and real research approach.

Based on our experience, we have observed three broad methodological building blocks of any meaningful social media intelligence programme:

1. Social media intelligence platforms (software platforms designed to enable social listening from different sources of data providing a real-time access to various metrics through the means of interactive dashboards)
2. AI-led advanced analytics (text, picture and video analytics designed to make sense of unstructured data using NLP, machine learning, data mining and statistical analysis etc.)
3. Human-driven insight discovery (individual researcher contribution to finding insights from social media data using analytical frameworks).

On their own, each building block brings something useful to answering critical client questions. Combined, they bring powerful insight. To demonstrate these building blocks in action, we highlight some COVID-19-related examples from our social intelligence platform, Synthesio, to better understand the status and implications of the pandemic.

For example, in the below chart we focus on people’s fears generated by the COVID-19 crisis. Combining a top-down approach with a more consumer-centric one that leverages the power of deep-learning algorithms (a bottom-up approach), we are able to surface additional new fears that we did not expect to see – such as the negative impact of the lockdown period on the education of kids and the emergence of racism and blame game during the crisis.

The area of social intelligence and analytics is exciting and still emerging. In this paper we dispel some of the the common myths around social intelligence and analytics and look at the key factors to consider when evaluating the pros and cons of different research approaches.

Social intelligence is the ability to collect, monitor and analyse available social media data feeds (including social media networks, blogs, forums and comments) to understand what is being said about a topic, brand, organisation or other entity.