Don’t Kill the Analyst Just Yet

Interview on how Ipsos applies text analytics, including the pitfalls to avoid.

The author(s)

  • Jean-Francois Damais Customer Experience, Global
Get in touch

Text Analytics clocks in as the #4 “emerging methods” priority for market researchers in the 2015 GRIT (Greenbook Research Industry Trends) report. Only mobile surveys, online communities and social media analytics poll higher…although of course text analytics is key to social media analytics done right, and it’s also central to #5 priority “Big Data analytics.” GRIT reports text analytics as tied with Big Data analytics for the #1 method “under consideration.”

On the customer-insights side, Temkin Group last year reported, “When we examined the behaviours of the few companies with mature Voice of the Customer (VoC) programmes, we found that they are considerably more likely to use text analytics to tap into unstructured data sources… Temkin Group expects an increase in text analytics usage once companies recognise that these tools enable important capabilities for extracting insights.”

Clearly, the market opportunity is huge, as is the market education need.

Jean-François Damais was a speaker at the 2015 LT-Accelerate conference, which looks at text, sentiment, and social analytics for consumer, market research, and media insights, which took place 23-24 November in Brussels. As part of the conference he gave us this interview.

Seth Grimes: You spoke at the LT-Accelerate conference on text analytics in market research. You wrote in your presentation description that text analytics work at Ipsos has grown 70% each of the last two years. What proportion of projects now involve text sources? What sources, and looking for what information and insights?

Jean-François Damais: Virtually all market research projects involve some analysis of text. In the customer experience space in some of our key markets (i.e US, Canada, UK, France, Germany), I would say that 70-80% of research projects require significant Text Analytics capabilities to extract and report insights from customer verbatims in timely fashion. However, other markets (i.e Eastern Europe, LATAM, MENA, APAC) are lagging behind so the picture is a bit uneven. Generally speaking, text analytics plays a key role in Enterprise Feedback Management, which is about collecting and reporting customer feedback within organisations in real-time to drive action and growing at a very rapid pace. In addition, the use of text analytics to analyse social media user generated content is increasing significantly. But interestingly more and more clients now want to leverage text analytics to integrate learnings across even more data sources to get a 360 view of customers and potential customers. So, on top of survey and social, we quite often analyse internal data held by organisations such as complaints or compliments data, FAQs etc… and bring everything back together to create a more holistic story. Text Analytics can really help when it comes to data integration. Of course, technology is an enabler but will not give you all the answers. We believe that analytical expertise is needed to set up and carry out the analysis in the right way, but also to interpret, validate and contextualise text analytics. This is key.

Seth Grimes: Despite the impressive expansion of text analytics use at Ipsos, my impression is that research suppliers and clients often don’t understand the technology’s capabilities, and the tool providers haven’t done a great job educating them. Does this match your impression, or are you seeing something different?

Jean-François Damais: I would agree with you on the whole. There are still a lot of misconceptions and half knowledge in the industry. I do feel that text analytics providers would benefit from being more transparent about the benefits and limitations of their software, and how they can be applied to meet a business need. Currently it feels that everyone is ready to make a lot of promises that are difficult to live up to and I sometimes feel that this is counter productive. I am referring to the focus on accuracy levels, level of quality across languages, level of human input needed, how unique or better or one size fits all one’s technology is compared to the rest of the market etc. In 2014, we conducted a comprehensive review of many of the text analytics tools currently available and identified pros and cons for each. Although each of the tools presented us with different strengths, challenges and functionalities, we gained the following learnings:

  • There is no perfect technology Knowing the strengths and weaknesses of the technology used is key to getting valuable results.
  • There is no miracle “press a button” type solution Even the best tools need some human intervention and analytical expertise.
  • There is no “one size fits all” tool Depending on the type of data or requirements some tools and technologies might be better suited than others.

My colleague Fiona Moss and I have recently written a POV on how to successfully deploy Text Analytics. The full paper can be found on the Ipsos website. The benefits of text analytics technology are huge and I do agree that focus should be put on educating users and potential users to make the most out of it.

Seth Grimes: How did you personally get started with text analytics, and what advice can you offer researchers who are starting with the technology now?

Jean-François Damais: I got started in 2009 when Ipsos Loyalty launched text analytics services to its clients. At the time, this capability was very much a niche offering and seen by most organisations as an added value and nice to have. But things have gone a long way since then and text analytics capabilities now support some of our biggest client engagements and are now a key tool in our toolkit.
Here is what I would say to any keen researcher (or client):

  • Know your purpose 
  • Manage your organisation’s expectations 
  • Place the analyst at the heart of the process 
  • Choose the right text analytics tool(s) given your objective 
  • Learn the strengths and weaknesses of the tool(s) you are using 
  • Don’t give up!

Seth Grimes: Where do you see the greatest opportunities and the biggest challenges, when it comes to text sources and the information they capture, and for that matter, with the range of structured and unstructured sources?

Jean-François Damais: To some extent, what applies to text analytics applies to big data more generally. There has been a significant increase over the last few years in the volume and variety of sources of unstructured data, including feedback from customers, potential customers, employees, members of the public and information systems. There is a huge value that quite often lies buried in this data. So the opportunity comes from the ability to extract actionable insights and intelligence. While the potential is huge, there are a number of pitfalls organisations need to avoid. One of the most dangerous is the belief that technology in itself, regardless how state-of-the-art, is enough to derive good and actionable insights.

Seth Grimes: Quoting an Ipsos case study you wrote: “Even when data has been matched to a suitable objective, analysis can be a daunting task.” What key best practices do you apply for data selection and insights extraction, from social sources in particular?

Jean-François Damais: The analysis of social media presents itself with significant challenges that go well beyond text analytics. The traditional approach to social media monitoring has been to trawl for everything – the temptation to do so is huge, as we now have access to web trawling technology which can span the web and return a wealth of data at the “press of a button”. Unfortunately in most cases this leads to analysis paralysis as the data collected is huge and mostly irrelevant, with a lot of redundancies. This type of information overkill with no insights is discouraging, time consuming and costly. We try to structure our “social intelligence” offer around a few principles designed to address some of these challenges. The first thing is to search for specifics. Mining web data or big data more generally speaking is very different from analysing structured research data coming from structured questionnaires. You just cannot analyse everything, or cross tabulate everything by everything. The vast amount and diverse nature of such data means that we need a different approach and knowing what you are looking for is key. If you want specific answers, you need specific questions. It is also about adapting and evolving. It does take time to test and refine the set up in order to obtain valuable insights and answers. Companies should not underestimate the amount of time it takes to design, analyse and report social media insights.

Seth Grimes: What’s the proper balance between software delivered analysis and human judgment, when it comes to study design, data collection, data analysis, and decision making? Are there general rules or do you determine the best approach on a study-by-study basis?

Jean-François Damais: As mentioned above, we firmly believe that analytical expertise is needed to make the most out of text analytics software. However, the amount of human intervention varies according to what type of analysis is required. If it is just about exploring and counting key concepts/patterns in the data then minimal intervention is needed. If it is about linking different data sources and interpreting insights then a significant human element is needed. Technology is very important, but it is a means to an end. It is the knowledge of the data, how to manipulate and interpret the results and how to tailor these to the individual business questions that leads to truly actionable results. This places the analyst at the heart of the process in most of the projects that we run for clients.
Test Analytics

Seth Grimes: Finally, I’ve been working in sentiment analysis and emerging emotion-focused techniques for quite some time, but the market remains somewhat sceptical. What’s your own appraisal of sentiment/emotion technologies, in general or for specific problems?

Jean-François Damais: No technology is perfect but we can make it extremely useful by knowing how to apply it. Again, I think the realisation comes with experience. We work with clients who tell us that text analytics have brought in significant and tangible benefits – both in terms of time/cost savings and also additional insights and integration. My view is that as a whole the industry should focus a bit more on communicating these tangible benefits and a little bit less on who has the best sentiment engine and the highest level of accuracy.

This interview was originally conducted by Seth Grimes, Alta Plana Corporation.

The author(s)

  • Jean-Francois Damais Customer Experience, Global

New Services