Conversations with AI Part II: Unveiling AI quality in qualitative workstreams
The potential applications of AI are no longer abstract ideas but tangible possibilities that can transcribe audio and video files, generate images and videos, and even write code. While there is excitement about the productivity and creativity that AI can bring, there is also a sense of caution due to the biases, errors, and hallucinations that can occur in AI systems.
To navigate this sea of change, we emphasizes the importance of cautious and thoughtful use of AI technologies. A strategic approach to research-on-research allows us to understand the speed and intensity of these changes and harness the power of AI in market research. By studying the impact of AI on qualitative research, we can embrace these changes and develop strategies to leverage this technology to our advantage.
In Conversations with AI Part II conducted a series of tests to evaluate the utility and risks associated with AI outputs. They assessed AI suppliers in transcription, translation, and sentiment analysis using qualitative research platforms.
The results showed variations in the quality of AI transcriptions, with machine transcriptions achieving higher accuracy in widely used languages like English. The accuracy of AI translations depended on the use of human transcriptions as starting points.
In terms of sentiment analysis, AI models showed varying levels of accuracy, with some models performing as well as human-coded sentiment analysis. However, there is room for improvement in sentiment analysis at the sentence level before AI can operate independently.
In thematic analysis, generative AI provided good topline summaries but struggled to elevate insights and draw business implications compared to human researchers. Human intervention is still necessary to bridge the gaps in domain expertise, bias identification, and hallucination prevention.
While AI presents opportunities for innovation and growth, businesses need to carefully assess and evaluate AI tools to unlock their maximum value while mitigating potential risks.
Want to learn how generative AI and qualitative research will benefit each other?
In Conversations with AI, we explore the need to align with experts in the art of the question that can get the most out of AI solutions, including quality and accuracy, while defending data privacy.
