Designing a 'Smarter' Mystery Shopping Program

Countless research studies show that brands which put the customer first and deliver a superior experience relative to their competition, achieve greater customer loyalty, higher profits and better returns for shareholders. Yet, delivering a consistently superior experience in today’s complex world has never been more difficult. Mystery shopping as a research methodology exists to solve this problem. 

However, most enterprise mystery shopping programs are poorly designed, poorly executed, and deliver minimal business impact. In this paper, we discuss how to design a “smarter” mystery shopping program that allows organizations to achieve an advantage over their competition. By imbedding intelligence and technology into mystery shopping programs, Ipsos creates programs that are better designed, better executed and have a greater impact on your businesses’ bottom line.

Want to learn more about this approach? Download and read this point of view or access the live webinar by clicking here.

Customer experience has always mattered

Countless research studies show that brands which put the customer first and deliver a superior experience relative to their competition achieve greater customer loyalty, higher profits and better returns for shareholders.

Brands with the best customer-first approaches are consistently ensuring standards are met every time and at every touchpoint. In recent years, a convergence of factors—heightened consumer expectations, a complex multichannel retailing environment, wide-spread adoption of online and mobile brand interactions and the power that social media has given individual consumers—has made delivering an exceptional customer experience increasingly difficult.

Mystery shopping exists to alleviate these issues and concerns for organizations.

What is Mystery Shopping?

Mystery shopping is an observational research methodology that ensures brands consistently deliver on the promises they make to customers.

It leverages trained “shoppers” to go through product or service interactions across your enterprise and report back with a detailed and objective account of their experience. Mystery shopping helps companies understand what their average customer is experiencing, what staff behaviors should be acknowledged or rewarded, and what parts of the sales or customer service journey can be improved.

The results are aggregated and used to identify and remedy systemic breakdowns of the desired and intended customer experience. When used effectively, mystery shopping research can be a powerful performance management tool that creates awareness around important standards/initiatives and drives organizational behavior change— all in the spirit of better serving the customer.

However, in our experience of designing and implementing enterprise-wide mystery shopping programs, we often find that organizations are misusing the methodology. This results in a program that does not deliver tangible value to the business or meet the needs of program stakeholders. So, while it’s important to understand what mystery shopping “is” and how it can benefit your organization, it is also important to understand what mystery shopping is not:

It is not a way to “spy” on employees to catch them in the act of doing wrong.

It is not a replacement for voice of the customer measurement or other forms of direct customer feedback. (Nor is Voice of the Customer (VOC) a replacement for mystery shopping).

Mystery shopping is not limited to measuring compliance to brand standards in brick-and- mortar locations only. Today, a mystery shopping program can be deployed across any customer channel: in-store, telephone (retail location or contact center), and across online and mobile touchpoints (website, in-app, email, etc.).

“When used effectively, mystery shopping research can be a powerful performance management tool that creates awareness around important standards/initiatives and drives organizational behavior change”.

Making Mystery Shopping Smarter, by Design

Beyond these methodological considerations, when we diagnose what has gone wrong with many organizations’ existing mystery shopping programs, we regularly find the issues are rooted in poor program design. In addition to deploying one or more of the tactics outlined above, we also find that organizations routinely:

Ask the wrong questions, too many questions, or questions that are not important to their customers.

Experience quality issues originating from shoppers being recognized, which result in mystery shop inquiries that are time-consuming and divert the focus away from making organizational improvements.

Don’t derive real impact or tangible business value from their programs—often because they focus on the wrong metrics or don’t properly analyze results and disseminate program findings.

At Ipsos, we’ve been looking at what needs to be in place to deliver better design, better execution and better impact across our mystery shopping

programs. Regardless of your organization’s experi- ence with mystery shopping, we have developed specific steps and a proven blueprint that should be followed to build a successful program and increase its return on investment. Here are seven simple steps to make your mystery shopping program smarter, by design:

STEP 1

Start with Strategy

Like many research programs, the absence of clear mystery shopping program objectives that look to solve a specific business question often result in a diluted program that does not deliver clear value.

Reference your own organizational strategy for clear direction on your customer-facing initiatives and how you would like to leverage mystery shopping as a tool to enhance the customer experience.

From there, define clear objectives that align with this strategy and ensure that these objectives are addressed by the mystery shopping program. Here are a few thought starters when designing your enterprise’s mystery shopping strategy:

Measuring adherence to specific brand standards or brand promises made to your customers.

Better understanding your own or your competitor’s omnichannel and digitally convergent experience.

Mystery shopping your competition to discover best practices, reasons for customer churn/ defection, pricing strategies, and sales forecasting.

After a new product launch, measuring training effectiveness and/or point of sale signage compliance.

STEP 2

Design with Your Customer in Mind

Once you have developed your strategy, the next step is to design with your customer in mind. One common mistake made by organizations is devel- oping a mystery shopping program that is based on what is important to them, not their customer.

For example, while it might be important to you that your employees always wear a nametag, this may or may not be important to your customers.

It might be more important that associates are friendly, helpful and knowledgeable. Furthermore, a clean restroom or short wait time during checkout might be more important to them than whether the associate attempted to upsell a more expensive product.

There are various techniques we use to ensure customer feedback is properly embedded in your mystery shopping programs. This includes using customer journey maps, conducting driver analysis with your customer satisfaction data, and leveraging focus groups and pilots to inform the design of your program. Give thought to key “moments of truth” where you can either create pain or delight from the customer’s point of view. What are the customer’s expectations of you regarding those moments? Of your competition? What language does the customer use to talk about those moments?

It can also go well beyond these primary research methods, such as linking sales data to channels or individual locations and/or employee data. The program may be tweaked over time to account for updated training techniques and promotional activities.

But don’t skip this important step. When organizations fail to view their program through the customer’s eyes, it often results in adverse behaviors such as “score chasing” vs. driving outcomes such as reducing customer churn, improving satisfaction/ likelihood to recommend or driving new customer acquisition and retention.

“While it might be important to you that your employees always wear a nametag, this may or may not be important to your customers. It might be more important that associates are friendly, helpful and knowledgeable.

Furthermore, a clean restroom or short wait time during checkout might be more important to them than whether the associate attempted to upsell them to a more expensive product.”

STEP 3

Define Shopper and Shop Requirements

The next best practice is to define the shopper and shop requirements for the program. We often see that companies are not deliberate or strategic in this step, and it often leads to biased program results, unnecessary costs, or ill-informed insights.

Defining the shopper requirements starts with identifying your target customer profiles (ideally through customer segmentation data). You will want to think of both demographic and psychographic profiles of your customer base. Knowing that there will likely be multiple segments, you must determine which will be served by your mystery shopping program. You may decide to look at a nationally representative sample, or you may decide that there is a need to focus on a specific or underserved segment. This will then lead to the specific “personas” that shoppers will either need to fit or the scenarios they will need to role play (which we elaborate upon in Step 5).

After defining who the shopper will be, you must decide the “rules” or requirements for the mystery shop itself. You will need to think about specifics such as the stores you would like shoppers to visit (or contact centers you’d like them to place calls to), how many times you want to visit that store, the daypart or day of week to conduct shops, how much time the shopper will need to spend in the store/on the phone, whether they will need to make a purchase, etc.

STEP 4

Design a Sound Survey Instrument

Once you have an outline of what the shop and shopper requirements are, the next step is to design a sound survey instrument that best suits your strategy. This is the survey the mystery shopper will fill out after the interaction with your brand. The survey itself will capture what you are looking to measure and should be built in an organized and systematic manner. In our experience, poor survey design is the single biggest factor contributing to poor program design and quality issues.

Regardless of strategy, the survey must be kept largely objective. Avoid subjective, perception- based Likert-scale questions within the survey and instead focus on discrete, binary questions.

In a covert in-person setting, the survey should be as short as possible (less than 30 questions is ideal) to avoid issues with shopper recall and should adhere to the objectives you developed at the outset of the program.

In revealed audits, the survey can be lengthy and detailed but should be organized in a cohesive way that matches the expected visit flow or agenda of the auditor.

For phone or digital mystery shops, take advantage of the shopper’s ability to take notes and ask for more detail on the survey form.

Finally, design the survey scoring or weighting scheme with a data-driven approach, by assigning points to each question that are proportional and based on elements identified in Step 2 (for example through a driver analysis).

STEP 5

Create Realistic Shopper Scenarios and Thorough Briefing Materials

You should also design the mystery shop scenarios and briefing materials in parallel with the survey instrument. The mystery shop scenario differs from the survey itself as it is the script, situation or “persona” that you would like the mystery shopper to role play while in the store (or on the phone, online, etc. depending on your program). Briefing materials are leveraged to train the shoppers on how

the interaction should play out. This is critical to ensuring consistency, quality, and overall success in your program, no matter what size or scale you are looking to deploy.

For years, shopper briefing materials have been deployed via written instructions that outline each step of the process. While this can be effective, designing a smarter program means leveraging technology across each phase of the program—including shopper briefing. Deploying video briefing materials via short, engaging multimedia clips is an effective way to illustrate what you want to accomplish during the mystery shops, while also demonstrating to the shopper where they may encounter pivot points throughout the shop scenario.

You should also consider developing an online shopper certification to ensure they understand and have retained the materials. This allows for an added level of confidence and credibility throughout the organization on who is conducting the mystery shops and the results that come back.

Another best practice includes conducting calibration shops (several test shops conducted by program stakeholders) to ensure your intended mystery shop design is feasible and capturing the type of information you intend.

STEP 6

Formulate a Smart and Strategic Sampling Plan

The sixth step is to formulate a smart and strategic sampling plan. There is no clear-cut answer to how many shops you should conduct, the number of shoppers or scenarios you should deploy, and/or how frequently you should be visiting a specific location.

The “every location, every month” method of mystery shopping is obsolete and will often result in spending more on mystery shopping than is realistically required.

However, if you tie your sampling plan back to your strategy and objectives, you will be in a better position to answer these questions. Whenever available and applicable, there are several areas worth considering:

Leverage sales data to potentially shop at the highest (or lowest) grossing locations as they may warrant more regular measurement.

Leverage customer experience data to potentially visit lower-performing locations more frequently.

If specific geographies, regions or districts within your organization receive a disproportionate number of customer complaints, it may make sense to mystery shop them more often.

We often help our customer “get more, for less” by deploying many of the tactics outlined above to develop a “smart sampling” approach. This involves combining program objectives, business or customer data (such as sales or customer satisfaction data) with statistical methods to determine the volume and frequency of mystery shops. Over time, your sample plan can (and should) be adjusted to meet the changing needs of your organization and your customer. This will lead to stronger economies of scale and will optimize the program spend for your organization.

Designing a smart sampling plan also involves determining the right channels to shop (brick and mortar, contact center, online and mobile) as well as developing the right mix of shop type and complexity. We have helped countless customers optimize their budget by including MicroShops (short, tactical, crowdsourced mystery shops) into their sample plan that can often be executed faster and cheaper than traditional mystery shops.

“The ‘every location, every month’ method of mystery shopping is obsolete and will often result in spending more on mystery shopping than is realistically required.”

STEP 7

Design Your Analytical Plan

The last step in designing an optimal mystery shop program is to design your analytical plan. Too often, companies rush to field data without proper thought to insight creation, who will access results of the program, where and how. This often leads to a predicament where results aren’t actionable, or don’t return the answers you need to address your specific business question.

First, you need to develop a full analytical plan before launching and executing the mystery shops themselves. An analytical plan is simply an outline of the reporting deliverable you expect to receive upon completion of the study. The best analytical

plans often work backwards, considering the business questions that need to be answered, who the stake- holders are, and how you plan to get the information to them. An underdeveloped analytical plan, or one that does not reach across all levels of the organi- zation, puts you at risk of having too much or unfocused data and falling short on actionability.

Summary

A smarter designed and optimized mystery shopping program will increase the long-term impact on your enterprise. By measuring the proper customer journey and the right channel(s), you can drive improvements throughout the business. Leveraging the tips above will help improve the design and operational execution of your mystery shopping program, drive more impactful results and help create true behavior change across your organization.

Ipsos Channel Performance is one of the largest mystery shopping providers in the world. We complete over a million walk-in, phone, and online shops every year across all industry segments and organization sizes. Our clients benefit from our highly experienced team, global reach, local presence, research expertise, and innovative approach to designing and managing mystery shop programs.

We partner with our clients to size channel opportu- nities, measure brand promises and execution, and drive compliance and sales conversions to improve performance across all sales and service channels— physical, contact center, and digital—and help deliver profitable growth. Using our expert research services—Channel Sizing, Customer Counting, Mystery Shopping, and Execution Measurement— we deliver business impact via analytics, insights and advisory services.