As shown in Lesson 1, a CX survey can be a very powerful research tool BUT it is imperative that you not only know when a survey is the right choice, you must also understand the issues inherent in every survey.
As a matter of transparency, it’s important for us to disclose that SupportEXP offers both survey and mystery shopping research options. Our goal is to help our clients choose the RIGHT methodology for their desired outcome, NOT push them into one or the other.
To illustrate the challenges the survey methodology presents, let’s look at one of the most common objectives financial institutions share: Improving Frontline Execution. While it will become readily apparent that a survey isn’t the best approach for achieving that objective, keep in mind that these issues impact ALL surveying initiatives, even those where a survey is actually the perfect research method.
Surveys Can Do a Lot of Things…
But Here Are 10 Reasons Why Surveys Can’t Drive Deep Change in Frontline Execution
1) The Intention Issue: It’s essential to clarify from the beginning the objective of your research initiative. Driving deep change in frontline execution is an objective that demands nothing less than behavioral transformation. If this is not clear from the outset, you will inevitably fail to capture, much less make use of, data that effects behavioral level change. You’ll just have more data, more interesting information – that nobody wants to really own.
- Getting the RIGHT research instrument that firmly aligns to your reason for doing research is the first reality to face. If the reality you need to impact is your frontline execution, you need a precision tool that can assess employees at the behavioral level. Such precise measurement is the only method for identifying and improving the service and sales skills that create an effective (and might I add, elite) frontline execution team. The only measurement instrument calibrated to such precision is the CX mystery shopping instrument.
2) The Recall Issue: A survey instrument is designed to capture, after the transaction, general insight from those who were not expecting to give customer experience feedback to you. They had no idea that they should be remembering exact details of the experience so it can be improved. So you won’t get information on particular behaviors that were or were not demonstrated: how that employee demonstrated empathy (which, by the way, is the #1 point of failure of frontline teams when it comes to service that actually results in building relationships). You won’t get details about why your sales process ended in failure to deepen the customer relationship (which is the #1 point of failure of frontline teams when it comes to sales – not knowing HOW to advance in genuine, relationship-building ways). You just can’t get that level of detail from a survey instrument.
- If your objective is building an elite frontline execution team, there is no substitute for behavioral level analytics. Your data-gathering method MUST reliably capture the behaviors the employee demonstrates during an actual experience with a customer. A mystery shopping approach informs the customer in advance of the specific behaviors that go into the financial institution’s service and sales culture, so they know what they should be observing and what they should be reporting in their feedback.
3) The Limbic Issue: I like to call this the “Limbic Light Up.” If the experience was so offensively rude or difficult that it hit the brain’s epicenter of emotion – you’ll likely hear from THAT customer. (I say likely because there are those you will NOT hear from – even though you need to. Those are the ones who have already given up on you or who might fear a bit of retaliation, whether real or imagined from you if they actually told you the truth and nothing but the truth.) You will also hear from those customers who are ecstatic about their service. Either way, it’s colored by emotion.
- If the goal is improved frontline execution, then it is objective feedback you need from customers on a mission with you, reporting actual, observable and experienced behaviors with the intention of helping you build a service-conscious culture. Impressions, perceptions and reflexive responses gathered from survey feedback still leave you asking, “What needs to actually change moment-by-moment to get to predictable outcomes?” This is a critical reason to think beyond surveying.
4) The Vigilante Issue: Social media, Yelp, etc., have shifted the power to influence the perception of your brand to the consumer. One offended consumer can accomplish much! Online or mobile vigilante justice is not only easy to carry out in response to offenses in service delivery, it can be conducted with anonymity: “Why respond to a survey when we can get our justice in a big way that gives some sense of resolution?”
- A mystery shopping research method reframes the request for feedback as being constructive, purposeful, cooperative…and not adversarial.
5) The Depth Issue: When I want something fixed, I want to know not only who is going to fix it, but how it’s going to get fixed and what we are going to do to lock in change. A survey is valuable in providing insight into general areas where there is friction, but it cannot solve the problems of who, how, or what.
- Transforming frontline execution requires knowing who is going to fix it, how it’s going to get fixed and what you are going to do to lock in change. A well-implemented CX mystery shopping solution does all three, and you need all three to fuel growth through actually possessing the right skills to build relationships through your frontline team.
6) The Distortion Issue: What’s in a number? That’s an important question when you’re determining strategy and success (or lack of it). While we love the broad nature of survey collection, we don’t love that you can’t control how customers determine whether you’re a 10 or a 2, or some number in between. And then you have to ask the all-important question, “How does this measure up based on a standard of service to actually differentiate?”
- The tool, the scale, and the data analytics have to be about more than, “What did you feel about us?” Instead, you have to get to this in your tool, your scale and your data analytics: “Here’s our exact performance experience standard, now how did we do against it?”
7) The Fatigue Issue: You’re not the only one asking for customer feedback. Consumers today are inundated with requests for their opinion. If you haven’t partnered with them intentionally (including training, education, and some sort of remuneration for their evaluation of your service) then there’s not a lot of time the average consumer is going to spend to give you that feedback (unless of course you’ve hit their limbic system). A reluctant or hurried respondent is not going to think too much about their answers — and you need more than smiley face responses to steer strategy.
- The risk of fatigue is eliminated by partnering with customer-researchers trained to evaluate and give objective responses, who know how frequently they’re going to be asked to evaluate and submit their feedback. The sense of contributing to a shared enterprise is a motivating factor for your customers – you know, that “People Helping People” thing?
8) The Gaming Issue: Sadly, for you, when employees know they are going to be evaluated for their customer service, some inevitably come up with ways to influence the system in their favor. They may be especially motivated if their scores are tied to job or pay status. They try to coax certain responses from customers, or dissuade them from responding altogether. Intentionally (or even unintentionally) rigging the system to engineer a higher score only creates a false sense of security, and undermines the purpose of getting a true picture of your customer base.
- Effective feedback systems guard against gaming by employees. One way they do this is by educating/training the evaluators in the objectivity of experience feedback. This is an exclusive characteristic of a mystery shopping program.
9) The Guilt Issue: It’s that darn guilt thing. Not knowing how your survey data will be used causes some customers to worry that they would be responsible for, or contributing to, repercussions an employee might face for less than perfect service. While it may speak well of human nature, this is not a time to NOT know. You need honesty when an unsatisfactory experience occurs.
- Using objective research conducted by customers committed to giving honest answers for the organization’s benefit is a major safeguard against skewing of feedback by guilt.
10) The Usability Issue: With surveys, you’ll have many data points that can tell you a lot about your customer base, but the feedback is not going to be coachable at the behavioral level. The tradeoff for volume, unfortunately, is an inherent lack of precision.
- An effective methodology for strengthening frontline execution will provide targeted feedback that Joe, Jane, or Sally can use to build skills that will improve the very next customer experience they deliver. A mystery shop delivers CX intel that can have an immediate impact on individual performance.
All of this means that, if strengthening frontline execution is your objective, a survey is probably not the measurement instrument for the job. Surveys, while great for certain purposes – like giving your customers a channel for expressing themselves– are not intended to measure and address the frontline behaviors that build relationships.
Which means that, if you want to truly transform your CX culture, you need to address the behaviors demonstrated every day on the front line. Not just any customer feedback instrument is going to enable you to do that. A CX mystery shopping method of data gathering speaks directly to these behaviors and provides intel – exact data analytics – for improving the skills essential to effective frontline execution.
This lesson has given a lot of reasons why surveys are inherently unsuitable for one of your most significant competitive advantages, strong frontline execution. Not to fear – in later lessons, we will explore with brutal honesty why mystery shopping is not appropriate for every research initiative, and what distinguishes a good mystery shopping program from a more suspect one.
Best-in-class organizations recognize that it’s the strength of their customer relationships that differentiates them. Building relationships so that you keep your customers so close to you that they don’t even look at the competition starts with frontline execution, and it has to happen one unique customer experience at a time. True CX transformation demands nothing less.
Up Next – Lesson for Leaders 3: What NPS Metrics Lack
Net Promoter, Net Promoter System, Net Promoter Score, NPS and the NPS-related emoticons are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.
