The Complete Picture: To Mystery Shop or Conduct Satisfaction Surveys?

One of the recurring themes I’ve heard from customer experience professionals is their company is not Mystery Shopping their front line employees because they have a Customer Satisfaction (CSat) program in place.  It is rather surprising how many believe that customer surveys and mystery shopping are interchangeable methodologies.  Although monitoring the customer experience through a CSat program is crucial to a company’s success, many don’t realize CSat only provides a partial picture of their customer experience.   Quite often this incomplete view exposes their company to reduced revenue or possibly, increased regulatory risk. Although these two methodologies do overlap in monitoring interpersonal skills, mystery shopping goes well beyond, providing information on actual conditions and adherence to sales process.   For many industries, including financial, technology, retail and travel, an integrated CSat and mystery shopping program is crucial to truly monitoring the customer experience.

As an illustration, following are the actual results of mystery shops conducted with branches from three different banks (their names have been changed to protect the guilty).  Keep in mind that all three calls were handled by friendly, polite and attentive branch representatives who would have received high ratings on a CSat survey.  For context, Bank 1 was a market laggard, Bank 2 had been the market leader, and Bank 3 was actively carving market share from the other two.

Bank 1:  Thank you for calling Bank 1, how can I help you?

Me:  What are your CD rates?

Bank 1:  We’ve got a 12 month CD with a .5% rate, a 24 month with a .7% rate, and a 36 month with a 1.0% rate.

Me:  Thank you very much.

Bank 1:  Thank you for calling Bank 1.


Bank 2:  Thank you for calling Bank 2, how can I help you?

Me:  What are your CD rates?

Bank 2:  We’ve got a special right now, an 18 month CD with a .6% rate.

Me: Thank you very much

Bank 2 : Thank you for calling.


Bank 3:  Thank you for calling Bank 3, how can I help you?

Me:  What are your CD rates?

Bank 3:  We have a lot of alternatives – How much are you planning to save and how long would you like to hold it?

Me:  I’ve got about $10,000 from an inheritance, I really don’t know how long because I don’t plan on spending it anytime soon.

Bank 3:  We have a number of ways you can save your money, but based in what you’ve told me, I think it would be best if you can come down and meet with our financial professional, Doris.  She can make sure that your money is put into an account that will be best for you.  I’d like to set a time when you can see her, when can you come down to the branch?


One can quickly see how Bank 3 was offering a superior sales culture than the other two banks, but how would a Bank 3 manager monitor execution of this strategy in over 1,000 branches?

This is where mystery shopping separates itself form CSat.  Had the call to Bank 1 been measured on an appropriate mystery shopping questionnaire, it would be obvious to a region manager that proper sales procedures were not being followed.  Often non-compliance to company procedures is not hit or miss; with sufficient data points patterns or practices emerge, empowering managers to correct the issue.

So, when do you need to augment your CSat monitoring with mystery shopping?  Here are some questions to ask yourself:

  • Does the success of my product or service depend on sales people’s ability to ask questions, and then synthesize recommendations to potential customers?  What will happen if they aren’t?
  • Is consistency of customer experience crucial to our company’s success?
  • Will my company grow if customers are serviced by polite order takers?
  • What revenue consequences will my company face if sales processes aren’t being followed?
  • What regulatory risks am I exposed to by the actions or words of my front line sales people?
  • Are there signage and POS displays that must be in place and working?

If answering Yes to any of the above questions, CSat alone will likely provide an incomplete picture of your customer experience.

Mystery shopping enables management to compare store performance on a level playing field, testing each with a similar customer inquiry.  Sales recommendations can be measured through training mystery shoppers to all use consistent responses during needs assessment; any deviation in product/service recommendation by store staff immediately raises an alert.  Physical plant can be monitored through images collected by smartphone enabled shoppers.

Companies have benefited by merging their CSat and Mystery Shopping data streams into an index.  Finding correlations between the two data streams increases revenues though early pinpointing issues, whether they are training, personnel or facility related.  Some have even expanded their feedback loop by merging other data stream such as store geography, profitability, or revenue.

In short, take a review of your customer experience metrics – You may find that you are only getting an incomplete picture.