Dumb Ass Stupid Management: Survey This
Have you seen the proliferation of surveys lately? I get surveys about my happiness or unhappiness from pharmacies, auto dealers, electronics outlets, hobby shops, restaurants, reservations sites, credit card companies, hotels, and so forth. Most of the surveys are very poorly done. That’s because the people designing them do so for technology ease, not customer ease. It’s because the intent is to reduce survey responses to numbers to be compared against “standards” or “norms.” It’s because HR people swallow them up and executives never actually see them.
Executives seem to believe that surveys are useful when, in fact, they are amazingly self-selective: Very happy and very unhappy people respond, but not the masses in the middle. It’s like the irritating notice on the phone that “this call may be recorded for quality purposes.” If the call is recorded, it’s for legal purposes, because the quality in those companies never improves.
If executives want to learn about the performance of their businesses, they should shop their own companies, but quietly. I once sat across the aisle from a guy being attended to by two flight attendants at a time on Delta. I found out he was the president of the company traveling “incognito” although everyone knew who he was among the crew. I asked him later, “How was the service?” “Oh, great,” he said, “it’s a wonderful airline!” He thought he sampled the actual service! (And why the hell wasn’t he back in coach instead of in first class?)
Steven B. Levy
From the inside of a corporation: There are two other reasons to do surveys.
1) The Lexus method. As you’re getting ready to close the deal, the salesman says, “In a few weeks, you’ll get a survey asking about your experience. Is there anything we’ve done that would prevent you from rating us a 10?” This question does two things. First, it primes you to feel good about the company, because you want to be a nice guy and give them a 10. Consider it a presumptive close of sorts. Second, and more importantly, if anything, even a little thing, is bugging you, they want to hear about it now, while they can address it, before it festers… or becomes grist for a DASM post. If you walk away unhappy, you’ll be less likely to return, and you’re likely to share your disgruntlement with friends and on social media. In addition, answering an objection often helps cement the sale. If they do anything with these surveys other than scan any written comments and then throw them away, I’d be disappointed. They’re statistical nonsense, but the survey isn’t the point – asking this question in the right time in the sales process is the point.
2) Longitudinal surveys. Asking the same questions in the same order (usually of different responders) and comparing trends over time can suggest the direction in which you’re headed – improving or devolving. A 6.4 on a question, or a 130 NSat rating, is meaningless. However, a 6.4 this month after a 6.2 last month and a 5.8 the month before is useful though not dispositive information. Even spotting a trend doesn’t tell you what’s driving the trend, but at least it’s a data point that you can use in figuring out what’s really going on. The survey itself has no statistical validity, but careful – and eyes-open – analysis of the trends can be useful.
All that said, I do believe about 90% of these are indeed BS. But not 100%.
Alan Weiss
Thanks for writing. If you believe 95% are BS, then the discussion is really moot. What happens, especially with auto dealer surveys for example, is the owners “manage the survey.” Customers are told by employees in sales and service that they’ll get a survey and that if the employee doesn’t get a “10” they won’t get a full bonus. Crap like that. The higher survey ratings put the dealership in better positions with manufacturers and banks. Tasca Ford is famous for this where I live. It’s total nonsense. The search for “10” in your example, is futile. Nothing and no one is ever perfect. Here’s the question: Is there anything at all we haven’t yet discussed that would cause you to hesitate before receiving my proposal? If so, let’s talk about it now. I want the sale, not some “score.”