ForeSee to E-Gov: Don’t Mess Up Now! Part 2

by Dave Lewan

When I enthusiastically encouraged the digital arm of the federal government to “not mess it up now” regarding its new strategy to implement new initiatives that would provide an agency’s services and content via a mobile device (phone or tablet), I really meant it. So much so, in fact, I felt it deserved three posts to illustrate my point.

In the first post, I talked a lot about the need for a proven methodology to measure customer or, in this case, citizen satisfaction. I used to think that having a sound methodology went without saying.  I’m starting to second guess my naivety. I’ve seen (even of lately) too many companies and now government agencies rallying around metrics that don’t have a methodology let alone a proven one.

Anyone can invent or establish a set or system of methods, principles, and rules and call it a methodology. If it were suddenly known that Louis Feinberg, Jerome Horwitz, and Moses Horwitz had slapped together a few ideas and called it the LCM 3-S satisfaction metric, touted it as the next best thing since the ACSI, and sold it for free (yes, free), you might not buy into it…but I would almost guarantee that some would. There would be companies and government agencies getting behind such a thing without understanding what quality, if any, it would bring to the table.

My point is that while yes, having a methodology is a good step, what really provides you with value is to look at how credible, reliable, precise, accurate, actionable, and predictive the methodology is. How widely accepted is the measure? Does it have a sound track record of results? Is it based on a scientifically and academically proven and rigorous methodology? Will management trust it?

Most important of all, is there financial evidence that it works? Is there any actual, scientific proof that this methodology will produce the results you want and need it to? You should be looking for academic, peer-reviewed research to back up any claims about what this methodology can do for your agency, department or program.

So, what does it mean to be a credible, reliable, precise, accurate, actionable, and predictive methodology? Here is the breakdown:

• Credible: How widely accepted is the measure? Does it have a good track record of results? Is it based on a scientifically and academically rigorous methodology? Will management trust it?
• Reliable: Is it a consistent standard that can be applied across the customer lifecycle and multiple channels? When all remains the same do we get the same results with every measurement?
• Precise: Is it specific enough to provide insight? Does it use multiple related questions to deliver greater accuracy and insight?
• Accurate: Is the measurement right? Is it representative of the entire customer base, or just an outspoken minority? Do the questions capture self-reported importance or can they derive importance based on what customers say? Does it have an acceptable margin of error and realistic sample sizes? From our private sector experience, most customers will report that a lower price is important to them, but lowering the price may not induce them to buy.
• Actionable: Does it provide any insight into what can be done to encourage citizens to return to the site, or recommend it? Does it prioritize improvements according that would have the greatest impact on the experience citizens have on the site? A score without actionable insight helps us keep score but not improve!
• Predictive: Can it project the future behaviors of the constituency base on their satisfaction with the site visit? There just aren’t enough time, money and resources to go around. The goal is to invest our efforts and dollars in those things that will yield value. Without predictive capability we are left to shoot at our targets in the dark.

Metrics that don’t have the above qualities can do more harm than good. They will provide you with a false sense of security that may lead you to make poor decisions based on bad data – “garbage in … garbage out”. If you think citizens are happy with your web or soon-to-be mobile experience…and they are not… they will not be visiting again anytime soon, or recommending to their friends and family – important behaviors you need to be successful and gain critical funding you need.

The stakes are pretty high. So, don’t let The Three Stooges (Louis “Larry” Feinberg, Jerome “Curly” Horwitz, and Moses “Moe” Horwitz) decide what your next step in your digital strategy will be.

Next time I will discuss the “must haves” of a proper survey technology. Subscribe to my posts or to The ForeSee Blog for more.