We are using cookies to give you the best experience on our site. By continuing to use our website without changing the settings, you are agreeing to our use of cookies. View our Privacy Policy.
Relayware Partner Relationship Management

Metrics You Might Miss: Consistent Answers to Questions

Posted on April 2, 2014 by Chris Bucholtz

Partner training materials are usually viewed as a source of two sets of metrics: how much are partners using them, and how many are earning certifications and accreditations? Those are the bottom-line metrics, but there are things to be learned by viewing testing results in a more granular way. Unless you do this, you may have a blind spot into the effectiveness of your testing.

Certifications are often issued when partner personnel achieve an acceptable score – 70, 75 or 80 percent. That’s thought of as acceptable, and thus a certification is issued and the partner is confident that their employee is ready to work with your product. Few of them may have achieved a perfect score – and if the questions they missed are scattered among the test questions, that’s OK.

However, if they’re all missing the same question of group of questions, you have a problem. They’re going into the field with your stamp of approval but all lacking knowledge in a specific area. That kind of blind spot doesn’t just hurt them – it also hurts your brand when everyone representing you to customers is unclear about the same aspect of your products. That looks like a reflection on you, the vendor – and it should, especially because there’s a way to spot these issues.

They’re immediately detectable if you examine scoring across tests broken down into individual questions. Ideally, you should see consistent scores across the entire range of questions. If you see outliers where scores are poor, look into what those questions are. First, make sure the questions are not poorly-worded or otherwise confusing. If the questions are well-written, you may have problems with your training content – it’s not getting across to the people taking the test. Go back to the content and make sure it’s as effective as you need it to be.

This analysis can also reveal other issues with your testing. If you have questions that are successfully answered by the vast majority of the people taking the test – 95 percent or more – you may want to substitute a more challenging question or just drop that question entirely. Having “gimmes” in the mix artificially inflates scores and may help some partner personnel get scores that gain them certifications when they instead should be directed back to the training materials.  To read more on Channel Enablement and Training check out our lastest whitepaper!