Measuring Sim Performance – Can Kirkpatrick cut the mustard?
My colleagues and I often talk about measurement – examples of businesses, university courses, etc. where games and simulations built in Thinking Worlds have improved performance, whether on-the-job or in-the-class. It’s always great for Caspian when we can show clients how effective games and simulations are. For clients, credible evaluation of interventions can save money, time, and L&D jobs. The trouble though, is that arriving at meaningful, bulletproof data can take a lot of time and effort – Kirkpatrick levels 1, 2, and 3 certainly don’t cut it (http://donaldclarkplanb.blogspot.com/2006/09/donald-talks-bollocks.html).
So when I asked whether it’s possible to conduct a rigorous but fairly quick evaluation of interventions – whether training interventions or not – our Head of Instructional Design, Brian Bishop, suggested APE, his Amalgamated Process for Evaluation. In the attached paper that he wrote while working on his PhD in Human Performance Technology, Brian describes APE. The A part of APE results from Brian borrowing the best of Kirkpatrick, Brinkerhoff’s Success Case Method, and the Full-Scope Evaluation suggested by Dessinger and Moseley, altering their timing, and throwing away the frivolous, pie-in-the-sky stuff. Although the paper looks very academic at first, stay with it – it make sense, and gives us in the real L&D world a way to evaluate that might just finally work.
Check out the paper here.