WP2 1 Evaluation Strategy Design » History » Version 9

« Previous - Version 9/10 (diff) - Next » - Current version
Steve Welburn, 2012-08-08 04:08 PM


WP2.1 Evaluation Strategy Design

In order to evaluate the (training) materials, it is necessary to:
  • identify specific (learning) objectives which they aim to meet
  • evaluate whether the materials meet those objectives
    additionally, we need to
  • identify the overall purposes of the materials
  • evaluate whether the cumulative objectives satisfy these overall purposes

Methods of Evaluation

Design review

Pre-course (evaluation of materials)

On-going evaluation of course

  • Informal / Formal Review e.g.:
    • Questionnaire to see how easy it is to find relevant material / test users knowledge
    • Focus Groups (I-Tech guide)
  • In-course / formative evaluation (see I-Tech)
    • Assessment of level of knowledge within training group
    • Checking progress with participants
    • Trainer assessment - self assessment and from other trainers if possible
    • Pre- and Post-course questionnaires - assess change in answers (true/false + multiple-choice)
  • Post-course / summative evaluation
    • Debriefing of trainer (did it work ? to time ? did it engage people ?)
    • Questionnaire for participants Sample training evaluation forms
    • Medium-term review of usefulness of course content / adoption of techniques (e.g. 2-3 months after course)
Kirkpatrick's Four Level Evaluation Model
  • Reaction - to the course ("motivation" may be more appropriate)
    • Pacing, was it enjoyable
  • Learning - from the course
    • Did the facts get across ?
  • Behavior - changes after the course
    • Did participants actually manage their data better
      • during research ?
      • at the end of research ?
    • Have data management plans been produced for grant proposals ?
  • Results or Impact - to a wider community
    • Did they publish data ?
    • Was any data loss avoided ?
Review content for:
  • reading level
  • correctness
  • organization
  • ease of use

based on the target audience

Tools and links

Bristol Online Surveys

I-Tech Training Toolkit

Instructional System Design approach to training

Free Managemnt Library - Evaluating Training and Results

Lingualinks Implement A Literacy Program - Evaluating Training

Training Works!... ...what you need to know about managing, designing, delivering, and evaluating group-based training