Sound Data Management Training

h1. WP2.1 Evaluation Strategy Design

In order to evaluate the (training) materials, it is necessary to:

* identify specific (learning) objectives which they aim to meet

* evaluate whether the materials meet those objectives

additionally, we need to 

* identify the overall purposes of the materials

* evaluate whether the cumulative objectives satisfy these overall purposes

h2. Methods of Evaluation

Design review

Pre-course (evaluation of materials)

* "Cloze Test":http://en.wikipedia.org/wiki/Cloze_test to assess readability

* I-Tech Instructional Design & Materials Evaluation "Form and Guidelines":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html

* Focus Groups ("I-Tech guide":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html) e.g. some of C4DM

* Internal use of materials i.e. C4DM

On-going evaluation of course 

* Informal / Formal Review e.g.:

** Questionnaire to see how easy it is to find relevant material / test users knowledge

** Focus Groups ("I-Tech guide":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html)

* In-course / formative evaluation (see "I-Tech":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/dev_eval.html)

** Assessment of level of knowledge within training group

** Checking progress with participants

** Trainer assessment - self assessment and from other trainers if possible

** Pre- and Post-course questionnaires - assess change in answers (true/false + multiple-choice)

* Post-course / summative evaluation

** Debriefing of trainer (did it work ? to time ? did it engage people ?)

** Questionnaire for participants "Sample training evaluation forms":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html

** Medium-term review of usefulness of course content / adoption of techniques (e.g. 2-3 months after course)

"Kirkpatrick's Four Level Evaluation Model":http://www.nwlink.com/~donclark/hrd/isd/kirkpatrick.html

* Reaction - to the course ("motivation" may be more appropriate)

** Pacing, was it enjoyable

* Learning - from the course

** Did the facts get across ?

* Behavior - changes after the course

** Did participants actually manage their data better

*** during research ? 

*** at the end of research ?

** Have data management plans been produced for grant proposals ?

* Results or Impact - to a wider community

** Did they publish data ?

** Was any data loss avoided ?

Review content for:

* reading level

* correctness

* organization

* ease of use

based on the target audience

h2. Tools and links

"Bristol Online Surveys":http://www.survey.bris.ac.uk/

"I-Tech Training Toolkit":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/index.html 

Instructional System Design approach to training

* "Analysis, Design, Development, Implement, Evaluate (ADDIE)":http://www.nwlink.com/~donclark/hrd/sat6.html

Free Managemnt Library - "Evaluating Training and Results (ROI of Training)":http://managementhelp.org/training/systematic/ROI-evaluating-training.htm

Lingualinks "Implement A Literacy Program - Evaluating Training":http://www.sil.org/lingualinks/literacy/ImplementALiteracyProgram/EvaluatingTraining.htm

"Training Works!...":http://www.reproline.jhu.edu/english/6read/6training/Tngworks/index.htm ...what you need to know about managing, designing, delivering, and evaluating group-based training