WP2 1 Evaluation Strategy Design » History » Version 4

Steve Welburn, 2012-08-08 03:34 PM

1 1 Steve Welburn
h1. WP2.1 Evaluation Strategy Design
2 1 Steve Welburn
3 2 Steve Welburn
In order to evaluate the (training) materials, it is necessary to:
4 2 Steve Welburn
* identify specific (learning) objectives which they aim to meet
5 2 Steve Welburn
* evaluate whether the materials meet those objectives
6 2 Steve Welburn
additionally, we need to 
7 2 Steve Welburn
* identify the overall purposes of the materials
8 2 Steve Welburn
* evaluate whether the cumulative objectives satisfy these overall purposes
9 1 Steve Welburn
10 1 Steve Welburn
h2. Methods of Evaluation
11 1 Steve Welburn
12 2 Steve Welburn
* Informal / Formal Review e.g.:
13 2 Steve Welburn
** "Cloze Test":http://en.wikipedia.org/wiki/Cloze_test to assess readability
14 2 Steve Welburn
** Questionnaire to see how easy it is to find relevant material
15 2 Steve Welburn
** Internal use of materials
16 4 Steve Welburn
** "I-Tech Instructional Design & Materials Evaluation Form and Guide":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html
17 4 Steve Welburn
* In-course evaluation (see "I-Tech":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/dev_eval.html)
18 4 Steve Welburn
** Assessment of level of knowledge within training group
19 4 Steve Welburn
** Checking progress with participants
20 4 Steve Welburn
** Trainer assessment - self assessment and from other trainers if possible
21 4 Steve Welburn
** Pre- and Post-course questionnaires - assess change in answers (true/false + multiple-choice)
22 1 Steve Welburn
* Post-course questionnaire
23 4 Steve Welburn
** "Sample training evaluation forms":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/forms.html
24 2 Steve Welburn
* Medium-term review of usefulness of course content / adoption of techniques (e.g. 2-3 months after course)
25 1 Steve Welburn
26 4 Steve Welburn
I-Tech consider four levels of results from training:
27 4 Steve Welburn
* Reaction - to the course
28 4 Steve Welburn
** Pacing, was it enjoyable
29 4 Steve Welburn
* Learning - from the course
30 4 Steve Welburn
** Did the facts get across ?
31 4 Steve Welburn
* Behavior - changes after the course
32 4 Steve Welburn
** Did participants actually manage their data better
33 4 Steve Welburn
*** during research ? 
34 4 Steve Welburn
*** at the end of research ?
35 4 Steve Welburn
** Have data management plans been produced for grant proposals ?
36 4 Steve Welburn
* Results or Impact - to a wider community
37 4 Steve Welburn
** Did they publish data ?
38 4 Steve Welburn
** Was any data loss avoided ?
39 4 Steve Welburn
40 2 Steve Welburn
Review content for:
41 2 Steve Welburn
* reading level
42 2 Steve Welburn
* correctness
43 2 Steve Welburn
* organization
44 2 Steve Welburn
* ease of use
45 3 Steve Welburn
46 2 Steve Welburn
based on the target audience
47 1 Steve Welburn
48 1 Steve Welburn
h2. Tools
49 1 Steve Welburn
50 1 Steve Welburn
"Bristol Online Surveys":http://www.survey.bris.ac.uk/
51 4 Steve Welburn
"I-Tech Training Toolkit":http://www.go2itech.org/HTML/TT06/toolkit/evaluation/index.html