3.1 Who?
3.2 Why?
3.3 What?
3.4 How?
3.5 The
evaluation
3.6 Final
checklist
3.7 Lessons
learnt
3.8 Models
for timetabling
3.9 Sample
questionnaire
3.5 Evaluation of intercultural communication
training sessions
Gabriela S. Matei
Evaluation is Integral to Training Processes
At the end of our workshop or course, we need to make interim judgments
about how the training work was received by participants. We need to do
summative evaluation. You will probably want to create and manage your
own evaluation forms, or use ready-made ones that you can adapt to your
specific needs for the specific event you will have conducted. To assist
you, here are a few examples of evaluation methods and instruments used
in the National Training Events (intercultural communication training
workshops) held within the framework of the ICCinTE project of the ECML:
A. Evaluation questionnaires, using basic two or three questions:
- What did you like in the workshop?
- What did you dislike in the workshop?
- What is the most important thing you've learned during the workshop?
B. Questionnaires in the form of "Continue the following
statements":
- The most important things I'm taking away from this workshop. ..
- What surprised me was. . . . .
- I would like to recommend to the trainers. . . . . . ..
- I would also like to say. . . . .
C. Group/poster evaluation
A4 or A3 sheets spread on the desks with the following headings:
- Something important I learned during this workshop....
- Something I enjoyed very much in this workshop....
- Something I didn't really like in this workshop....
- Something I would definitely like to try out in my
own teaching....
- Something I would like to give to the workshop
participants....
Using no more than six such posters, both participants and trainers
walk around and write a line or two on each poster. When finished, the
sheets are posted on the walls/ whiteboard, and everybody walks around
to read them.
D. Visual evaluation:
The Thermometer poster with a "content" and "atmosphere"-side
to place coloured stickers in. Before participants left the classroom
they each placed a sticker on the poster which indicated their degree
of contentment with the workshop.
This form of evaluation is very quick and easy to do but generates
no detailed information.
Evaluation Issues
Here are several evaluation issues that you need to think about if your
evaluation procedures are to be useful.
- Information is typically more useful than judgments from participants.
Opinion can fluctuate with emotional state, for example. So it may be
more appropriate to elicit information about what participants experienced,
rather than their judgments.
- Evaluation information is not always reliable.
Simply because a group of trainees has told us that they found a particular
activity unhelpful is not necessarily the trigger to radical revision
of a programme. We must beware the knee-jerk reaction to what others
have noticed. This may be difficult when we don't have experience,
though / hence the value of team training.
- Participants use evaluation for their own purposes.
Participants use evaluations for all sorts of 'non-evaluatory'
reasons, and this is another reason for them not being wholly reliable.
- End-of-course evaluation is not always very helpful to either
trainers or trainees.
'Final day evaluation' has become a ritual. Some trainers
like nothing better than to retire to a quiet place and read the completed
evaluation forms, and luxuriate in the warm glow that they provide.
This may not be entirely helpful, though. How participants feel and
what they think important on the last day is significant, but it's
transitory. How the training influenced the participants' work
in the longer term is what we really need to know.
Post Training Activities and Practices
There are several ways in which we can avoid some of the worst features
of the issues raised. The ideas which follow are designed to assist in
building a repertoire of effective practices for obtaining feedback on
training sessions or workshops/courses.
What makes for effective post-training evaluation?
Here is an experienced trainer talking about her practice.
One thing I've done several times in my training
work is to elicit expectations at the beginning of a workshop, process
them into a list which I type up and then give these out at the end of
the workshop. Each participant gets a copy and ticks those expectations
which have been met for him or her. I sometimes vary this with a Lickert
scale, where I ask participants to rate an activity or input on a 1-5
scale.
This trainer is avoiding the problem of empty or biased opinions by asking
the participants about their own thoughts and opinions
and whether or not they believe that these have been influenced by the
training event. This has advantages over producing something before the
training event, and in effect meeting predetermined expectations.
Different Procedures
- Share the objectives of the training event with participants at the
beginning, and then provide them with an opportunity to discuss these
at the beginning and the end of the event.
- Explain why you are eliciting feedback whenever you do.
- Allow participants to know your criteria for interpreting their responses;
then tell participants how you have analysed and used their feedback
/ your learning or raised awareness.
- Provide ample time / at the right time / for feedback.
- Eliciting feedback at different times of the day, and incorporating
changes made on the basis of this information / AND tell participants
about what you have done and why.
- Use elicitation devices like 'If I (trainer) were to do this
activity/session again next week, what do you think I should do differently,
if anything?'
- Allow written feedback to be anonymous.
- Eliciting feedback from trainees some time after the workshop /
3 or more months later, for example.
Ideas for Self-Evaluation
Even experienced trainers are very self-critical at times, perhaps overly
so. But that doesn't mean we should avoid some sort of self evaluation,
which is helpful for developmental reasons. As with participant evaluation,
we need to gather INFORMATION in order to begin the reflective process.
Useful sources of information, as well as participant feedback, include:
- A 'teaching log' / a journal in which you record
your thoughts about the training at regular intervals during the training
event. Reading this material through at 'distance' from
the event can provide valuable insights. You can even share extracts
with participants when you feel more confident.
- Recordings / video, audio, photographic / of sessions
can provide a wealth of information of, for example, how we use language,
our body language and facial expressions, our movements in the training
room.
- Peer feedback. Work with a partner and ask him/her to observe you
and provide you with feedback. Team training can be very effective.
Other After-Training Activities
Very often we may want to stay in touch with a group, or to assist them
in staying in touch. It's always interesting to find out what individuals
have done as a result of a training event, and whether the work done has
been of longer term value.
The internet provides several means:
- Setting up a team or open blog for individuals to contribute to;
- Setting up discussion forums on an institutional website;
- E-mail bulletins to which participants contribute;
- Short experience reports which can be collected and circulated as
e-newsletters.
Of course, going to visit teachers in schools is hard to beat,
but that may not always be possible.
One interesting way of combining follow-up evaluation and self-evaluation
over a longer term is to do this:
'MESSAGE
IN A BOTTLE'
At
the end of a training event, ask participants to write a letter to themselves
about the training event, describing and evaluating it in as much detail
as they want. They write their letters and give them to the trainer
who undertakes to post them to self-addressed envelopes in 6 months'
time, or send them to e-mail addresses provided by the participants.
Participants agree to respond to the letters by writing, 6 months later,
to the trainer, telling them how they felt when they got the letter
and the extent to which they had changed since they wrote the original
letter. This part could be done on e-mail.
A Final Word
We develop our capacities and skills as trainers through our experience
above all. By eliciting regular feedback from participants, and by self-evaluation
and reflection, we generate valuable information on what we do and how
we come across to trainees which enables us to improve.
A final reminder from a trainer on evaluation:
Elicit feedback on specific aspects of training practice;
this is more helpful than eliciting judgments or opinions from trainees.
We must learn not to be over-reliant on our participants' good feedback
if we are to be genuinely self-critical.
next chapter: 3.6 Final
checklist
|