3.1 Who?
3.2 Why?
3.3 What?
3.4 How?
3.5 The evaluation
3.6 Final checklist
3.7 Lessons learnt
3.8 Models for timetabling
3.9 Sample questionnaire

3.5 Evaluation of intercultural communication training sessions

Gabriela S. Matei

Evaluation is Integral to Training Processes

At the end of our workshop or course, we need to make interim judgments about how the training work was received by participants. We need to do summative evaluation. You will probably want to create and manage your own evaluation forms, or use ready-made ones that you can adapt to your specific needs for the specific event you will have conducted. To assist you, here are a few examples of evaluation methods and instruments used in the National Training Events (intercultural communication training workshops) held within the framework of the ICCinTE project of the ECML:

A. Evaluation questionnaires, using basic two or three questions: 

• What did you like in the workshop?
• What did you dislike in the workshop?
• What is the most important thing you've learned during the workshop?

B. Questionnaires in the form of "Continue the following statements":

1. The most important things I'm taking away from this workshop. ..
2.  What surprised me was. . . . .
3.  I would like to recommend to the trainers. . . . . . ..
4.  I would also like to say. . . . .

C. Group/poster evaluation

A4 or A3 sheets spread on the desks with the following headings:

• Something important I learned during this workshop....
• Something I enjoyed very much in this workshop....
• Something I didn't really like in this workshop....
• Something I would definitely like to try out in my
  own teaching....
• Something I would like to give to the workshop 
  participants....

Using no more than six such posters, both participants and trainers walk around and write a line or two on each poster. When finished, the sheets are posted on the walls/ whiteboard, and everybody walks around to read them.

D. Visual evaluation:

The Thermometer poster with a "content" and "atmosphere"-side to place coloured stickers in. Before participants left the classroom they each placed a sticker on the poster which indicated their degree of contentment with the workshop.

This form of evaluation is very quick and easy to do but generates no detailed information.

Evaluation Issues

Here are several evaluation issues that you need to think about if your evaluation procedures are to be useful.

1. Information is typically more useful than judgments from participants.
   Opinion can fluctuate with emotional state, for example. So it may be more appropriate to elicit information about what participants experienced, rather than their judgments.
2. Evaluation information is not always reliable.
   Simply because a group of trainees has told us that they found a particular activity unhelpful is not necessarily the trigger to radical revision of a programme. We must beware the knee-jerk reaction to what others have noticed. This may be difficult when we don't have experience, though / hence the value of team training.
3. Participants use evaluation for their own purposes.
    Participants use evaluations for all sorts of 'non-evaluatory' reasons, and this is another reason for them not being wholly reliable.
4. End-of-course evaluation is not always very helpful to either trainers or trainees.
   'Final day evaluation' has become a ritual. Some trainers like nothing better than to retire to a quiet place and read the completed evaluation forms, and luxuriate in the warm glow that they provide. This may not be entirely helpful, though. How participants feel and what they think important on the last day is significant, but it's transitory. How the training influenced the participants' work in the longer term is what we really need to know.

Post Training Activities and Practices

There are several ways in which we can avoid some of the worst features of the issues raised. The ideas which follow are designed to assist in building a repertoire of effective practices for obtaining feedback on training sessions or workshops/courses.

What makes for effective post-training evaluation?
Here is an experienced trainer talking about her practice.

One thing I've done several times in my training work is to elicit expectations at the beginning of a workshop, process them into a list which I type up and then give these out at the end of the workshop. Each participant gets a copy and ticks those expectations which have been met for him or her. I sometimes vary this with a Lickert scale, where I ask participants to rate an activity or input on a 1-5 scale.

This trainer is avoiding the problem of empty or biased opinions by asking the participants about their own thoughts and opinions and whether or not they believe that these have been influenced by the training event. This has advantages over producing something before the training event, and in effect meeting predetermined expectations.

Different Procedures

• Share the objectives of the training event with participants at the beginning, and then provide them with an opportunity to discuss these at the beginning and the end of the event.
• Explain why you are eliciting feedback whenever you do.
• Allow participants to know your criteria for interpreting their responses; then tell participants how you have analysed and used their feedback / your learning or raised awareness.
• Provide ample time / at the right time / for feedback.
• Eliciting feedback at different times of the day, and incorporating changes made on the basis of this information / AND tell participants about what you have done and why.
• Use elicitation devices like 'If I (trainer) were to do this activity/session again next week, what do you think I should do differently, if anything?'
• Allow written feedback to be anonymous.
• Eliciting feedback from trainees some time after the workshop / 3 or more months later, for example.

Ideas for Self-Evaluation

Even experienced trainers are very self-critical at times, perhaps overly so. But that doesn't mean we should avoid some sort of self evaluation, which is helpful for developmental reasons. As with participant evaluation, we need to gather INFORMATION in order to begin the reflective process. Useful sources of information, as well as participant feedback, include:

• A 'teaching log' / a journal in which you record your thoughts about the training at regular intervals during the training event. Reading this material through at 'distance' from the event can provide valuable insights. You can even share extracts with participants when you feel more confident.
• Recordings / video, audio, photographic / of sessions can provide a wealth of information of, for example, how we use language, our body language and facial expressions, our movements in the training room.
• Peer feedback. Work with a partner and ask him/her to observe you and provide you with feedback. Team training can be very effective.

Other After-Training Activities

Very often we may want to stay in touch with a group, or to assist them in staying in touch. It's always interesting to find out what individuals have done as a result of a training event, and whether the work done has been of longer term value.
The internet provides several means:

• Setting up a team or open blog for individuals to contribute to;
• Setting up discussion forums on an institutional website;
• E-mail bulletins to which participants contribute;
• Short experience reports which can be collected and circulated as e-newsletters.

Of course, going to visit teachers in schools is hard to beat, but that may not always be possible.
One interesting way of combining follow-up evaluation and self-evaluation over a longer term is to do this:

A Final Word

We develop our capacities and skills as trainers through our experience above all. By eliciting regular feedback from participants, and by self-evaluation and reflection, we generate valuable information on what we do and how we come across to trainees which enables us to improve.

A final reminder from a trainer on evaluation:

Elicit feedback on specific aspects of training practice; this is more helpful than eliciting judgments or opinions from trainees. We must learn not to be over-reliant on our participants' good feedback if we are to be genuinely self-critical.

next chapter: 3.6 Final checklist