|
Quality Assurance in the Public Sector
A Romanian Case Study
Miruna Carianopol
INTRODUCTION
External evaluation is one the main duties of the School Inspectorate.
There are school inspectorates in all the 42 counties of Romania. The
City of Bucharest, with its almost 500 educational establishments (nursery,
primary, secondary, post-secondary and vocational schools), has one municipal
inspectorate and 6 district ones.
External evaluation is all-comprehensive: it is applied to all the educational
establishments, be they state or private, to all the teachers and all
the subjects included in the national and in the local curriculum.
A. TYPES OF EXTERNAL EVALUATION
There are several types of inspections in the public sector schools
in Romania, some of which focus on the teacher, some on the whole school.
They are usually planned in annual or in semester inspection plans either
by the institution or by the inspectors themselves. The main types of
inspections are listed below:
1. Inspection of teachers
- 'special inspection' (4 classes per teacher, on teacher's request)
- 'current inspection' (2 -4 classes per teacher, at the inspector's decision)
2. Inspection of schools
- 'theme inspection' (at least once a month, on topics agreed on by inspectors,
schools are allotted to inspectors by inspectorate)
- general, 'RODIS' inspection (one-week evaluation of 9 aspects of school-life,
7-8 inspectors involved, in 3 stages: pre-inspection, inspection, post-inspection)
B. BRIEF DESCRIPTION OF EACH INSPECTION TYPE
The 'special inspection', for instance, is meant first, to evaluate
the teacher's professional capacities and secondly, to counsel him/her
on planning, methodology, use of materials, class management, students'
assessment. It is initiated by the teacher whenever he/she needs
to take an exam to further their career. (It might take place whenever
the teacher needs counselling from the inspector). This inspection is
part and parcel of the tests necessary in order to obtain this promotion.
The time of the inspection is set by the teacher and it takes place in
three stages: two pre-inspections and one 'special inspection'.
The two pre-inspections, which are at least one month apart, are meant
to allow for giving feedback and counselling and are a most welcome, though
time-consuming recent development. Recommendations made after a pre-inspection
are the starting point for the next pre-inspection or for the special
inspection itself. Any of the three stages of the 'special inspection'
consists in three class-observation activities, a test passed to a fourth
group of students and a discussion between the teacher and the inspector,
which is an opportunity for teacher self-evaluation, inspector feedback
and recommendations. In this type of inspection the inspector may be replaced
by an assistant-inspector (i.e. a full-load teacher who is appointed annually
by the inspectorate to replace the inspector in inspections focused on
the teacher. In Bucharest, for instance, there are 2 English inspectors
and 24 English assistant-inspectors for over 1,200 teachers of English).
The evaluation criteria are those in the observation sheet of the RODIS
inspection (see the RODIS inspection below), namely: planning, strategies,
students' academic standards, relationships between teacher and students
/ students and students, students' attitude during the lesson, resources
used.
Another type of inspection focusing on the teacher is the 'current
inspection'. Its aims are first, to counsel the teacher and, secondly,
to evaluate his/her activity. It is initiated by the inspector and is
performed by a team of one inspector and one assistant-inspector. They
inspect all the English teachers in the respective school. Incidentally,
this is a very good opportunity to train or simply to liaise with assistant-inspectors
in real-life situations. This inspection can last between two to four
class-observation activities for each teacher, depending on when the inspector
and the assistant-inspector reach a consensus. In principle, every teacher
should undergo a 'current inspection' every two years.
The RODIS inspection is a whole-school inspection. It follows
a set of regulations included in Regulamentul de organizare si desfasurare
a inspectiei scolare, issued by the Ministry of Education and Research
in 1998, and whose initials in Romanian give the name 'RODIS'.
The aims of this type of inspection are to offer counselling to the school
and to evaluate its performances. The team of inspectors is made of up
to 8 people who look at 9 criteria of evaluation:
- the academic standards reached by the students,
- the degree of support given by the school to
students for their personal development,
- the quality of the teachers' activity,
- the efficiency of the school management,
- the way the school curriculum is developed (including
extracurricular activities),
- the school's relationship with parents,
- the school's relationship with the local community,
- the way the school's legislation is implemented,
- the pupils' attitude to the education offered
by the school.
This inspection is performed in several stages:
- In the pre-inspection stage, the coordinator of the team of
inspectors visits the school and collects materials that the inspectors
need to consult before the inspection proper begins.
- The inspection proper lasts for one week.
- The post-inspection stage: the coordinator's report, based
on the conclusions reached by the team of inspectors, has to be completed
within a week after the inspection. At this stage, the school can negotiate
with the coordinator, can bring new evidence to modify the conclusions.
After this stage of negotiation, the conclusions are final and the inspection
report is written by the coordinator and sent to the school and to the
local inspectorate. The recommendations made in the report are the starting-point
for the school's improvement plan which is to be monitored by the local
inspectorate during the post-inspection stage.
The selection of schools for this type of inspection is made by the inspectorate
at the beginning of each school semester. In principle, every school should
go through a RODIS inspection every four years. In fact, in Bucharest
there are so many schools and so few inspectors that the cycle is of at
least ten years.
Another type of school inspection is the 'theme inspection'. It
takes place regularly (usually, once a month) in all educational establishments
and refers mainly to school management. At the beginning of the school
year, each school is allotted to an inspector for monitoring and counselling.
Periodically, each inspector visits 'their' schools and checks on the
school's activity according to a set of inspection aims set by the inspectorate
for the current month, but also on problems specific to that school. The
inspection aims are set by an Inspection Board of the municipal inspectorate
and are applied to all the educational establishments in Bucharest. Before
the theme inspection actually takes place, inspectors are briefed on how
to evaluate the inspection's aims in order to ensure a standardized evaluation.
For instance, in the theme inspections taking place in February, with
a follow-up in May, one of the aims would be to check on the way the school
passes on information about the upcoming national examinations of June
/ July to pupils / students and to parents The inspectorate can thus obtain
a partial, but immediate diagnosis of the way educational establishments
are run and can take appropriate and corrective steps in due time.
Of course, there is also a type of inspection which is initiated by the
inspectorate (or the Ministry) whenever a complaint is made regarding
a teacher's / school's activity. In such cases a team of inspectors is
appointed by the inspectorate / Ministry to check on the truth of the
complaint. In such cases the inspection can last several days.
All types of inspection are monitored and evaluated by the Inspection
Board chaired by the Deputy General Inspector in charge with Inspection
and Curriculum.
This table summarizes the information given above:
|
|
C. INSPECTION TRAINING
The types of external evaluation listed above have been prepared by a
series of training programmes targeted at different professional groups.
These programmes were implemented either by the Ministry of Education
together with The World Bank, or by the British Council in Romania. In
most cases a cascading strategy was adopted: training of national trainers
who, in turn train regional trainers, who, in turn train headteachers,
teachers, mentors, etc. As a result, the following groups have been trained
so far:
- trainers: headteacher trainers, inspector trainers, teacher trainers,
mentor trainers (by the Ministry of Education and by the British Council)
- teachers (by the Ministry of Education and by the British Council)
- mentors (by the British Council)
- headteachers (by the Ministry of Education)
- inspectors (by the Ministry of Education together with CfBT, UK, and
by the British Council)
- assistant inspectors (by the English inspectors)
D. INSPECTION REGULATIONS AND INSTRUMENTS
Another preparatory step has been aimed at devising regulations and
evaluation instruments. An important instrument is the 'RODIS',
a set of regulations referring to the whole-school inspection mentioned
above. In addition, other instruments have been devised locally, either
by local inspectorates or by individual inspectors: the annual / semester
inspection plan, the class observation sheet, etc.
E. CONCLUSIONS
After more than four years of running external evaluation as described
above, we can draw some conclusions:
- there is an obvious and most welcome focus on counselling in
most types of inspection
- a change of atmosphere and perception of inspections has taken
place, due to the transparency of evaluation criteria and to the fact
that schools / teachers are notified well in advance about any kind
of inspection
- the RODIS inspection has set new values in school management,
which has had a positive washback-effect on schools' current activity
in preparation for a RODIS inspection: schools have started focusing
on 'non-traditional' areas, such as the degree of support given by the
school to students for their personal development, the school's relationship
with the local community, the way the school's legislation is implemented,
the pupils' attitude to the education offered by the school
- the RODIS inspection gives a chance to all schools, including
vocational schools, whose academic standards are lower but whose management
is modern and 'alive'
- access to inspection results: since the results of whole-school
inspections are public, the local community, including the media, has
access to these results. This, coupled with recent measures to decentralize
school funding, forces schools to feel even more accountable to local
communities than they used to.
Areas that still need improving:
- The data collected through all the types of inspection mentioned above
have to be processed more systematically, so as to allow for an even
better informed diagnosis;
- In addition to all inspectors' training in applying the RODIS set
of regulations, training in carrying out the subject inspections should
also be standardised. At the moment, there are concerns for generalising
the positive experience of the inspectors of English, who have gone
through formal training, as part of the British Council's ELT policy.
Formal training in this area would ensure that inspectors of different
subjects acquire common sets of methodological values and develop more
coherent messages for their teachers. It would have a beneficial effect
also on assistant inspectors, so that all inspectors get to 'speak the
same language'
- Increasing the effectiveness of the cascading strategy in training
programmes, so that messages do not lose impact in the implementation
stage, when going down the 'cascade'
- Familiarising schools and inspectors with the principle of 'added
value' both in whole-school inspections and in special inspections
- Introducing peer-observation on a larger scale as a means of professional
development
- Increasing the number of the resources (inspectors and assistant-inspectors)
in accordance with the actual needs of the public system
Although the problems listed above are not easy to solve, some steps
have already been taken.
SHARING
It is a widely acknowledged fact that the ELT community in Romania have
had better opportunities to acquire new skills, to adopt interactive teaching
methods, to access new information and be more open and flexible toward
it. Because the gap tends to widen between ELT and non-ELT professionals,
it has become a matter of urgency for ELT professionals to disseminate
their expertise toward non-ELT areas. On the other hand, there is growing
awareness that subject inspectors, assistant inspectors and teachers of
different subjects need to communicate and share experience and expertise.
This is imperative because, due to their different training backgrounds,
all these people may, and actually do understand different things by such
things as classroom methodology, classroom management, error-correction,
criteria for selecting materials / textbooks, student assessment criteria,
subject competitions, etc. An answer to this major problem is to have
common events in which professionals in different subjects come together
and share experience and expertise. Such an event was arranged in Bucharest
in November 2001 when inspectors and assistant inspectors of Romanian,
English, French, German, Spanish, Italian, Russian and Latin (almost 200
people) spent a day together in plenaries and workshops on topics related
to methodology, local curricula, student assessment criteria, etc.
FROM THE WHAT TO THE HOW
Moreover, whereas so far focus has been laid on what is actually going
on in the classroom - what methods are used, what materials / textbooks
teachers have at their disposal, what the local curriculum consists of,
etc - we feel the time has come for a shift towards the how of it, namely
towards evaluation and self-evaluation.
Finally, whereas so far focus has been laid on what is actually going
on in the classroom (what methods are used, what materials / textbooks
teachers have at their disposal, what the local curriculum consists in,
etc), the time has come for a shift towards evaluation and self-evaluation.
As a matter of fact, it is felt by more and more teachers, school managers
and inspectors that evaluation and assessment need to be even more updated,
to correspond better to the changes produced in classroom practice by
young teachers and by more motivated and better informed students.
|