Quality Assurance in the Public Sector
– A Romanian Case Study –

Miruna Carianopol

INTRODUCTION
External evaluation is one the main duties of the School Inspectorate.

There are school inspectorates in all the 42 counties of Romania. The City of Bucharest, with its almost 500 educational establishments (nursery, primary, secondary, post-secondary and vocational schools), has one municipal inspectorate and 6 district ones.

External evaluation is all-comprehensive: it is applied to all the educational establishments, be they state or private, to all the teachers and all the subjects included in the national and in the local curriculum.

A. TYPES OF EXTERNAL EVALUATION
There are several types of inspections in the public sector schools in Romania, some of which focus on the teacher, some on the whole school. They are usually planned in annual or in semester inspection plans either by the institution or by the inspectors themselves. The main types of inspections are listed below:

1. Inspection of teachers
- 'special inspection' (4 classes per teacher, on teacher's request)
- 'current inspection' (2 -4 classes per teacher, at the inspector's decision)

2. Inspection of schools
- 'theme inspection' (at least once a month, on topics agreed on by inspectors, schools are allotted to inspectors by inspectorate)
- general, 'RODIS' inspection (one-week evaluation of 9 aspects of school-life, 7-8 inspectors involved, in 3 stages: pre-inspection, inspection, post-inspection)

B. BRIEF DESCRIPTION OF EACH INSPECTION TYPE
The 'special inspection', for instance, is meant first, to evaluate the teacher's professional capacities and secondly, to counsel him/her on planning, methodology, use of materials, class management, students' assessment. It is initiated by the teacher whenever he/she needs to take an exam to further their career. (It might take place whenever the teacher needs counselling from the inspector). This inspection is part and parcel of the tests necessary in order to obtain this promotion. The time of the inspection is set by the teacher and it takes place in three stages: two pre-inspections and one 'special inspection'.

The two pre-inspections, which are at least one month apart, are meant to allow for giving feedback and counselling and are a most welcome, though time-consuming recent development. Recommendations made after a pre-inspection are the starting point for the next pre-inspection or for the special inspection itself. Any of the three stages of the 'special inspection' consists in three class-observation activities, a test passed to a fourth group of students and a discussion between the teacher and the inspector, which is an opportunity for teacher self-evaluation, inspector feedback and recommendations. In this type of inspection the inspector may be replaced by an assistant-inspector (i.e. a full-load teacher who is appointed annually by the inspectorate to replace the inspector in inspections focused on the teacher. In Bucharest, for instance, there are 2 English inspectors and 24 English assistant-inspectors for over 1,200 teachers of English).

The evaluation criteria are those in the observation sheet of the RODIS inspection (see the RODIS inspection below), namely: planning, strategies, students' academic standards, relationships between teacher and students / students and students, students' attitude during the lesson, resources used.

Another type of inspection focusing on the teacher is the 'current inspection'. Its aims are first, to counsel the teacher and, secondly, to evaluate his/her activity. It is initiated by the inspector and is performed by a team of one inspector and one assistant-inspector. They inspect all the English teachers in the respective school. Incidentally, this is a very good opportunity to train or simply to liaise with assistant-inspectors in real-life situations. This inspection can last between two to four class-observation activities for each teacher, depending on when the inspector and the assistant-inspector reach a consensus. In principle, every teacher should undergo a 'current inspection' every two years.

The RODIS inspection is a whole-school inspection. It follows a set of regulations included in Regulamentul de organizare si desfasurare a inspectiei scolare, issued by the Ministry of Education and Research in 1998, and whose initials in Romanian give the name 'RODIS'. The aims of this type of inspection are to offer counselling to the school and to evaluate its performances. The team of inspectors is made of up to 8 people who look at 9 criteria of evaluation:

• the academic standards reached by the students,
• the degree of support given by the school to students for their personal development,
• the quality of the teachers' activity,
• the efficiency of the school management,
• the way the school curriculum is developed (including extracurricular activities),
• the school's relationship with parents,
• the school's relationship with the local community,
• the way the school's legislation is implemented,
• the pupils' attitude to the education offered by the school.

This inspection is performed in several stages:

• In the pre-inspection stage, the coordinator of the team of inspectors visits the school and collects materials that the inspectors need to consult before the inspection proper begins.
• The inspection proper lasts for one week.
• The post-inspection stage: the coordinator's report, based on the conclusions reached by the team of inspectors, has to be completed within a week after the inspection. At this stage, the school can negotiate with the coordinator, can bring new evidence to modify the conclusions. After this stage of negotiation, the conclusions are final and the inspection report is written by the coordinator and sent to the school and to the local inspectorate. The recommendations made in the report are the starting-point for the school's improvement plan which is to be monitored by the local inspectorate during the post-inspection stage.

The selection of schools for this type of inspection is made by the inspectorate at the beginning of each school semester. In principle, every school should go through a RODIS inspection every four years. In fact, in Bucharest there are so many schools and so few inspectors that the cycle is of at least ten years.

Another type of school inspection is the 'theme inspection'. It takes place regularly (usually, once a month) in all educational establishments and refers mainly to school management. At the beginning of the school year, each school is allotted to an inspector for monitoring and counselling. Periodically, each inspector visits 'their' schools and checks on the school's activity according to a set of inspection aims set by the inspectorate for the current month, but also on problems specific to that school. The inspection aims are set by an Inspection Board of the municipal inspectorate and are applied to all the educational establishments in Bucharest. Before the theme inspection actually takes place, inspectors are briefed on how to evaluate the inspection's aims in order to ensure a standardized evaluation. For instance, in the theme inspections taking place in February, with a follow-up in May, one of the aims would be to check on the way the school passes on information about the upcoming national examinations of June / July to pupils / students and to parents The inspectorate can thus obtain a partial, but immediate diagnosis of the way educational establishments are run and can take appropriate and corrective steps in due time.

Of course, there is also a type of inspection which is initiated by the inspectorate (or the Ministry) whenever a complaint is made regarding a teacher's / school's activity. In such cases a team of inspectors is appointed by the inspectorate / Ministry to check on the truth of the complaint. In such cases the inspection can last several days.

All types of inspection are monitored and evaluated by the Inspection Board chaired by the Deputy General Inspector in charge with Inspection and Curriculum.

This table summarizes the information given above:

C. INSPECTION TRAINING
The types of external evaluation listed above have been prepared by a series of training programmes targeted at different professional groups. These programmes were implemented either by the Ministry of Education together with The World Bank, or by the British Council in Romania. In most cases a cascading strategy was adopted: training of national trainers who, in turn train regional trainers, who, in turn train headteachers, teachers, mentors, etc. As a result, the following groups have been trained so far:

• trainers: headteacher trainers, inspector trainers, teacher trainers, mentor trainers (by the Ministry of Education and by the British Council)
• teachers (by the Ministry of Education and by the British Council)
• mentors (by the British Council)
• headteachers (by the Ministry of Education)
• inspectors (by the Ministry of Education together with CfBT, UK, and by the British Council)
• assistant inspectors (by the English inspectors)

D. INSPECTION REGULATIONS AND INSTRUMENTS
Another preparatory step has been aimed at devising regulations and evaluation instruments. An important instrument is the 'RODIS', a set of regulations referring to the whole-school inspection mentioned above. In addition, other instruments have been devised locally, either by local inspectorates or by individual inspectors: the annual / semester inspection plan, the class observation sheet, etc.

E. CONCLUSIONS
After more than four years of running external evaluation as described above, we can draw some conclusions:

• there is an obvious and most welcome focus on counselling in most types of inspection
• a change of atmosphere and perception of inspections has taken place, due to the transparency of evaluation criteria and to the fact that schools / teachers are notified well in advance about any kind of inspection
• the RODIS inspection has set new values in school management, which has had a positive washback-effect on schools' current activity in preparation for a RODIS inspection: schools have started focusing on 'non-traditional' areas, such as the degree of support given by the school to students for their personal development, the school's relationship with the local community, the way the school's legislation is implemented, the pupils' attitude to the education offered by the school
• the RODIS inspection gives a chance to all schools, including vocational schools, whose academic standards are lower but whose management is modern and 'alive'
• access to inspection results: since the results of whole-school inspections are public, the local community, including the media, has access to these results. This, coupled with recent measures to decentralize school funding, forces schools to feel even more accountable to local communities than they used to.

Areas that still need improving:

• The data collected through all the types of inspection mentioned above have to be processed more systematically, so as to allow for an even better informed diagnosis;
• In addition to all inspectors' training in applying the RODIS set of regulations, training in carrying out the subject inspections should also be standardised. At the moment, there are concerns for generalising the positive experience of the inspectors of English, who have gone through formal training, as part of the British Council's ELT policy. Formal training in this area would ensure that inspectors of different subjects acquire common sets of methodological values and develop more coherent messages for their teachers. It would have a beneficial effect also on assistant inspectors, so that all inspectors get to 'speak the same language'
• Increasing the effectiveness of the cascading strategy in training programmes, so that messages do not lose impact in the implementation stage, when going down the 'cascade'
• Familiarising schools and inspectors with the principle of 'added value' both in whole-school inspections and in special inspections
• Introducing peer-observation on a larger scale as a means of professional development
• Increasing the number of the resources (inspectors and assistant-inspectors) in accordance with the actual needs of the public system

Although the problems listed above are not easy to solve, some steps have already been taken.

SHARING
It is a widely acknowledged fact that the ELT community in Romania have had better opportunities to acquire new skills, to adopt interactive teaching methods, to access new information and be more open and flexible toward it. Because the gap tends to widen between ELT and non-ELT professionals, it has become a matter of urgency for ELT professionals to disseminate their expertise toward non-ELT areas. On the other hand, there is growing awareness that subject inspectors, assistant inspectors and teachers of different subjects need to communicate and share experience and expertise. This is imperative because, due to their different training backgrounds, all these people may, and actually do understand different things by such things as classroom methodology, classroom management, error-correction, criteria for selecting materials / textbooks, student assessment criteria, subject competitions, etc. An answer to this major problem is to have common events in which professionals in different subjects come together and share experience and expertise. Such an event was arranged in Bucharest in November 2001 when inspectors and assistant inspectors of Romanian, English, French, German, Spanish, Italian, Russian and Latin (almost 200 people) spent a day together in plenaries and workshops on topics related to methodology, local curricula, student assessment criteria, etc.

FROM THE WHAT TO THE HOW
Moreover, whereas so far focus has been laid on what is actually going on in the classroom - what methods are used, what materials / textbooks teachers have at their disposal, what the local curriculum consists of, etc - we feel the time has come for a shift towards the how of it, namely towards evaluation and self-evaluation.

Finally, whereas so far focus has been laid on what is actually going on in the classroom (what methods are used, what materials / textbooks teachers have at their disposal, what the local curriculum consists in, etc), the time has come for a shift towards evaluation and self-evaluation. As a matter of fact, it is felt by more and more teachers, school managers and inspectors that evaluation and assessment need to be even more updated, to correspond better to the changes produced in classroom practice by young teachers and by more motivated and better informed students.