Follow Us:

UNESCO IITE Project: Development and use of indicators of ICT in education for the Baltic and CIS countries

By Dr. Boris Kotsik, Senior Programme Manager, UNESCO IITE

Dr. Kotsik's report presented the highlights of the research conducted by the UNESCO Institute for Information Technologies in Education (IITE) in the Baltic countries and Commonwealth of Independent States from 1999 to 2002. His presentation outlined the process of conceptualisation, design, implementation, monitoring and evaluation of these projects.

The creation of knowledge societies depends on both knowledge use and the way schools and educational institutions master the known and upgraded methods of generating, analyzing and disseminating information. The continuous monitoring necessary to provide such a transformation poses certain challenges, involving the following aspects:

  • National, sub-regional and regional potential concerning the use of ICTs in the fields of policy, planning and budget
  • Acquisition, analysis and dissemination of upgraded data and statistical information on ICT use in education
  • Circulation of research results, information on the effective policy, innovations and experiences

General Description The project sought to acquire and process data on ICT usage in education in the sub-regional level, while identifying related problems and making recommendations.

Objective The project was conducted to enhance the potential ICT use for national education capacity-building.

Result The research findings were submitted to UNESCO Member States to assist them in the cration of national action plans for ICT application in education development.

Data Gathering The statistical data was studied and collected in two stages: firstly, the questionnaire was compiled and then specialists were appointed to be responsible for submission of data on indicators for the project.

History of the IITE Project for Baltic and CIS States

The UNESCO Institute for Information Technologies in Education was founded in 1998 to conduct studies on the impact of ICT in education. IITE activities for 1999-2000 included:

1999 - Provision of data on Russian Federation for OrbIT – 2000 Report
2000 - IITE Newsletter #3, 2000 http://www.iite.ru/img/upload/indic.doc
March 2001 - International Expert Meeting ICT in Education: State-of-the Art, Needs and Perspectives – Indicators and Information System
June 2001 - Experts' Meeting in IITE Newsletter #2
October 2001 - Publication of the Final Report of the Expert Meeting; launching of a sub-regional project for Baltic and CIS States
April 2002 - Finalisation of the results of sub-regional project; sub-regional project proposals for South East Europe
May 2002 - National project proposals for Russian Federation
June 2002 - Dissemination of the results of sub-regional project
July 2002 - National pilot project proposals for Belarus

Details of the UNESCO IITE Project for Baltic and CIS States

1. Components

  • Questionnaire development
  • Questionnaire expertise and amendment
  • Establishment contacts with ministers of education
  • Appointment of data provision experts
  • Conducting a workshop on data provision
  • Data collection and assessment
  • Data precision and primary processing
  • Data secondary precision and processing
  • Statistical report composition and dissemination

2. Data structure

The questionnaire was based on the most relevant indicators determining the efficiency of ICT usage in education. The six indicator groups are:

 

1. Official Documents on ICT Usage in Education • Official documents in ICT in secondary education valid till 2002
• Official documents in ICT in secondary education currently in force
2. ICT in Educational Institutions' (EI) Curricula

• Available state curriculum on Informatics and/or ICT
• Informatics and/or ICT as a separate subject
• ICT usage to support other subjects implied in curricula on these subjects

3. Hardware and Equipment

• Computer classrooms availability
• The average number of students per one computer equipped with computer classrooms
• Percentage of IBM- and Apple-compatible computers in total number computers
• Percentage of schools equipped with one or more multimedia system in total equipped with computer classrooms number
• Percentage of schools equipped with local network to total EI equipped with computer classrooms number

4. Software


• Percentage of computers with installed OS DOS™ to total number of computers in computer classes
• Percentage of computers with OS Windows ™ or Apple Macintosh to total computer number in computer classes
• Percent of computers with other OS to total computer number in computer classes
• Percentage of schools with any educational software for teaching the corresponding subjects to total number of schools with computer classes:

    • Elementary school subjects
    • Science subjects
    • Humanities
    • Informatics or ICT

• Percentage of educational software designed by the specialists within the country to total number of educational software used in schools for teaching of the corresponding subject curriculum (elementary school, science subjects, humanities, informatics/information technologies)
• Percentage of educational software designed by foreign specialists to total number of educational software used in schools for teaching of the corresponding curriculum (elementary school, science subjects, humanities, informatics/ information technologies

5. Global communication means

• Percentage of schools without Internet access
• Percentage of schools with limited Internet access – only e-mail
• Percentage of schools with access via dial-up channel
• Percentage of schools with access via dedicated line
• Percentage of schools with own web-pages

6. Personnel

• Percentage of elementary school teachers, subject teachers (except teachers of Informatics and/or Information Technologies), teachers of Informatics and administration who have undertaken the computer literacy course from 1 September 1999 until present time

    • Less than 50 hours (index 1)
    • 50-100 hours (index 2)
    • Over 100 hours (index 3)

• Computer skills of elementary school teachers, subject teachers and teachers of Informatics and administration

    • Elementary computer literacy
    • Proficiency in ICT field

3. Data Processing and Presentation

The data was collected by questionnaire in MSWord format, transformed to the MS Excel table, processed according to appropriate formulas and presented in diagrams.

4. Dissemination of the Results

The survey results were published as a Statistical Report and disseminated through:

  • The net of Focal Points cooperating with IITE
  • National Commissions for UNESCO
  • UNESCO Field Offices
  • Educational institutions and organizations
  • Series of workshops and consultative meetings in the framework of sub-regional projects, i.e.:

Consultative workshop for Developing Performance Indicators for ICT in Education, Quezon City, the Philippines, 28-30 August 2002.

Capacity building of Human Resources: ICT for Education Capacity-Building towards the Construction of a Knowledge Society – project proposals in wake of the High-Level Conference on Strengthening Cooperation in Southeast Europe.

Experiences for Consideration

  • Project participants were both involved and interested in the data collection;
  • The lack of clear organisational tools for such data acquisition led to difficulty in obtaining homogenous and reliable data
  • Due to the diversity of national, geographical, historical, social and economic features under study, comparative data analysis was difficult
  • A thorough reworking and clarification of indicators is needed, along with identification of appropriate means of data obtaining, processing, presenting, keeping and disseminating
  • Continuous research nedd to be organised systematically to gain accuarate information

Based on this, the following can be recommended:

  • Use the materials of the statistic report in shaping national programmes of ICT development in education;
  • Conduct similar research at the local level to formulate national and local programmes of education development;
  • Consider the recommendations on each indicator when identifying the priorities and choosing the most efficient trends in education development;
  • Disseminate best practices

Key Considerations Affecting Research Outcomes

1. The project was under the Council of Ministers of Education of CIS States, covering the post- Soviet Union territory where about three generations of communist rule had produced a rather strict and centralised system of education, with Russian prevailing as the main means of communication. Thus many tasks for this project were simplified, though levels of education development of the countries still vary significantly.

2. As the National Ministries of Education were responsible for the project, no additional budgeting was required, only managerial support. Data was gathered by heads of departments, senior specialists, staff of national Centres of Informational Technologies in Education or Centres of Teacher Training, Retraining and Educational Support. Being facilitated by official letters from the Ministries, the obtained data acquired the status of official information.

A meeting was held on 23 November 2001 in Moscow to discuss the features and details of the information collected, and the unification and simplification methods of data collection, attended by representatives of participating ministries of education, international experts (who analyzed the questionnaire), data gathering experts, IITE hosts and managers.

3. Distinction was made between direct information (national policy documents, curricula), statistical data collected from different sources (amount of hardware), expert estimates (computers age and types, educational software availability, levels of teachers computer confidence) and indirect data gathered from various sources (Internet access, number of teachers retrained in computer literacy). The need to define exact and inexact data was defined, along with what are mean and middle values and the difference between statistical data and expert estimates.

Recommendations were made for the project to include data on ICT application in education in regular national statistics, to correlate with sociological surveys and information obtained by independent sources. Concerns were expressed that since data collection was facilitated by high-ranking government officials, some of this data may be prone to adjustment.

Being obtained from independent sources, the Institute had to work with figures that are contradictory to official statistics. This raised the concern that the financial and managerial support of the project should be independent from national governance.

Data from different areas were in forms that are not compatible with other areas, so the project specialists had to restructure the initial data three or four times. As a result, recommendations were prepared to modify the content of the questionnaire and the method of its completion.

4. The above mentioned 'latent uncertainties' were due partly to the data gatherers' not following instructions and partly to ambiguities in the definition of indicators, data gathering methods and the processing of data:

In their calculations, some experts mistakenly considered the total number as a simple average of numbers for urban and rural schools, while the result depends on the ratio of rural/urban schools.

The experts' calcualtion of the student-computer ratio was taken as the ratio of total number of students in the country to total number of computers in schools (which computes to 500 to 1000 students per computer unit, and this was used as basis for concluding that, on average, a school of 500 pupils in the country has one-half to one computer unit). The experts did not consider that the student-computer ratio was intended as a measure of didactic usability of computers in schools, relative not to total number of students in the country, but only those schools with computer classrooms.

5. It remained unclear how to separate data for elementary and secondary schools in situations where schools comprise a single educational institution, so the Survey referred to secondary education consisting of elementary, basic and secondary stages. This should be more clearly defined in the questionnaire and indicator description, along with how to deal with the cases when one and the same school has both dial-up and dedicated channel access to Internet, or when the same teacher has attended 100-hours computer courses consisting of four 25-hour modules.

During the data precision and processing, some indicators were excluded from the survey data analysis and/or from visual presentation because of their minor values and large amount of missing data. The excluded indicators numbered 13 out of the total 27.

Recommendations for Further Studies

  • An updated indicator system for the next step of the sub-regional project Indicators of ICT usage in secondary education in Baltic and CIS States;
  • An indicator system for the national pilot projects for Belarus and Russian Federation;
  • An aggregated indicator model for overall evaluation of ICT usage in education;
  • A model for integration of indicator model with overall socio-economic data and education evaluation;
  • Indicator models for different education levels, forms and domains – pre-primary, primary, higher, vocational, supplementary, education for people with special needs;
  • Requirements, structure and implementation of information system for automated indicator model data collection, processing, storage, presentation and dissemination;
  • Position paper on ICT Integration in Education for policy- and decision makers of the UNESCO Member States;
  • Instruction materials and training courses on ICT Integration in Education for policy- and decision makers of the UNESCO Member States;

Issues to be Reflected for IS Composition

  • Target audience (countries, languages, professional groups, level of computer competence)
  • IS purpose (information, inventory, monitoring, research, decision-making, complex)
  • Scale (global, regional, national, local)
  • GIS data type if any (vector, raster, combined, 3D)
  • Data storage, updating and administering (periods, type of access):

-Data processing, modeling and analysis
-Data control, visualisation and output
-Database features (type, volume, functions, number of users)
-Distributed access features
-Data acquisition (web-forms, spreadsheets, text-forms, blanks, DBMS applications)