Follow Us:

Proposed set of indicators for ICT in education

The proposed draft set of indicators and parameters or guidelines below were developed by the participants and resource persons based on their group discussions during the last two days of the Consultative Workshop.

Standards/Parameters for the Development of ICT Indicators

1. Definition of Information and Communications Technology (ICT) – ICT is the term used to describe the tools and the processes to access, retrieve, store, organise, manipulate, produce, present and exchange information by electronic and other automated means.

2. Scope of education to be covered by ICT indicators – (primary, secondary, and non-formal education)

3. Criteria for Developing and Assessing ICT Indicators

 

Criterion
Description
Direct measure
  • Indicator is intuitively understood (high face validity)
  • Indicator is a direct measurement, rather than a proxy that depends on assumptions for its validity
  • Indicator is supported by a body of research
Objective
  • Indicator is unambiguous about what is being measured
  • Different people will collect comparable data based on the indicator
  • Definition remains stable over time, so change can be measured
  • Indicator is unidimensional (measures only one thing)
  • Indicator can be quantitative or qualitative, as long as it is clearly and consistently defined and interpreted
Adequate
  • Either by itself or with a minimal companion set of indicators, the indicator provides reasonable confidence that it accurately measures the attribute
  • Object is to have as few indicators as possible per attribute (should be three or fewer) — more is not necessarily better
  • Number of indicators will depend on the complexity of the object, or what is being measured
Quantitative
  • Quantitative indicators are more objective than qualitative ones
  • Qualitative indicators should be adequately specified to be objective and consistent
Disaggregated
  • The more disaggregated the indicator, as per district or province, the more easily data can be manipulated to answer questions not anticipated at the outset - this also helps indicate how evenly ICT use is spread across a country, as infrastructure, development and social economic levels can vary greatly from region to region
Practical
  • Data can be collected at reasonable cost, given their utility
  • Data is available and can be collected at suitable time intervals
  • Data can be readily collected in various projects for comparison
Reliable
  • Indicator is reliable within the context of the evaluation purpose and resources
  • Data-collection process is consistent across different time and space scales, using comparable methods and sampling procedures. Indicator is based on representative data

4. Performance Indicator Categories Based on the preliminary discussion by the four Workshop resource persons, an examination or review of existing indicators used in many countries worldwide, the following categories of performance indicators were adopted:

  • Policy
  • Technology Infrastructure and Access
  • ICT Curriculum
  • Teaching and Teaching Support Staff
  • Learning Process and Outcomes

 

INDICATORS FOR POLICY, TECHNOLOGICAL INFRASTRUCTURE AND ACCESS

1. Policy

Indicators
Definition Purpose From whom to Collect How to Collect
1. Presence of a national policy for ICT in education (formal and non-formal) The policy must be articulated and detailed Confirm the existence of purposeful endeavour Ministry of Education
website,
interview, questionnaire
2. Presence of a master plan with a timeframe   Evidence of intention Ministry of Education Interview, questionnaire
3. Presence of a budget plan
(and appropriations)
  Measure of implementation Ministry of Education Interview, questionnaire
4. Presence of a body responsible for implementing the master plan (Subcontracted) implementing agent Evidence of implementation Ministry of Education Interview, questionnaire
5. Percentage of a national education budget allocated for ICT (total amount in US Dollars, percentage of total budget) Purchase of hardware and software, software development, maintenance,
in-service teacher training,
definitive source of funds
Evidence of commitment Ministry of Education Interview, questionnaire
6. Mechanism for monitoring and evaluating the implementation of the programme Programme monitoring at both the national and local levels Evidence of commitment Ministry of Education Interview, questionnaire
7. ** Does ICT in education policy refer to equity of access for remote schools, minorities, girls, children with special needs? This refers to Digital Divide issues   Evidence of attention to the Digital Divide  

2. TECNOLOGY STRUCTURE AND ACCESS

 

A. Enabling environment

Indicators
Definition
Purpose
From whom to Collect
How to Collect

1. *No. of schools with

  • electricity
  • computers
  • telephone
  • intranet
  • Internet
  • TV/VCR/VCD/DVD
  • radio
These should be used for educational purposes Context of ICT development Ministry of Education, Schools Questionnaire
2. *Number of computers per 100 students

Data must be in bands

Open to guesstimates
  Measure of implementation Ministry of Education, Schools Questionnaire
3. Number of hours per week for ICT-aided instruction
Data must be banded
    Ministry of Education, Schools Questionnaire

4. Percentage of schools using the following equipment for educational purposes:

  • Scanner
  • Colour printer
  • Dot matrix printer
  • Digital camera
  • LCD projector
These should be for educational purposes   Ministry of Education Interview, questionnaire

 *To be disaggregated into formal, non-formal, primary and secondary education, and by province or district

 

B. Internet Connectivity

Indicators
Purpose
From whom to Collect
How to Collect
1. *Number of computers connected to the Internet

Data must be in bands
Measure of connectivity Ministry of Education, Schools  
2. *Hours a month the school uses the Internet School heads /
ICT coordinators of schools

Questionnaire
3. *Number of schools with websites produced by students School heads /
ICT coordinators of schools

 

* To be disaggregated into formal, non-formal, primary and secondary education.

 


C. Speed / Bandwidth / Satellite / Wireless

Indicators
Purpose
From whom to Collect
How to Collect
1. Percentage of schools with broadband, ADSL, narrowband, wireless Measure of quality of connectivity Ministry of Education Interview, questionnaire

D. Systems and Hardware

 

Pre-Pentium Pentium Non-Pentium Peripherals available to school children
1. *Number of PCs running on the Windows platform *Number of PCs with pre-Pentium processors
• Without CD drive
• With CD drive
  Does your school have the following equipment that you use for educational purposes:
• Colour printer
• Dot matrix printer
• Scanner
• Multimedia projector
• UPS

* To be disaggregated into formal, non-formal, primary and secondary education.


3. ICT Curriculum

Disaggregated by:

  • Geographical location (rural or urban)
  • Educational level
  • Type of education (formal, non-formal, and special education)
  • Subject
  • National minorities
Indicators
Definition
Purpose
From whom to Collect
How to Collect
1. Existence of a curriculum that incorporates ICT:
  • compulsory
  • optional
ISCED – definition of primary and secondary education

To be defined:
compulsory
optional

General purpose: to acquire, update, improve, introduce data in order to monitor and efficiently use ICT

  • Basis for teaching
  • Basis for providing educational content
  • Basis for organisational infrastructure
  • Basis for curriculum construction
From the ministries, national curriculum development centres, regional or local education centres, research and teacher training institutes, subject area centres
  • Get documents
  • Surveys
  • Document analysis and review

2. Existence of ICT as a separate subject

  • in primary education
  • in secondary education
    On the basis of national education systems analysis, the mechanism of data obtaining should be defined within the project and suggested in the instructional manual for pilot testing.  
3. Number of schools using ICT as a separate subject Specify overall number of schools, disaggregated by territorial, level of education and other parameters   Ministries of Education, schools, curriculum development centres, regional or local education centres, subject area centres, teachers
  • Expert opinion
  • Curriculum statements
  • Content analysis of curriculum
  • Evidence of additional instructional materials
4. The extent of ICT integration in the curriculum (none, some, much) ICT – material intentionally designed for manipulation by automated means      
5. The extent of ICT integration in teaching and learning programmes Integration is valid only for levels 2,3,4.

Integration:
Existence and number of ICT based lessons in the curriculum relative to the overall curriculum time for the subject

     
6. Number of schools incorporating ICT
(low, medium, high)

Existence of state approved ICT based instructional materials for usage in appropriate educational domains

  • Percentage of educational content covered by ICT based instructional materials

Instructional material is defined as support for teaching and learning ICT

Subject - this listing will be further refined based on an international standard

     

4. Teaching and Teaching Support Staff (includes all staff fulfilling teaching roles like school administrators, school librarians, teachers but excluding maintenance staff)

Disaggregated by:

  • Gender
  • Geographical location (rural or urban)
  • Age
  • Subject taught by a teacher/ librarian/administrator/ICT coordinator
  • Educational level
  • Type of education (formal, non-formal, and special education)
  • National minorities
  • Socio-economic status

 

 

Indicators
Definition
Purpose
From whom to Collect
How to Collect
1. Percentage of teaching professionals who acquired pre-service training Define pre-service training based in ISCED To determine the skills, knowledge, attitude, access and use of the workforce to ICT to serve as a basis for improving teaching staff ‘s efficiency and effectiveness in the use of ICT. Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)
2. Percentage of teaching professionals who received training in the last 3 years against the total number of education staff of appropriate category     Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)

3. Type of ICT training

  • Basic
  • Advanced
Advanced training would include creating web pages, telecollaboration, multimedia, spreadsheet, data analysis   Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)

4. Length of training

  • Less than 10 hours
  • 10 to 30 hours
  • 31 to 70
  • More than 70 hours
    Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)

5. Percentage of teaching professionals
who use computers for teaching purposes

  • Rarely
  • Sometimes
  • Regularly
  • Always
Teaching purposes include lesson preparation, teaching, reporting, recording, communicating, assessing

Comment: Categories (rarely, sometimes etc. should be explained)

Addition by Dr. Boris Kotsik: Teaching purposes can be divided.
  Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)
6. Existence of technology development plan on different levels     Ministries, teacher training institutes, non-formal education centres, schools Surveys (interviews, questionnaires, telephone interviews, expert estimates)

5. Learning Process and Outcomes

Disaggregated by:

  • Gender
  • Geographical location (rural or urban)
  • Subject (This listing will be further refined based on an international standard.)
  • Educational level
  • Type of education (formal, non-formal, and special education)
  • National minorities
  • Socio-economic status

 

COMMENTS:

How can socio economic status be measured?
Look for area based determinants of economic status

 

Indicators
Definition
Purpose
From whom to Collect
How to Collect
1. Number of hours of ICT access per learner per school year School year should be standardised across countries To determine the skills, knowledge, attitude, access and student’s use of ICT to serve as a basis for improving student’s specific and general learning outcomes Teachers and students
Existing documents and secondary sources

Random sampling – some schools, all students and teachers

  • Cost of data collection
  • How can data be collected?
  • Concern for validity of students' responses
2. Number of learners with ICT access outside school Access is the possibility to use ICT for structured learning.

Use is the process of manipulation and exchange of information through ICT for structured learning.
To define access to ICT    
3. Number of learners who demonstrate only basic ICT skills Basic and advanced ICT skills – can be taken from the description of the national curricula      
4. Number of learners who demonstrate advanced ICT skills        
5. Purposes for which learners use ICT
• Informative
• Functional
• Creating
•Communication
Informative: ability to acquire and use information

Functional: ability to use and manipu-late existing inform-ation for educational purposes

Creating: ability to compose, compile, produce new information

Communication: ability to exchange information
     

Situating Indicators Based On Level Of ICT Development

It is felt that many countries are at different stages of development as far as ICT use in education is concerned and, therefore, indicators to measure ICT use and impact may not necessarily be standard or uniformly applicable to all countries. In the region, for example, one country may already be advanced as far as teacher training or access indicators are concerned, but may not even have developed an official policy. Or another country may have formulated an official policy, but may not have begun integrating ICT into the curriculum. The matrix below is adapted from the Applied Morel's Matrix, which originated from the UNESCO IITE High-Level Decision-Makers in ICT Educational Policy in 2001.

The four categories have been defined as follows:

1. Emerging
2. Applying
3. Integrating
4. Transforming

Indicators
Emerging
Applying
Integrating
Transforming
Policy
       
Technology Infrastructure        
ICT in the Curriculum        
Teaching and Teaching Support Staff        
Learning Process and Outcomes        

See also Performance Indicators on ICT for Education Matrix