Follow Us:

Developing performance indicators for ICT use in education: Australia's experience

By Dr. Michelle Bruniges, Director, Strategic Information and Planning,
New South Wales Department of Education and Training

Dr. Bruniges presented the highlights of Australia’s experience in integrating ICT in the education system, and in developing monitoring and evaluation mechanisms to track how ICT has impacted on the performance of teachers and learners.

1. Australia's Policy On The Use Of ICT In Education

Each of Australia's seven States and Territories has a Minister for Education and Training. Together with the Commonwealth Education and Training Minister, they form the Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA). MCEETYA is the key body in Australia that formulates national education policy, goals and objectives.

The implementation of MCEETYA's national policies at the state and territory level (such as how ICT will be implemented in schools) is the jurisdiction of the individual state and territory Ministers of Education.

2. Goals And Objectives Of The ICT For Education Programme

Australia’s National Goals for Schooling and ICT The key national goal that frames Australia’s ICT programmes in schools specifies:

when students leave school, they should … be confident, creative and productive users of new technologies, particularly information and communication technologies, and understand the impact of those technologies on society.

Australian policy on implementing ICT in education is geared to actualise the role of education and training in the strengthening of "an equitable, imaginative and economically strong knowledge society… Education and training will continue to grow in importance as Australia's economy and society become more knowledge-based and globally integrated."

ICT is integrated into education to improve and increase the quality, accessibility and cost-efficiency of the delivery of education, while taking advantage of the benefits of networking learning communities together to equip them to face the challenges of global competition.

To achieve this shared vision for all Australian students, teachers, schools and school systems, MCEETYA focused on the following priorities:

  • Ensuring that the education and training sector is able to provide all learners with opportunities to develop their ability to use technology confidently;
  • Supporting education and training workers to acquire and maintain the skills needed to take full advantage of the potential of ICT to transform learning;
  • Providing effective and affordable access to the Internet for all learners, regardless of their geographic location;
  • Promoting collaboration in the development and dissemination of high quality digital educational content, services and applications that enable learners to gain maximum education benefits from the online revolution, while also developing a market and generating export income;
  • Sharing leading practice and research on ICT issues;
  • Working across agencies at all levels of government to ensure the development of a policy and regulatory framework that supports the uptake of ICT in education and training.

MCEETYA's strategies are grounded in pursuing collaborative action throughout all education and training sectors. To date, four national strategic action plans have been developed:

  • The National Education and Training Action Plan
  • The action plan for the vocational education and training sector
  • The action plan for the higher education sector
  • The action plan for the school education sector, Learning in an On-line World

Monitoring and Reporting on Australia’s National Goals To actualise the National Goals for Schooling, MCEETYA established the National Education Performance Monitoring Taskforce (NEPMT), whose responsibility was to develop key performance measures for monitoring and reporting the achievement of the National Goals for Schooling on a nationally comparable basis.

A key priority for the initial development of performance measures was Information and Communication Technology (ICT), which was considered crucial to Australia’s educational, economic and social well-being. The NEPMT commissioned a project in 2000 to develop key performance measures for monitoring the ICT knowledge and skills of Australian school students. The outcome of this process was a report to the NEPMT, Monitoring Progress towards the National Goals for Schooling: Information and Communication Technology (ICT) Skills and Knowledge. The report:

  • Describes the context of learning ICT skills and knowledge as an outcome of schooling;
  • Identifies and describes performance measurement approaches, definitions and sources of data currently used in schooling, research and national and international reporting on ICT skills and knowledge;
  • Evaluates the ICT measures currently in use in the context of the National Goals for Schooling in the 21st Century;
  • Proposes a strategy for monitoring the profile of ICT skills and knowledge of school students;
  • Identifies further work required to establish an operational monitoring programme;
  • Explores the costs and benefits to school systems of introducing a national monitoring programme.

The NEPMT proposed the definition of ICT as technologies used for accessing, gathering, manipulation and presentation or communication of information. (July 2001).

In July 2001, MCEETYA established a new national taskforce, the Performance Measurement and Reporting Taskforce (PMRT) to continue the work of the NEPMT and further investigate key recommendations of the NEPMT report. The most recent achievements of the PMRT, which were recently endorsed by MCEETYA in July 2002, include the:

  • Development of a Measurement Framework for National Key Performance Measures as the basis for reporting on progress toward the achievement of the National Goals for Schooling.
  • Development of an Information Framework for the National Report on Schooling in Australia (2002). This document is published annually in hard copy and electronically (http://www.curriculum.edu.au/anr) and represents the key public accountability mechanism for reporting on the performance of Australia’s schools and school systems.

In relation to ICT, MCEETYA endorsed the recommendations of the PMRT to conduct the national monitoring of ICT skills and knowledge of students in Year 6 (aged 10-12 years) and Year 10 (aged 14-16 years) by means of three-yearly sample assessments, commencing in 2005.


3. Scope Of ICT Use In Education

a. NEPMT evaluation of ICT usage in Australian schools

The NEPMT report, Monitoring Progress towards the National Goals for Schooling: Information and Communication Technology (ICT) Skills and Knowledge, released the findings concerning the scope of ICT use in Australian schools in 2000:

  • 71% of schools had a student-computer ratio of 15:1 or less and this ratio is decreasing each year.
  • 37% of the computers in schools were in laboratories and 31% in classrooms.
  • Laptop computers comprised 16% of all computers used for educational purposes in schools - most of these in the non-government school sector.
  • Secondary schools generally had lower student-computer ratios than primary schools.
  • Secondary schools were more likely to place computers in laboratory settings and offer specific ICT courses, rather than integrating their use in classrooms across the curriculum.
  • Most computers used for educational purposes in Australian schools ran at 100MHz or faster and students had access to printers, modems, scanners, file servers and digital cameras.
  • Common applications in schools included integrated packages, reference CDs, educational games and virus protection.

It is difficult to make an accurate estimate of current levels of student participation in the ICT learning area in Australian schools, because a great deal of the learning takes place in integrated settings. In addition, some Australian students participate in studies that are specifically directed to the ICT learning area and there is evidence to suggest that the degree of such participation is increasing.

b. Longitudinal Surveys of Australian Youth

This research programme, jointly managed by the Australian Council for Educational Research (ACER) and the Commonwealth Department of Education, Science and Training, looks at subject choice in the final year of secondary school (Year 12 students aged 16-17 years) using a national, stratified sample of students who were in Year 9 in 1995 (aged 13-15 years).

One aspect of this study looks at enrolments in the eight key learning areas and at Computer Studies and related subjects within the Technology learning area. The overall growth in enrolments can be summarised by the findings that:

“The growth area from 1993 appears to be in the Technology Key Learning Area, accounting for almost one-fifth of enrolments (18 per cent) compared to just over one-tenth (11 per cent) of students in 1993……….Computer Studies was the subject in this area most frequently chosen by students”


c. PISA: An International Study of Student Outcomes

The OECD’s Programme for International Student Assessment (PISA) aims to measure how well students nearing the end of their compulsory schooling are prepared for adult life. The assessment is forward-looking, focusing on students’ ability to meet real-life challenges, rather than testing whether they have mastered a particular curriculum.

The first PISA survey was carried out in 2000 in 32 countries, with about 265,000 students doing the assessments. In Australia 231 schools participated in PISA, with over 5,000 students doing the assessment. Some of the key findings of the survey are:

  • Nearly 85% of Australian students have access to computers at home almost every day compared to 63% of their counterparts in other OECD countries.
  • At the same time, 9% of Australian students who never have access to computers at home, compared with 23% of the OECD population.
  • 43% of Australia’s students use a computer almost every day at home compared to the OECD average of 38%.
  • Just over 31% of Australian students access the Internet almost every day, while a further 32% access it a few times a week.

d. Australian Bureau of Statistics Surveys on Student Access to Computers and the Internet at Home

The Australian Bureau of Statistics’ publication, Household Use of Information Technology (May, 2001) conducted four quarterly surveys in 2000, which presented a quite different picture of computers and access to the Internet in Australian homes from the OECD’s PISA study. It revealed:

  • The proportion of Australian households with access to a computer at home has increased steadily from 45% in 1998, to 48% in 1999 and 53% in 2000.
  • The increase in the number of households with access to the Internet continues to rise strongly. In 2000, 33% of Australian households had access to the Internet at home, up from 22% in 1999 and 16% in 1998.
  • The growth in home Internet access is much higher than the growth in home computer access. For households with a home computer, the proportion which also had home Internet access was 63% in 2000, up from 47% in 1999 and 37% in 1998. The proportion was expected to rise to 77% in 2001.

4. Manner Of Introduction Of ICT In Schools

Development of on-line curriculum and resources Australia’s national action plan for schools, Learning in an Online World, includes as one of its key action areas: “Access to and application of online resources and services that support continuous improvement in curriculum practice, in classroom and distance settings, and in school administration.”

The plan's framework for action set four goals:

  1. A viable market will be established for the generation of quality online curriculum content for Australian school education.
  2. All students will have access to quality digital education materials that support Australian school curricula and that optimise opportunities provided by new technologies for learning.
  3. EdNA Online will support access by educators and students to high quality public domain online resources and services relevant to Australian curricula. EdNA Commercial will provide access to relevant commercial online products and services.
  4. Schools and schooling systems will provide education services using efficient and effective online business practices.

The following section of this paper examines progress towards the achievement of these goals.

National Online Content Initiatives (Le@rning Federation: Schools Online Curriculum www.thelearningfederation@edu.au) The Commonwealth, States and Territories are jointly funding this initiative. Preliminary work has accomplished:

  • Developing standards for interoperability, intellectual property and educational soundness including accessibility;
  • Devising quality assurance processes;
  • Agreeing on curriculum priority areas for development of content during the years 2001 – 2006, including:

-Science P-6 & 9/10 (2001 – 2003)
-Maths/Numeracy P-9 (2001 – 2003)
-LOTE Chinese, Indonesian, Japanese (2001 – 2006)
-Australian Studies P-10 (2002 – 2006)
-Innovation, Creativity & Enterprise P-10 (2002 – 2006)

The 2001-2006 Schools Online Curriculum Content Initiative will:

  • Develop the information exchange system;
  • Scope and develop online curriculum content;
  • Continually refine the standards and the content development processes;
  • Develop an Australian education market place.

EdNA Online (http://www.edna.edu.au/) EdNA Online developed a gateway to digital resources and services that supports access by K-12 educators and students to high quality public domain online resources and services relevant to Australian curricula. This was achieved by building quality collections of material for education and training, including:

  • 15,400 core quality online resources items and over 350,000 items linked to the core collection;
  • Improved quality of metadata for customer focussed resource discovery;
  • Online resources aggregated through a distributed management system;
  • Conferlinks, a collation of resources supporting themes at conferences;
  • InfoLinks – a collation of resources supporting themes in publications.

 

Curriculum Corporation (www.curriculum.edu.au) The Ministerial Company, Curriculum Corporation, is a major Australian innovator in the development of ICT based resources for the K-12 sector. Most work has been undertaken within government-funded projects, although a number of private sector relationships have also played important roles. Work completed covered a range of forms of digital delivery of resources, online support for teachers and the conduct of market research and development of underpinning technology to support more flexible and effective resource production.

State and Territory Initiatives to Support Online Learning. As well as the national initiatives described above, each of the States and Territories has made considerable progress in the development and use of online materials. Table 1 presents a summary of some of these developments at the end of 2001.


Table 1: State and Territory Initiatives to Support Online Learning

http://www.asptrial.e2c.wa.edu.au/about_e2c/about_e2c.html5. Teacher Professional Development

New technologies are already being used extensively by Australia’s teachers. In a report commissioned by the Curriculum Corporation, researchers investigated how Australian primary and secondary school teachers were identifying, storing, using, reusing and sharing online and offline resources and how they anticipated the new media would impact on teaching and learning.

The study found that 73% of the teachers surveyed described themselves as quite confident at using a computer or having advanced or high level skills and only 2% described themselves as beginners. 83% of the teachers indicated that they had been using computer-based curriculum resources with their students for a period of 1-10 years and 42% had been using them in their classrooms for longer than five years.

Quality Teacher Programme (QTP) In the 1999 Federal Budget, the Government announced it would provide $77.7 million over three years to States and Territories to strengthen the skills of the teaching profession under what is known as the Quality Teacher Programme (QTP). Information technology is one of six priority areas of the QTP. Reports from States and Territories in August 2001 indicated that about 25% of participants in the QTP had undertaken professional development in information technology. This is one of the largest areas of participation across the six priority areas.

Teacher Professional Development Models for the Integration of ICT The first phase of this Commonwealth funded project was completed in 2001. It involved a detailed examination of existing models of pre-service education and in-service professional development, both in Australia and overseas. Planning also began on the second phase, focusing on developing effective collaborative mechanisms to facilitate collegiality and the sharing of information through the use of online networks. The outcomes of this project will inform future government decisions about the development and resourcing of professional development programmes in ICT.

ICT Competency Standards for Teachers Work commenced on this project in 2001 to examine ways in which different Australian school systems are using criteria, standards and benchmarks for beginning and existing teachers to underpin effective use of ICT in curriculum practice. The ICT requirements included in teacher education courses were also reviewed. The aim is to develop a framework for describing teacher competency standards that can be used to inform the work of teacher education faculties and education authorities.

6. Connectivity – IT Systems, Software And Hardware

One of the key areas identified in the national action plan for Australian schools, Learning in an Online World, is that of infrastructure where the goal is to provide:

“Access to an advanced information and communication technology infrastructure that supports good teaching and learning and delivers efficiencies in business practice.”

Bandwidth The national action plan for Australian schools, Learning in an Online World, identifies a need to ensure bandwidth for school education that is:

  • Commensurate with the current and emerging needs of schools;
  • Accessible from all parts of Australia, both urban and rural;
  • Available at a cost that enables schools to participate in and contribute to the information economy.

7. Measuring Student Outcomes

To measure the impact of ICT on student performance, MCEETYA endorsed a Measurement Framework for National Key Performance Measures as the basis for reporting on progress toward the achievement of the National Goals for Schooling. It will conduct national monitoring of the ICT skills and knowledge of students in Year 6 (aged 10-12 years) and Year 10 (aged 14-16 years), by means of three-yearly sample assessments, commencing in 2005.

Key Performance Measure (KPM) is defined, in the Australian context of national reporting, as that which “quantifies a dimension of student participation or achievement and enables progress to be monitored against National Goals”. The principles underpinning the development and use of KPMs are:

  1. Student outcomes information is the focus of the reporting agenda.
  2. Performance measures should take account of State and Territory curriculum and assessment frameworks.
  3. Assessment techniques should be innovative and model good assessment practice and wherever possible, assessment materials developed for national sample assessments should be available for use by systems and schools.
  4. Collection and use of data for national purposes will in all respects conform to the guidelines provided in the report Data Principles and Protocols agreed by the PMRT.
  5. Access to data collections will be available to interested parties subject to privacy and confidentiality provision (as specified in Data Principles and Protocols).
  6. Where performance in different years in a particular domain is the focus of measurement, a single domain scale should underpin the measurement of student achievement.
  7. The key performance measures should enable the range of student achievement to be reported.

As a result of these Ministerial decisions, the PMRT is addressing the challenge of developing key performance measures for reporting on student ICT outcomes on a nationally comparable basis. It is anticipated that this work will involve:

  • Defining content domain
  • Developing a scale of competence
  • Developing assessment methods that include a mix of tests on paper and on-line and teacher performance assessment tasks.

To progress this work, some of the challenges that lie ahead include:

  • Reaching agreement that the focus should be on student skills and knowledge outcomes data and not access and attitudes, at least in the initial trial stage;
  • Defining the domain to be assessed in Year 6 and Year 10;
  • Defining standards, based on the achievement scale, in terms of what students typically know and can do;
  • Trialling a range of assessment instruments, including online delivery;
    Conducting a trial in a sample of schools across states and territories and across the different school sectors and including schools known to experience variations in access to ICT.

8. Issues And Challenges And How They Are To Be Addressed

Evidence of Uneven Distribution of Access The Foundation for Young Australians, in partnership with the International Youth Foundation and the Lucent Technologies Foundation, investigated the issue of public access to the Internet for disadvantaged groups of young people in Australia. Their findings suggest that there is significant inequity in the way access is distributed across the population.