Standards for evaluation

"If you don't know where you are going, then it doesn't matter which road you take"
(Cheshire Cat, Alice in Wonderland)

Indicators give us means to assess, but what exactly are we assessing? What are we aiming for in establishing ICTs in education, where exactly are we going? To answer this question, certain sets of standards and benchmarks are clearly needed, to define what is expected of ICTs and their effect on teaching and learning. Some countries have been making efforts in this direction. Whilst in Australia, this is restricted to national technical standards and Education Queensland Minimum Standards for Teachers in Learning Technology, America has identified Standards for Students, Connecting Curriculum and Technology, Educational Support Standards and Standards for Student Assessment and Evaluation of Technology, as discussed fully here.

1) National Educational Technology Standards - http://cnets.iste.org/
In an attempt to enable stakeholders in PreK-12 education to develop
national standards for educational uses of technology that facilitate school improvement in the United States, the NETS Project aims to define standards for students in the areas of integrating curriculum technology, technology support and standards for student assessment and evaluation of technology use. These standards to be developed or refined by NETs include standards for students, standards for teachers and standards for school administrators.

Standards for Students
The technology foundation standards for students are divided into six broad categories. Standards within each category are to be introduced, reinforced, and mastered by students. Teachers can use these standards and profiles as guidelines for planning technology-based activities in which students achieve success in learning, communication, and life skills.

• Technology Foundation Standards for Students: students demonstrate a sound understanding of the basic operations and concepts
• Social, ethical, and human issues: students understand the ethical, cultural, and societal issues related to technology and develop positive attitudes toward technology uses that support lifelong learning, collaboration, personal pursuits and productivity
• Technology productivity tools: students use technology tools to enhance learning, increase productivity, and promote creativity
• Technology communications tools: students use telecommunications to collaborate, publish, and interact with peers, experts, and other audiences
• Technology research tools: students use technology to locate, evaluate, and collect information from a variety of sources and process data
• Technology problem-solving and decision-making tools: students use technology resources for solving problems and making informed decisions

Standards for Teachers

• Technology Operations and Concepts: teachers demonstrate a sound understanding of technology operations and concepts and demonstrate continual growth in technology knowledge and skills to stay abreast of current and emerging technologies.
• Planning and Designing Learning Environments and Experiences: teachers plan and design effective learning environments and experiences supported by technology, to support the diverse needs of learners
• Teaching, Learning and the Curriculum: Teachers implement curriculum plans that include methods and strategies for applying technology to maximize student learning
• Assessment and Evaluation: Teachers apply technology to facilitate a variety of effective assessment and evaluation strategies
• Productivity and Professional Practice: teachers use technology to enhance their productivity and professional practice
• Social, Ethical, Legal and Human Issues: teachers understand the social, ethical, legal and human issues surrounding the use of technology in PK-12 schools and apply those principles in practice

Standards for School Administrators

• Leadership and Vision: inspire a shared vision for comprehensive integration of technology and foster an environment and culture conducive to the realisation of that vision
• Learning and Teaching: ensure that curricular design, instructional strategies and learning environments integrate appropriate technologies to maximise learning and teaching
• Productivity and Professional Practice: apply technology to enhance their professional practice and to increase their own productivity and that of others
• Support Management and Operations: ensure the integration of technology to support productive systems for learning and administration
• Assessment and Evaluation: Use technology to plan and implement comprehensive systems of effective assessment and evaluation
• Social, legal and Ethical Issues: understand the social, legal and ethical issues related to technology and model responsible decision-making related to these issues

This is the first step of a long-term project aimed at describing the conditions needed to support the use of technology for learning, teaching and institutional management.

2) Technical Standards for Online Education and Training - Australia
Recognising a strategic involvement in a range of standards-related activities as being an essential component of effective use of online technologies, DETYA, the Commonwealth Department of Education, Training and Youth Affairs in Australia commissioned standards building work on behalf of EdNA, the Education Network Australia to provide a solid body of information and advice on which to base activities.

The scope of this project is to research and advise on issues related to technical standards relevant to information technology, online delivery, resource discovery and some aspects of information management in education and training. Standards are grouped into the following areas:

• Network Infrastructure: general network infrastructure, end-user terminals, security, authentication and content filtering
• Content Formats: storage and presentation of components of online content (text, graphics, audio and video files) plus programming languages and accessibility issues
• Resource Description and Resource Discovery: covers metadata, search interfaces and controlled vocabularies
• General Applications: General network applications - web publishing, e-mail, workgroups and video-conferencing
• Delivery Platforms and Content Packaging: software systems used to manage the delivery of online content to students and to manage communications functions as part of the learning process as well as data structures to support interoperability of educational content
• Administration and Management: student records and educational statistics

While this project is specifically concerned with Australian initiatives, the internet and the technical standards that underpin it are based within an international context. Indeed, in the valuable report Technical Standards for Online Education and Training includes considerable discussion of relevant international activities, projects and standards setting bodies.

3) Education Queensland - Minimum Standards for Teachers Learning Technology
Wisely recognising that improvement in student learning outcomes through the integration of computers in the curriculum is dependant on teachers’ confidence and competence in the area, Education Queensland is developing a set of standards that all teachers should meet.

Standards have been divided into four key areas:

• Information Technology skills - able to operate and use both hardware and software
• Curriculum Application, including classroom planning and management
• School Planning processes and procedures which promote continuity and coordination of learning technology activity at the whole school level, relating to the Management and Learning Technology (MALT) plan
• Student Centred Learning applying effective learning and teaching processes to learning technology

Teachers are expected to attain the Minimum Standards for Teachers - Learning Technology in all four areas. Some specific descriptors, however, are regarded as mandatory and teachers must achieve these:

School Planning

• A teacher is aware of and understands the learning, teaching and curriculum section of the school’s MALT plan
• A teacher is aware of learning technology professional development programs for staff

Student Centred Learning

• Understand the learner
  Identifies students’ prior experience with and attitudes to computers at home and at school
  Adapts computer based activities for students with special abilities, needs and interests
  Accommodates the learner as an individual and independent learner as well as a member of a group
• Understand the learning process
  Is aware of how computers can support all elements of the learning process: perception, processing and (re) presentation
  Provides a clear focus and purpose for computer based activities
• Supportive and challenging environment
  Creates a warm supportive atmosphere which fosters risk taking
  Uses open ended software and open ended tasks to promote problem solving
• Worthwhile partnerships
  Facilitates peer tutoring and cooperative learning processes
  Utilises support staff, teacher aides, parents and community members
  Social and cultural contexts
  Provides opportunities for learners to share their perceptions of technology in society

Furthermore, Education Queensland has developed a checklist of descriptors for teachers to use to assess their current level in each of the four key areas. Schools can gather data from staff to establish the current levels of teachers to inform the design and development of professional development programmes. The following process has been trialed in a number of schools and proved to be effective:

• brainstorm issues relating to current learning technology programs within the school;
• collect data on the two main areas -
• human resource audit (using minimum learning technology standards checklist);
• physical resource audit (hardware and software registers, location of resources, etc);
• collate data - establish a database;
• analyse data to inform future decision making about resourcing and professional development;
• design professional development programs for teachers
• survey teachers for preferred professional development modes
• implement professional development courses and training programs; and
• apply skills and strategies from professional development to classroom practice and school planning

Finally, models for the design of professional development, such as peer coaching, group training and observing other teachers are specified, as well as modes of delivery: online courses or visits to sites of best practice for example.

(For further details, please see http://education.qld.gov.au/curriculum/learning/technology/mst_int.html#introduction)

Clearly, countries are at different stages of development in ICT as a means to improving education, so to evaluate use should be undertaken using different sets of indicators applicable to the different levels or stages they have reached. Developing countries, such as Cambodia and Vietnam, for example, are still at very basic levels of implementation, focusing more on increasing connectivity and getting computers in schools. India is perhaps ahead of the game in this area, with numerous initiatives to connect all levels of society, from school pupils, to those living in remote rural areas to slum dwellers in cities, though focus is still firmly placed on delivering the goods, rather than assessing if they're working. Japan, Korea and the Philippines are similarly positioned. In America, South Africa, Europe and Australia, however, implementation is now fairly widespread, so attention can be turned to matters of efficiency and effectiveness in all areas of ICT for education, as has been shown here.

Furthermore, Mid-continent Research for Education and Learning (McREL) has been in the forefront of research, practice and evaluation related to standards-based education, collaborating with key organisations, and work with schools, districts, and states to improve their practices. Among the most notable of McREL's contributions to the field is a compendium of K-12 and Career Education standards, yet they also offer standards-related training and consulting services and the opportunity for schools, districts, and agencies to participate in the McREL Standards Consortium. In addition, McREL has authored a wide variety of publications and products related to standards, many of which can be downloaded from their site.