Linda Tyler is Group Executive Director for New Product Development in the Higher Education Division at ETS and has worked on many programs for the testing organization, from admissions tests to teacher licensure exams and learning products. Dr. Tyler has a B.A. and M.A. degree from the University of California, Riverside, and an MFA and Ph.D. in Musicology from Princeton University.

Technology is an ever-growing part of students’ lives.

For most young adults, the future will likely bring an even greater breadth of complex information and communication technologies, including those that have not yet been imagined. “Students will spend their adult lives in a multi-tasking, multifaceted, technology-driven, diverse, vibrant world—and they must arrive equipped to do so.”1

In the world of higher education, virtually every aspect of scholarship has been influenced by technology. Students are conducting research through the Web, drawing from academic journals, newspaper articles and speech transcripts. Some are receiving assignments online and e-mailing completed projects to their professors. Many are using spreadsheets, graph plotters, presentation programs and multimedia tools on a regular basis. The proliferation of distance education and e-learning has altered the traditional definition of “classroom.”2 A measure of success today is how well one can evaluate, manage and communicate all forms of information within a technological environment. These are Information and Communication Technology (ICT) skills.

In its 2005 report, Assessment of 21st Century Skills: The Current Landscape (Pre-publication draft), the Partnership for 21st Century Skills describes the need for mastering ICT skills:

In order to provide both flexibility and security in an era characterized by constant change, 21st century students need ‘knowing how to learn’ skills that enable them to acquire new knowledge and skills, connect new information to existing knowledge, analyze, develop habits of learning, and work with others to use information . . . And as technology increasingly becomes the medium for communication and information sharing, students need to be capable of harnessing technology to perform learning skills, such as communicating effectively with presentation software or juggling personal responsibilities with a personal digital assistant . . . ”3

An individual who lacks ICT skills has fewer opportunities for personal advancement, and a society that lacks an ICT-literate workforce will not compete in the global economy. Students who lack ICT skills cannot fully benefit from learning opportunities in the classroom or beyond it.4 Students who are less proficient in ICT may be unable to evaluate the validity of information they find through modern search engines, or draw meaning from it. They may not be able to compare information from numerous sources or communicate their findings effectively, ethically, and legally. The lost opportunity to acquire these skills in college may follow them throughout their careers. Whereas it used to be the case that only certain occupations required skills in technology, the U.S. Dept of Labor projects that eight of the ten fastest growing occupations in this country require “technological fluency.”5

Educators helped define and create a cognitive assessment of ICT literacy skills.

Despite widespread consensus about the need for ICT literacy among college students, there is little information available to tell us the dimensions of the need or what might be done to address it. Higher education institutions are just starting to identify ICT as a core competency (as opposed to “information literacy” or “technology literacy”). A few institutions, such as California State University and Purdue University, are taking that a step further—classroom faculty and library professionals are working together to integrate technology into the curriculum and create discipline-specific assignments that require critical use of information resources.9 Yet there is still a need to measure the effectiveness of these efforts and to evaluate whether students have obtained the ICT skills they need to be successful in an information-rich, technology-based society.

To address this critical need, the Educational Testing Service (ETS) in 2001 convened a multinational group of experts from education, government, nongovernmental organizations, labor, and the private sector. The panel defined what it means to be ICT literate and determined that there is a dire need for an assessment of a person’s ability to think critically and communicate effectively in a technological environment. The group released “Digital Transformation: A Framework for ICT Literacy,” an analysis of what was known and not known about ICT literacy, including recommendations for research and policy.10

Building upon that work, ETS joined forces with seven leading college and university systems—which together enroll about 25 percent of the nation’s college students—to create the National Higher Education Information and Communication Technology (ICT) Initiative. Together, they identified seven ICT objectives that needed to be measured in a higher education environment.  They also identified which types of information would help faculty and administrators gauge the effectiveness of current teaching strategies and curricula, identify best practices, and initiate better approaches. The seven objectives are: define, access, manage, integrate, evaluate, create, and communicate. The charter colleges worked with ETS to conceive, design, and build the first test of a person’s ability to use ICT to think critically and solve problems.

Table 1
Components of ICT Literacy*

Earlier this year, ETS launched the ICT Literacy Assessment. It is an interactive assessment that uses simulated software applications and simulations of authentic technology environments, such as an Internet-like database.11 Students are asked to complete several multi-step tasks that integrate the seven proficiencies and reflect real-life situations that are likely to occur in college, at work, and in their personal use of technology. One task might ask students to read three e-mails (each with an embedded link), analyze the authoritativeness of each source and compose an e-mail message summarizing their research findings. Other tasks might ask students to enter terms into a search engine to find information on the Internet, cite sources properly or answer questions about a graph they created. The scenarios effectively model how technology can be integrated across a variety of disciplines. Students are evaluated based upon their ability to effectively and efficiently perform cognitive and information management tasks in an ethical and legal way using digital technologies, communication tools, and networks.12

The first test administration, which ended in April 2005, yielded aggregate scores for participating institutions. The test body varied from campus to campus—for example, some institutions tested a random sample of rising juniors while others tested students enrolled in a particular course—and institutions will use results for a number of different purposes. Portland State University intends to use results to determine whether students who have been at PSU since their freshman year will perform better than students who transferred in. In the future, PSU may also use test results as a pre- and post-program assessment for selected university programs.13

Institutions can use the aggregated data in a variety of ways:

• Design courses to close the gap between the current state and basic proficiency
• Benchmark ICT proficiency and gauge students’ need for training
• Help guide curricula innovations and evaluate curricula changes
• Provide accreditation evidence
• Provide support for institutional ICT literacy initiatives

Based upon the first administration, ETS is making several adjustments to the assessment, including reducing the test time from two hours to 75 minutes. Two levels of the revised assessment will be released in January 2006, and will produce detailed score results for each test-taker and will give institutions the ability to analyze the data at an aggregate level. The Core Level test will be appropriate for students transitioning to college, and the Advanced Level test will be appropriate for rising juniors transitioning to upper-level coursework. The individual student results can be used in a number of ways:

• Satisfaction of an information or technology literacy requirement
• End-of-course assessment for an ICT-related course
• Placement of students into or out of course(s)
• Pre-requisite for certain courses
• Identification of students at risk
• Placement of students into distance learning program
• Determination of a student’s workforce readiness

ETS’s ICT Literacy Assessment is built upon the belief that a person’s cognitive skills—how they think, solve problems, and learn—have a bigger impact on that person’s ability to function in our technology-rich society than knowledge of any specific software package or hardware platform. Society needs citizens who not only know how to obtain information, but who can analyze and evaluate what they learn in order to develop an informed opinion.14

***The author retains all rights to this essay.  However, FYA-List subscribers may distribute the essay for non-commercial purposes.  ***