Since the mid 1970s, Steve Ehrmann's work has revolved two overlapping themes: (1) how best to use technology to improve education an d (2) helping educators use evaluative data to guide their progress in those directions. The computer thread began with a historical study of one of the earliest uses of computing in undergraduate education, dating to about 1960. The evaluation thread began with his work as Director of Educational Research and Assistance at The Evergreen State College.

The two threads came together when, from the late 1970s to the mid 1990s, Dr. Ehrmann served as a program officer, first with the Fund for the Improvement of Postsecondary Education (FIPSE) and then with the Annenberg/CPB Project. In 1992, while with Annenberg/CPB, he started the Flashlight Program for the Evaluation and Improvement of Educational Uses of Technology.

In 1998, with Steven W. Gilbert, he founded The TLT Group, a non-profit that helps institutions and educators improve teaching and learning with technology. The Flashlight Program is now part of The TLT Group. For more on Dr. Ehrmann's work, and The TLT Group, see http://www.tltgroup.org/.

General education, as many people define it, is what all students are supposed to learn from their college educations. Unfortunately, in many institutions, assessment tools for improving general education are quite limited. Only at a few institutions, do systematic assessments let faculty and students know whether those students are making satisfactory progress toward desired levels of competence as they move from course to course. It's even more rare that data gathering provides any clues to how the academic program might be improved so that more students can make better progress.

In recent years, two technology-supported approaches have widened the faculty's ability to see what's going on, within and across courses: electronic portfolios and online surveys.

1. Electronic Portfolios
   
   A “portfolio” is a thoughtfully organized collection of student work, usually including work other than, or in addition to, traditional academic papers. For example, web projects can be stored in portfolios, as can video recordings of student performances (oral presentations, participation in teams, dances). Portfolios often include student reflections about how the project provides evidence of their developing skills. These reflective statements are one way in which portfolio use is intended to deepen student learning.
   
   “Electronic portfolios” store those projects, or recordings of them, plus reflections and feedback, on computers so these records can easily be accessed online. Current software enables students to organize the work in several different ways: one “view” organized for an individual course, another view organizing the content to show progress toward goals of general education, another showing progress in the major, and yet another that might be used for employment or graduate school applications. The work can be used over a period of time by the student, by faculty, and, at some institutions, by people outside the institution (e.g., potential employers). This ability to revisit a project long after the project is completed is one of many distinctive values of electronic portfolios.
   
   Today there are many types and uses of electronic portfolios. Let's focus on just one of those uses: portfolios organized to show student progress over the years toward general education goals such as “communications skills” or “skills of inquiry.” In such portfolios, students store work that represents major stages of progress toward each competence, along with their own comments on how that work represents the skill. Also stored in the portfolio is feedback from faculty (and sometimes from others) about whether and how the work shows the required level of competence.
   
   In order for portfolio analysis to guide the improvement of general education, faculty members must define what the goals and levels of progress are. Faculty then create rubrics to help students and, later, assess whether that level of progress has been achieved. As student work is submitted, faculty use those rubrics to judge student progress by studying the students' portfolios. In other words, faculty members finally have a way to look at education in chunks larger than a single course. And if faculty can see general education as a whole, and discuss what they see, they have a better chance of intentionally improving that whole process of general education.
   
   Provost Doris Helms of Clemson University commented in an interview with the author that this use of electronic portfolios freed her institution “to think about general education as something other than a smorgasbord of courses.” At Clemson, for example, writing done by admitted students during the summer before the first year is the initial entry into student portfolios.
   
   Using portfolios to record student progress toward general education goals enables everyone to analyze patterns of learning that extend beyond single courses:
   
   
   • Students: Students are asked to examine their own projects in light of skills they need to master. This reflection is designed to help them deepen learning from work they have already done, while simultaneously helping the student plan and accelerate future learning.
     
     
   • Individual faculty members: Faculty members can look at student portfolios before the term begins in order to help decide how to teach their courses. The portfolio provides vivid evidence of the strengths and needs of students, helping the faculty member fine tune plans for the coming term.
     
     
   • The institution: When faculty members work in teams to examine student progress, they can use information to reconsider the organization of the curriculum. Provost Helms commented, “We'll use this as research-where are students learning what they're learning? For example, what are they learning while outside the classroom, in jobs, at home, and in extra-curricular experiences? What kinds of learning should we foster, more intentionally, outside the course?”
     
     
2. Online Surveys used for Course Research and Program Evaluation
   
   While the single best source of guidance for improving learning is close study of student work (projects, online discussion, etc.), those insights can be complemented and deepened through feedback from students. To this end, the TLT Group's Flashlight Program has developed banks and templates of survey item and an online system for creating, delivering, and analyzing such surveys. Collectively, these are known as Flashlight Online.
   
   Flashlight Online enables users to write their own questions or to select questions from almost 500 validated items in the Flashlight Current Student Inventory. Most users are faculty who gather information from their own students. Flashlight Online also allows users to share surveys or data. It can also be used for multi-institution benchmarking studies in which faculty use the same, or similar, studies to contribute to a common pool of data so that they and their institutions can compare their findings with the rest of the pool (e.g., “Evaluating Educational Uses of the Web In Nursing.”)
   
   A new version of Flashlight Online is now being developed with the support of the Fund for the Improvement of Postsecondary Education (FIPSE) as part of the BeTA (Better Teaching through Assessment) Project. BeTA will strengthen the use of student feedback about courses and faculty in two ways:
   
   a) Designing feedback: Every institution is different. For faculty, administrators and students planning the content of their institution's course feedback surveys, BeTA project workshops and materials will help them decide on lines of inquiry; which data are most important for improving courses and for making personnel decisions? BeTA surveys will usually include a mix of sources for their questions: some questions common to all courses, some designed for specific types of courses, some from specific institutional programs or colleges, and some authored by the faculty member for his or her own students.
   
   b) System for Designing and Administering Online Surveys with Single or Multiple Authors: The new survey system being designed for BeTA (Flashlight Online 2.0) will aid multiple authors in developing a multi-user survey. As an added feature, each author can keep some of the resulting data from other authors of the same survey, if need be. For example, a faculty member could add a question to the course feedback survey that asks her students, and only her students, to provide feedback on her use of PowerPoint without needing to worry whether that data might hurt her chances for promotion. Meanwhile, on the same survey might be questions from the writing program about whether feedback on student writing is helping students become more skilled writers and/or helping them with the content of the written work. Other questions might be added by the department. BeTA is designed to encourage people and programs to ask the kinds of risk-taking questions needed for real improvement.
   
   c) BeTA surveys will typically be delivered online. Because low response rates to online surveys are a widespread concern, BeTA is developing strategies to increase response rates. One reason for low response rates: as far as the typical respondent is concerned, spending time and thought on a typical course feedback survey produces no visible result. So BeTA will recommend that individual faculty, departments and the institution each report to students frequently on what they each are doing as a result of student feedback.
   
   
3. Concluding Thoughts
   Faculty at a growing number of institutions now can use two quite different sources of data to help them teach better

• Portfolios of current and past projects by students, organized to show student strengths and needs as they move toward educational goals - the institution's, the major's, and their own.
• o Surveys to gather student feedback about the processes of education.
  Each of these two sources of data can be used by individual faculty to improve teaching. The same tools can be used by faculty, working together, to observe and improve the larger patterns of teaching and learning that comprise general education.
  
  The author retains all rights to this essay. However, FYA-List subscribers may distribute the essay for non-commercial purposes.