Dr. Merilee Bresciani has been in higher education administration and faculty positions for over 17 years. In those positions, she has conducted enrollment management research, quantitative and qualitative institutional research, course-embedded assessment, and academic and administrative program assessment. As Assistant Vice President for Institutional Assessment at Texas A&M University, Dr. Bresciani enjoys assisting several units and departments with the development of their assessment plans, the identification and development of assessment tools and methods, and the use of their data for continuous improvement of student learning and development. Dr. Bresciani is currently authoring two books on program assessment and general education assessment, has developed and delivered several courses on assessment of student learning, and serves on the editorial board of the NASPA Journal.
Penny Addison holds a BA in Education from the University of NC-Chapel Hill and a MS in Speech Communication from Texas Christian University. Currently she is working on her PhD in Communication at Texas A&M. Her research interests include emotional labor, conflict, and small groups in the workplace. Penny has been collaborating with Dr. Bresciani on several assessment research projects.
University faculty and administrators across the country have expressed a desire for students to become more engaged community members and have applied a variety of programs to encourage student engagement. Establishing these programs is a crucial step in the process, however it is as important ñ if not more so ñ to evaluate and improve these initiatives once established While evaluation and improvement initiatives can be challenging, they may be more easily accomplished with the aid of technology. This article will outline a technology-enabled method to track student engagement in the college community.
Student Engagement and University Community
Student engagement and its relationship to academic success and persistence has been well articulated over the years (Astin, 1993; Flowers, 2004; Hu & Kuh, 2003; Koljatic & Kuh, 2001; Pascarella & Terenzini, 1991; Zhao & Kuh, 2004). Chickering and Gamsonís (1987) principles of good practice for undergraduate education can be considered indicators of student engagement. Subsequent research has examined how these and other indicators of student engagement enhance the experiences of college students.
For example, Hu and Kuh (2003) found that student engagement in college activities is the key factor in student learning and that ìstudent engagement in educationally purposeful activities has a strong positive effect on student self-reported gainsî in learning (p. 197). Student engagement research also indicates that general cognitive gains have been positively correlated with studentsí reports of how often they participate in educationally purposeful activities (Ewell and Jones, 1996). In addition, students who were more involved and had positive perceptions of the college environment reported greater gains in learning and intellectual development (Pike & Killian, 2001; Pike, Kuh, & Gonyea, 2003).
One of the most widely used sources of information on student engagement is the National Survey of Student Engagement (NSSE) [note: a related survey for two-year schools is called the Community College Survey of Student Engagement (CCSSE)]. NSSE was specifically designed to gather information on student engagement in order to ascertain whether an institution is providing quality undergraduate education (Kuh, 2003). However, NSSE is only one measure of student engagement, and it is necessarily broad, focusing on the whole of the college experience rather than the individual classes, programs, and initiatives that compose that experience. Kuh (2001) indicates the need for additional measures of student engagement when he says, ìthe greatest impact and utility of NSSE data will come when they are integrated with other institutional data about the student experienceî (p. 15). Thus, while the NSSE can measure general levels of engagement in students, it is up to the institution to use additional measures that focus on the specific needs of the institution.
Tracking Student Engagement
Demonstrating the contributions that various programs make to student learning is a challenge, and it is often difficult to track the level or degree to which students are engaged in their activities. However, the use of web-based tracking systems, such as the Student Tracking System (STS), within each program may facilitate efficient collection, storage, and analysis of student engagement data. Systems such as the STS can track the frequency with which students attend educational events, their responses to self evaluations of performance, and outcomes achievement. When combined with data from other sources (e.g., GPA, retention), these data allow researchers to identify the extent to which student engagement contributes to student success.
Prior to each event (also called a point of program access or PPA), the sponsoring department enters basic event information into a computer database. The department also has the option to enter evaluative criteria and instruments, and desired learning outcomes. After the initial event data is entered, the system is ready to catalog studentsí attendance at each event with the swipe of the student ID card.
After an event has been completed, students complete a self-evaluation in which they indicate their personal level of engagement in the educational activity. Subsequently, a mean engagement score for each student is entered into the STS, as well as an administratorís observation of the studentís level of participation. If learning outcomes are evaluated, that score is also entered into the STS (e.g., the studentís ability to identify ethical dilemmas in decision-making). These scores can later be used to make improvements, additions, or omissions to programs administered in the future.
The Student Tracking System - in Practice
The use of a Student Tracking System could, for example, be applied to a student leadership development program. In this example, a desired outcome is for students to be able to identify ethical dilemmas in decision-making. Participants are given the option to attend one of several voluntary workshops, each with specific means to teach students about how to identify ethical dilemmas in decision-making. Tracking is essential in this situation, as the delivery of this program is non-systematic and optional.
At each workshop, the student participant would sign in by swiping their student ID card, or by having their ID card scanned by an administrator. STS records the studentís information and their first PPA for this program. At this and each subsequent PPA the student is provided an on-line or paper self-evaluation which asks them to reflect on and record their level of participation in the activity; the desired learning outcomes for the activity can also be included on the evaluation.
At the end of the leadership development series, or any series of co-curricular or curricular activities and lessons, data collected in the STS can be merged with data from the student information system. A regression analysis can then be run to determine the correlation between the number of activities a student attends and where they stand on academic success indicators, such as their semester grade point average. This same analysis could be run with level of participation and level of learning for a particular event or program outcome as the dependent variables. A similar type of analysis could be run using the level of student learning for the event or program outcome as the independent variable.
Summary
Using ID card scanning technology and storage of data in a secure, web-enabled database can provide multiple users with an opportunity to easily gather information about what events students are attending, the level in which they are engaging in those events, and the level of learning that is occurring. Such data collection allows for more refined analysis of whether program and learning outcomes are being met, in a way that allows faculty and administrators to improve programs or better understand - - based on level of student engagement - - why program outcomes are or are not being met. In addition, such data collection and its merger with student bio-demographic and academic indicators can provide students, faculty, and administrators with data to better inform discussions about appropriate levels of participation by types of events in order to improve student engagement in the college community.
****
The authors retain all rights to this essay. However, FYA-List subscribers may distribute the essay for non-commercial purposes.
Requested citation for redistribution or reference:
Bresciani, M.J. & Addison, P. (2006). The Student Tracking System: A Technology-Enhanced Measure of Student Engagement. Essay for the First-Year Assessment Listserv. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. (http://www.sc.edu/fye/resources/assessment/essays/Bresciani-5.25.06.html)
*****
References
Astin, A. (1993). What Matters in College? Four Critical Years Revisited. Jossey-Bass, San Francisco.
Bresciani, M.J., Zelna, C.L., and Anderson, J.A. (2004). Techniques for Assessing Student Learning and Development in Academic and Student Support Services. Washington D.C.:NASPA.
Chickering, A. W., & Gamson, Z. F. (1987). Seven Principles for Good Practice in Undergraduate Education, AAHE Bulletin, 39(7), 3-7.
Ewell, P. T., & Jones, D. P. (1996). Indicators of ìGood Practiceî in Undergraduate Education: A Handbook for Development and Implementation. Boulder, CO: National Center for Higher Education Management Systems.
Flowers, L. A. (2004). Examining the Effects of Student Involvement on African American College Student Development. Journal of College Student Development, 45, 633-654.
Hayek, J., & Kuh, G. (2002). Insights into Effective Educational Practices. EDUCAUSE Quarterly, 25(1), 60-61.
Hu, S., & Kuh, G. D. (2003). Maximizing What Student Get Out of College: Testing a Learning Productivity Model. Journal of College Student Development, 44, 185-203.
Koljatc, M., & Kuh, G. D. (2001). A Longitudinal Assessment of College Student Engagement in Good Practices in Undergraduate Education. Higher Education, 42, 351-371.
Kuh, G. D. (2001). Assessing What Really Matters to Student Learning: Inside the National Survey of Student Engagement. Change, 33(3), 10-17, 66.
Kuh, G. D. (2003). What Weíre Learning About Engagement From NSSE. Change, 35, 24-32.
National Survey of Student Engagement (2004). Student Engagement: Pathways to Collegiate Success. 2004 Annual Survey Results. Bloomington, IN: Indiana University Center of Postsecondary Research.
Pascarella, E., & Terenzini, P. (1991). How College Affects Students: Findings and Insights from Twenty Years of Research. Jossey-Bass, San Francisco.
Pike, G. R., & Killman, T. S. (2001). Reported Gains in Student Learning: Do Academic Disciplines Make a Difference? Research in Higher Education, 42, 429-454.
Pike, G. R., Kuh, G. D., & Gonyea, R. M. (2003). The Relationship Between Institutional Mission and Studentsí Involvement and Educational Outcomes. Research in Higher Education, 44, 241-261.
Zhao, C., & Kuh, G. D. (2004). Adding Value: Learning Communities and Student Engagement. Research in Higher Education, 45, 115-138.
|
