Any resource or discussion about how to grade online coursework by necessity provides more than just tips on grading, it requires instructors to address issues regarding test proctoring and evaluation of the usefulness and validity of online assessment types.
Traditional timed exams are possible using the tools available in Blackboard Tests. Blackboard Tests provides online testing with numerous question types and options to assess student knowledge as well as automated grading. More information and guidance on how to use this functionality can be found in Blackboard Help’s Create Tests and Surveys.
Remote proctoring tools such as Respondus Monitor and ProctorU use a student’s webcam and video analytics to increase academic integrity during online exams. For detailed information on Respondus Monitor and ProctorU, visit the Office of Distributed Learning’s Test Proctoring. Additional Respondus resources including free training webinars and resources for students can be found on the Respondus Monitor website.
However, it is important to note that proctored remote exams have several drawbacks:
- They are often even more stressful for students than in-person proctored exams, which can negatively impact student performance. Such systems add significantly to a student’s “extraneous cognitive load” due to the burden of worrying about eye movement, external physical variables, instructor perceptions, etc.
- They require substantial planning and setup on the part of both the instructor and the student, and can generate many “false positive” flags that must be reviewed by an instructor after the exam.
- Not all students have access to the appropriate technology to use such services, and therefore instructors will have to make accommodations for such students.
- Planning must include what to do if the proctoring service or internet service crashes during the exam.
- Students and faculty have significant privacy concerns about third-party recorded remote proctoring.
- Students have reported that they find remote proctoring systems relatively easy to circumvent, and online resources provide ways to bypass these systems.
(from: Rutgers University, Remote Exams and Assessments: Tips for Exams and Alternative Assessments)
If you choose to use these systems, you can help your students use them successfully by doing the following:
The Practice Test
The single most important thing you can do to help your students be successful in taking their test with Respondus Monitor is to give them a practice test several days before their first real test. This requires students to download the necessary software and test their computer hardware and internet connection in a low-stakes environment. If they encounter problems, they can work with the Office of Distributed Learning, DoIT, and/or Respondus support to clear up any issues before they take a real test. In addition, the practice test will familiarize the students with the environment and the procedures for taking the test, and that will lessen any anxiety students might have about the experience.
Give Them Tips and Guidance to Help Ensure the Process Runs More Smoothly
These can be included in the syllabus or provided as a resource handout. Indiana University provides some helpful tips in Administer tests and quizzes using Respondus Monitor with LockDown Browser.
- Avoid wearing baseball caps or hats that extend beyond the forehead.
- If using a notebook computer, place it on a firm surface like a desk or table, not your lap.
- If the webcam is built into the screen, avoid making screen adjustments after the exam starts. A common mistake is to push the screen back, resulting in only the top portion of the face being recorded.
- Don't lie down on a couch or bed while taking an exam. There is a greater chance you'll move out of the video frame or change your relative position to the webcam.
- Don't take an exam in a dark room. If the details of your face don't show clearly during the webcam check, the automated video analysis is more likely to flag you as missing.
- Avoid backlighting situations, such as sitting with your back to a window. The general rule is to have light in front of your face, not behind your head.
- Select a distraction-free environment for the exam. Televisions and other people in the room can draw your attention away from the screen. Other people that come into view of the webcam may also trigger flags by the automated system.
Students cheat on in-person exams, and cheating on a test isn’t the only way to violate academic integrity. Concerns about exam cheating during remote instruction are then at odds with concerns that monitoring cheating should not be professors’ primary focus in teaching, especially during a pandemic. While instructors can’t completely stop students from cheating in online exams, you can do a great deal to make it less tempting and less prevalent and foster academic integrity. In the article 7 Ways to Assess Students Online and Minimize Cheating, Flower Darby, an instructional designer and author, with James M. Lang, of Small Teaching Online: Applying Learning Science in Online Classes, recommends a compromise approach:
- Break up a big high-stakes exam into small weekly tests that equal the weight of the high-stakes exam. The weekly tests can be just as rigorous, but if students do poorly on one or two of them, they can still recover from those.
- Make exams open-book, open-note. Students are more inclined to cheat under stress, so reducing the pressure they feel can go a long way.
- Have students sign an honor statement both at the start and end of each test, to affirm that they are practicing academic integrity.
- Ask students to explain their problem-solving process. Students may search online for answers to similar problems, but it’s harder to find student-generated explanations of the steps they took to solve them.
- For small, writing-intensive courses, get to know each student’s writing style in low- or no-stakes tasks. Assess them using a complete/incomplete scale or simple rubric in Blackboard.
- Additional ideas are provided in Flower Darby's article, 7 Ways to Assess Students Online and Minimize Cheating.
Are you using exams simply to measure content coverage, or do you want to measure how well they are learning (effort and growth) or their ability to solve problems, create, and collaborate (inquiry-based learning)? What do you hope students will be able to do by the end of your course, and in what ways can they demonstrate what they know?
Success in the “real world” typically depends on what has been accomplished, daily and weekly, not by what has been memorized every 3-4 weeks. Your grading and assessments can reflect this, by grading students on their productivity and progress, as well as requiring artifacts that measure productivity. To measure such learning, instead of having a timed, proctored remote exam, consider using alternatives.
Below is a list of suggested alternatives to exams that can be aligned with many different subjects and goals (from Rutgers University, Remote Exams and Assessments Tips for Exams and Alternative Assessments and UC Berkeley, Alternatives to Traditional Testing).
- Series of Quizzes or Chapter Tests: Offer low-stakes opportunities for students to demonstrate mastery of material and give you ongoing information about student understanding. Frequent quizzing has also been shown to reinforce student understanding. These quizzes can be set up in Blackboard, questions can be randomized, making cheating more difficult, and grading can be automated.
- Student-developed Quiz Questions: Writing quiz questions both builds and demonstrates students’ understanding of the material. Such an assignment can be used as a collaborative group activity.
- Open-book, Take-home Assessments: Many disciplines already have a tradition of take-home exams, typically involving more conceptual or applied questions that students cannot quickly look up in a textbook.
- Professional Presentations, Infographics or Demonstrations: Students can create audiovisual presentations using a variety of media such as PowerPoint, Prezi (students can set up a free account) or video creation tools, or other tools such as Piktochart for making infographics.
- Annotated Anthology or Bibliography: This kind of project gives students a choice in selecting works while assessing their higher-order abilities to evaluate sources, compare multiple perspectives, and provide rationales for their choices.
- Memorandum or Briefing: Students prepare a one- or two-page memorandum or briefing, with, for example, the following headings: background, problem, possible solutions with pros and cons, final recommendation, and others. Besides being a good exercise in synthesizing material, it’s an excellent way for students to practice being concise and direct.
- Fact Sheet: Students create a one-page fact sheet on a topic. Students must select relevant facts and explain them clearly and concisely.
- Peer- and Self-Review Activity: These allow for personal reflection on learning and peer-to-peer instruction, both of which reinforce and deepen understanding. Students will still need instruction in the task of providing constructive feedback as well as providing them clear rubrics laying out expectations for student work.
- E-Portfolio: A student-selected portfolio of work from the semester. Students compile their best or representative work from the semester, writing a critical introduction to the portfolio and a brief introduction to each piece.
- Non-Traditional Paper, Project, or Poster: Creative assignments work best when they have some “real-world” relevance and offer students some choice in delivery format.
- Group Project: Group projects require students to demonstrate mastery of subject matter and develop their ability to communicate and work collaboratively. It is crucial to make your assessment criteria and grading scheme clear, and to ensure that there are clear, explicit expectations for each team member.
- Open-Book Assessment in Quantitative Courses: STEM and other quantitative courses face a particular challenge in creating effective
online exams, in part because it's so easy to cheat and in part because so many questions
are computational. Large courses reliant on in-person exams should consider open-book
exams or frequent low-stakes assessments as alternative assessment strategies that
are relatively easy to grade and better assess progress through the semester. In Remote Exams and Assessments Tips for Exams and Alternative Assessments, Joe Guadagni from Rutgers Mathematics Department has compiled this advice:
- Ask more conceptual questions (e.g., "what is the next step in this problem?", "explain why this hypothesis in the theorem is necessary").
- Ask students to identify an error in a proof or computation (this is particularly effective since it can't be googled).
- Eliminate multiple-choice and fill-in questions in favor of show-all-work questions where students have to scan and upload their work.
- If using problems from a textbook, change not only the numbers but also the names (e.g., John to Alice) and the scenario (e.g., pulling a boat in to letting a kite string out). The reason for this is that textbooks will probably have many of their problems already solved online somewhere, for example, on Chegg.
- Use letters and variables in place of specific numbers.
- When randomizing the exam, don't just randomize numbers. Also randomize discrete parts of the problem. For instance, one version might have a problem like "maximize the volume of the box given its surface area" whereas another version might have "minimize the surface area of a box given its volume".
- Avoid questions that consist of only simple computations. For example, instead of "calculate this integral", present students with some application in which they also have to set up a proper integral. "Write an integral expression that is equal to the probability that..." or "write a triple integral which is equal to the mass of the region" are good alternatives. There are online calculators that will not only solve many computational problems, but also give step by step solutions. Adding more words and applications to a problem makes it more difficult to cheat and also tests the real learning goal: do students know how to apply basic principles?
When teaching online, some instructors feel the need to significantly increase the amount of graded assignments to more closely assess student learning. While ongoing knowledge checks can be a sound strategy, if you have to manually review each one it can lead to grading overload. Below are some tips for managing the grading load or reducing it to more feasible solutions.
- If assignments are complex and multi-part, it is more effective to break these down into smaller components such that the instructor can assess students at several points in the learning process and provide feedback.
- Provide students with explicit and detailed rubrics to accurately communicate expectations for student performance. Rubrics help mitigate bias by providing clear criteria for expectations and describes what learning looks like.
- Self and peer assessments can be used to reduce instructor workload, improve student learning experiences and build community.
- Look for opportunities to provide feedback to the entire class – i.e. an announcement or e-mail summarizing patterns observed in student assignments.
- Streamline grading feedback comments on assignments using Blackboard Annotate. Comments can be text, audio, or video.
- Mitigate unconscious bias by focusing on the content of an individual answer rather than the students’ overall submission or identity.
Assessment During A Crisis: Responding to a Global Pandemic, an August 2020 report by the National Institute for Learning Outcomes Assessment, provides “an overview of findings from NILOA’s national survey of assessment-related changes made in Spring 2020 in response to COVID-19; couples those findings with other reports released from March through July; and provides guidance in the form of “do’s” and “do not’s” for higher education and the field of assessment—looking beyond Fall 2020 toward what needs to be done.” A fundamental takeaway from the report's recommendations is that the teaching and learning environment won't be anything resembling normal while still in a pandemic situation. Some of the report’s recommendations include:
- Do not forget that we are in a pandemic. Still. Do not forget that it is also an inequitable pandemic. Do be aware of and address these systemic inequities.
- Do not cause further harm. Do not support, enable, or endorse policies that perpetuate further inequities or fuel negative perceptions of students.
- Do not require a higher-level of proof of learning in an online class than you would normally require in a face-to-face setting.
- Do use learning outcomes as a guide and means to design and focus educational offerings.
- Do modify assignments and assessments in ways that are flexible, utilize low-bandwidth, and are based in the principles of equitable assessment.
- Do not forget that this is not the educational experience students wanted or expected. Nor is this a test of online education.
- Do engage in trauma-informed and healing-centered pedagogy and assessment.
Flaherty, Colleen, Inside Higher Ed. Grading for a Pandemic. A thoughtful discussion on grading in a pandemic – how lenient should professor be with students right now?