Scaling Student Assessment Series

The Scaling Student Assessment Podcast series examines the need and potential techniques to consider as class sizes grow.

Ep. 1 - Scaling Exams and Quizzes

In this podcast episode, we dive into strategies for scaling up grading and logistics for assessments like two-stage exams.

Additional Resources for Scaling Exams

Ep. 2 - Scaling Mid-Semester Feedback

In episode 2 of our Scaling Assessment Series, we talk to Dyan Morgan, associate teaching professor in undergraduate biology to discuss methods of gathering mid-semester feedback.

Additional Resources for Scaling Midterm Feedback

Important Points for Midsemester Feedback

While it’s beneficial to think about responses in the aggregate (done by you, a TA, or perhaps most reliably by AI), be sure to put eyes on each response. In written feedback, students often voice concerns or personal struggles that may need individual attention.

If collecting information live in class, don’t shy away from simple but effective methods of data collection:

  • thumbs up, sideways, or down
  • green, yellow, and red colored voting cards
  • "Please stand if you..."

In a live class, restrict feedback to questions that are comfortable to answer.

There are many live polling platforms for collecting live (and often narrative) feedback. These sometimes offer free subscriptions below a user threshold. Here are just a few: Menti, Slido, Poll Everywhere, Socrative, Kahoot.

Anytime you ask students a question in class, make sure you're comfortable reacting to the responses live in front of them. If you aren't comfortable, consider a different method to collect information.

Ep. 3 - Scaling Student Voice

In this episode, Dr. Dean Richards, assistant teaching professor in the School of Education Department of Curriculum and Teaching sits down with CTE's host Drew Vartia to discuss why student voice in assessment is important.

Ep.4 - Scaling Student Voice - Bonus Episode

In this bonus episode on student voice, CTE's Drew Vartia interviews Yvonne Chen, professor of Journalism and Mass Communications, about strategies to incorporate student voice in course assessment. Yvonne describes how she works with students to create rubrics for course assignments.

Additional Resources for Scaling Student Voice

Ep.5 - Scaling Assessment with AI

In this episode, Doug Ward, CTE associate director and associate professor in Journalism and Mass Communications, along with Sara Wilson, associate professor in Mechanical Engineering and CTE Faculty Fellow, sit down with CTE's host Drew Vartia to talk about the possibilities of AI assistance in assessment.

Guidelines, Resources, and Perspectives: Using AI to Make Sense of Course Evaluations

  • You've asked students to write something to you. There's an implicit moral mandate that you--not just AI--read what they write.
  • Course evaluations must be anonymous, and this is not always the case. Sometimes students, especially freshmen in large courses, self-identify in their comments. Names must be redacted before using AI.
  • Occasionally a student will reach out for help with a personal problem through mechanisms like course evaluations. Such events are low-frequency but high-stakes. Make sure you catch these kinds of comments yourself so you can respond appropriately to the student. (And of course, redact their identifying information before uploading to AI.)
  • The reliability of AI is still evolving. At the end of your AI analysis, you should be able to answer the question: Is what I get back from the AI reasonably compatible with my intuitions after reading the comments myself? If not, can I dig deeper to find the discrepancy?

As this Inside Higher Ed article from 2023 self-describes, they detail "the unique challenges that AI presents as well as guidance for management and faculty to effectively engage with artificial intelligence."

In addition to some thought-provoking notions about AI's role in assessment, the section titled "Remember the Human Element" in John Spencer's blog post provides an poignant anecdote that underscores the importance of the instructor's humanity--something AI can't (and shouldn't) replace.

Traditional course evaluation questions can be considered prescriptive, in that they ask specific questions. At the other end, course evaluations often leverage an open-ended section, but this is static. What if there were a way to hear student thoughts on a course and drill deeper in real time? This 2018 article describes such an AI-driven tool.

Ep.6 - Scaling Alternate Grading

In this episode, Trevor Rivers, assistant teaching professor in the undergraduate biology program, sits down with CTE's host Drew Vartia to talk about methods to scale alternate grading in assessment.

Additional Resources for Scaling Alternative Grading Strategies

Alternative grading can take many forms, and each instructor may fashion their own system. However, most strategies fall into just a handful of categories, each with its own strengths and weaknesses. For the unfamiliar and those wishing to retool, the following may be particularly valuable:

For those wishing for a deeper dive, each of these sites draws on several primary sources and some secondary sources.

Peer evaluation is a strategy that can help with alternative grading in larger courses. An immediate and frequent question raised by instructors: But as non-experts, how good are students at evaluating each other? In small courses, it may be possible to improve peer evaluation skills through direct work with students. In large courses, such intimate coaching may not be feasible.

There are several technologies that help students develop skill in providing meaningful feedback to their student colleagues, and these vary considerably in cost, customizability, function, interface, and more. Here are just a few examples: Aropä, CrowdGrader, Mechanical TA, RiPPLE.

Ep.7 - Scaling Student Projects

Courtney Summers, a lecturer with Speech-Language-Hearing: Sciences & Disorders Department, talks to CTE's Drew Vartia about her experience with student projects as class sizes grow larger at the university.