Crash Course in Degree-Level Assessment

Learning Outcomes, Curriculum Maps and Assessment Plans. How to Design a Healthy Curriculum while Satisfying Reporting Requirements.

Back to Assessment of Student Learning

Our curricula are living, breathing structures that require occasional upkeep and revision. Whether in response to shifts in disciplinary thinking, changes to the composition of our student body, or departures of colleagues and their expertise -- these moments offer an opportunity for us to refocus on collective efforts around delivering an intelligible, well-scaffolded learning experience to our students.

At no point has the impetus for revision been more pressing than during the pandemic, which accelerated a need for innovative teaching that creates new opportunities for equitable student growth and discovery. Such innovation in the classroom is best supported by a well-designed curriculum, where every instructor makes contributions to the collective project of educating students.

 

When reflecting on the health of your program’s curriculum consider questions like:

  • Who are the users and consumers of your program’s curriculum? Is their usage one-time or ongoing? Short-term or long-term?
  • Why should students choose your program? What are the benefits you offer from a disciplinary standpoint? In terms of cognitive or transferable skills?
  • What are you most proud of? What do you wish you could do differently?
  • With whom do you talk about your program? Are some types of conversations easier to navigate? To whom do you wish you could spend more time talking?
  • Does your program recruit students easily? Do you retain the students you have and help them progress toward their degree in a timely manner?
  • Are you or your colleagues ever rudely (or pleasantly) surprised in the classroom by your students lack (or command) of your discipline’s subject matter?
  • Are there “notorious” classes in your program? Are there “popular” courses? Are there some that are consistently under- or over-enrolled?

Your answers to the above question very likely point to opportunity to incorporate diverse sources of information in your curriculum work.

These will range widely, from the received wisdom of your discipline to the top-of-mind concerns of prospective freshmen and their tax-paying parents. There’s no “correct” mix of inputs -- let you and your colleagues’ intuitions guide the process.

Academic Years 2021 - 2024

KU is asking its departments to engage in a degree-level design process that closely mirrors the four qualities of a well-designed curriculum.

A well-designed curriculum is a set of interrelated learning activities that are intentionally constructed to help students achieve a set of learning outcomes. It includes a plan for continuous assessment and evolution or renewal.

Specifically, a well-designed curriculum has:

  1. Clearly articulated Program Goals.
  2. A Curriculum Map that aligns courses with program goals.
  3. A plan for assessing student achievement of the goals.
  4. A process that promotes reflection and curriculum evolution over time.

Draft clearly-articulated learning outcomes for the degree program, which also requires defining what different “levels” of knowledge or skills acquisition within each of these goals looks like (i.e. “introductory” versus “advanced” engagement with an outcome). For undergraduate programs, this step is due February 2022. For graduate programs, this step is due in February 2023. For graduate programs, this step is due in February 2024.

A curriculum map that depicts whether and where these outcomes and levels are represented in each critical course within the program’s curriculum. This step is due for undergraduate programs in February 2023.

A plan to assess or evaluate students’ ability to meet the outcomes of the degree program. This plan will include the selection of specific assessment methods (direct vs. indirect; formative vs. summative) as well as specify who in the department is responsible for assessment. This step is due for undergraduate programs in February 2023. For graduate programs, this step is due in February 2024.

The adoption of processes or occasions within the department to discuss assessment results, make meaning of their implications for students, and then devise revisions to the curriculum in response. This work should be ongoing within each department.

In addition, during 2022 and 2023, departments were asked to note where their degree program’s learning outcomes align with KU’s new institution learning goals (ILGs). More information about this request can be found on the university’s assessment website, as well as in the annual report submission portal.

Four Qualities of a Well-Designed Curriculum

For ideas about the substance of your outcomes, consider the following starting points:

  • For common program goals in your discipline, look to similar programs at peer institutions or “best practices” guidance from your professional association.
  • Peruse the results of the KU Career Center’s “hopes and dreams” or “first destination” surveys to shape program goals that either target prospective students’ interests or position graduating students for successful careers.
  • Work with KU’s Alumni Association and Career Center to identify common employers of your graduates and talk with these employers to learn more about what they value.
  • Draw on Visual Analytics dashboards from AIRE to learn more about your current students. How can your program’s goals speak to their current needs or integrate with their coursework in the Core Curriculum or other departments at KU?

CTE has hosted workshops on curriculum maps. The workshop presentation (.pdf) from AY 22-23 has helpful information along with the toolkit on Mapping Learning from the National Institute of Learning Outcomes Assessment.

  • With program goals in-hand, think about levels of acquisition (i.e. introductory, developing) as well as where each of these levels appear in your course offerings.
  • It will be important to survey your colleagues about their impressions – whether formally or informally. In addition to creating “buy in” upfront, involving as many colleagues as possible will help shape a collective awareness of how the pieces of your curriculum work together to build student learning.
  • Consider working with CTE and AIRE to layer on top of this pedagogical map some additional information about student grades, patterns of student course-to-course movements, or progression/retention bottlenecks. These will all provide insight into course alignments, sequencing, and frequency.
  • Consider working with KU’s Career Center to embed a career skills or cognitive skills sequence within the curriculum. You might coordinate with KU’s Alumni Association to integrate narratives or presentations from recent graduates about post-graduation jobs prospects in your discipline’s field.

A commonly-encountered obstacle to routine, valuable assessment work at the department level is the lack of a long-run plan that helps faculty anticipate deadlines in advance and distribute responsibilities across multiple individuals. The CTE will host several workshops on drafting assessment plans during AY 2022-2023, but for now know that the critical components are:

  • Specify the method with which each learning outcome will be evaluated. A mixture of direct and indirect methods, and formative and summative methods should be utilized.
  • Identify targets in the curriculum (i.e. a single course, a sequence of courses, a lab or practicum experience, etc.) where the assessment method can be applied.
  • With methods and targets in-hand, you should identify both a timeline for the collection and analysis of the data, as well as individual owners of different pieces of the process. This can vary widely by department, but frequently these tasks fall to associate chairs, directors of undergraduate study, track directors, or curriculum committees.

Assessment “data” can take many forms. Cast your net widely to include things like:

  • Are there discipline-specific praxis or benchmarking activities, prompts, or test questions you can use to compare your program to similar ones nationwide?
  • Consider using KU-level survey instruments (like the NSSE) that tracks student responses at the program-level.
  • Many professional programs (like law, education, and medicine) survey employers and clinicians about their students’ performance in the workplace.
  • Explore new forms of assessment data generated within your department, perhaps in collaboration with CTE. These could include surveys, exit interviews, focus group sessions, pre- and post-test interventions, and holistic rubric tools.

The work of assessment is all for naught if it does not lead to well-reasoned conclusions that productively orient us toward curricular revisions. But arriving at this point – reflection and revision – does not occur by happenstance; it requires dedicated time and bandwidth. Here are some suggestions about how best to make meaning of your department’s assessment results and take the next steps toward revision:

  • Institutionalize an occasion to review the results of assessment with your colleagues, perhaps a dedicated faculty meeting or assessment-specific event.
  • Enlist the help of your students in interpreting the data, especially at the graduate level. Involve students in the actual collection and analysis of the results of your curriculum design and assessment.
  • Consider how to represent your innovations and revisions to audiences outside of your department. How can you show administrators, prospective students, or potential employers that you’re responsive and dynamic?
  • Can you dialogue with your recent alumni about the results of curriculum work?
  • Weigh whether a subset of the results align well with other strategic initiatives and focus on those as an efficient way of addressing multiple issues at once.