Degree Level Outcomes Guide

Degree-level learning outcomes signify to students the ultimate rewards of their time, effort (intellectual, emotional, and physical), and investment in a degree program. For departments, degree-level learning outcomes represent moments of agency, wherein a department can self-narrate its values in strategic and differentiating ways, conveying the promise and worth of the program and distinguishing it from others.

Writing degree-level outcomes

The SMART framework and learning taxonomies included on the "Course-level Outcomes Guide" page all apply here, and instructors are encouraged to become familiar with those ideas before proceeding: The information is foundational for authoring degree-level outcomes, which should be penned knowledgeably and collaboratively by department faculty. It also provides sufficient background to grapple with the following considerations, which are unique to degree-level outcomes (DLOs) and which are described in turn below:

  • Framing specific to DLO development
  • Common misconceptions about DLOs
  • Finding the right balance in DLOs
  • Assessing broad and high-level DLOs

Framing degree-level learning outcomes

Degree-level outcomes provide challenges not encountered with other types of outcomes: How does a department encompass the learning taking place across years, courses, and other experiences—many of which a department may not have direct knowledge of or control over—accurately, understandably, and in just a handful of statements? Like other outcomes, the best way to develop meaningful and actionable statements is to practice “Backward Design” (Wiggins & McTighe; 1998, 2005). You and your colleagues might reflect on the following questions, putting thoughts to paper as you go:

  • What does the department think the degree program prepares students to do? (In some programs, the net result of the learning activities--the doing--looks like preparation for academic work. How might that serve both student populations: those who will continue their formal education as well as those who won't?)
  • How does faculty composition, in terms of academic expertise, inform the degree program's structure and offerings? Does this composition result in emphasis in skill or knowledge that would be unusual to encounter elsewhere?
  • Are there distinct (preferably intentional) themes or arcs of learning within the degree program? What are they? Why do these exist and what are they intended to do?
  • How do you concisely describe the aggregate of the knowledge and skills students develop under your direct guidance? What about during time in courses and experiences outside the department but which are part of the degree?
  • For courses or experiences that don't fit into a theme or arc of learning, can you describe the ways those courses or experiences still support learning in the aggregate or perhaps in terms of personal development?
  • Is there space for other voices: What skills or knowledge do students want as a result of their time and effort toward earning the degree? After graduating, students radiate along any number of different paths: private or government employment, additional credentialing or licensure, graduate education, etc. What are the expected skills and knowledge there? To what extent does the program aspire to meet these external expectations?
  • Is an accrediting body already guiding degree-level learning outcomes statements? Do the guidelines permit any flexibility for the department to reflect its unique strengths, either by modifying provided outcomes statements or appending new ones?
  • If students earn a degree consisting of only 3-7 big takeaway skills or ideas, what would they be?

While this is a similar to the reflective framework recommended for course-level outcomes, the considerations above for degree-level outcomes are much broader. The ability to respond to these prompts may also be more limited: Whereas an instructor usually can conceptualize and describe a course in its entirety, departments may or may not have ready answers to one or more of the guiding questions above. The work warrants greater collective thought based on insights from many individuals.

The prompts above are all different ways of framing the same overall question: What do you want students to be able to do as a result of earning this degree? Your answer will be one or more degree-level learning outcomes.

Common misconceptions about degree-level outcomes

External messages around outcomes (and there are many) combined with internal feelings around outcomes (and there are many) can result in a cauldron of mistruths. It takes only a sip of this witch's brew to poison the assessment practitioner, leading them to write lifeless and poorly functioning outcomes. Several problematic notions are dispelled below.

Statements to dispel about degree-level outcomes:

The statement "Do it right, do it once," at least as relates to outcomes, tends to be rooted in fear:

  • Some faculty feel that, with outcomes successfully submitted, they've overcome an assessment hurdle of monumental uncertainty. (Who's looking at the outcomes? Are the outcomes "right?" What if they're not?) In this light, wanting not to repeat the experience is understandable.
  • Along similar lines, with an outcome in hand and data rolling in, faculty can feel "locked in." If the outcome changes, the collected data will be rendered useless. "Back to square one!"

Neither line of thought is generally true, and each is addressed in turn.

  • There are best practices in writing outcomes, and some outcomes are stronger than others. However, assessment is rooted in process rather than destination; there's no "right" outcome. Accordingly, outcomes can and should be adjusted over time. In fact, static outcomes, particularly with little effort toward capturing student progress toward those outcomes, may be a sign of dwindling reflective practice. A good rule of thumb is to revisit outcomes at least every few years. Are they still working well for the department?
  • Data showing student progress toward outcomes can be reinterpreted, recollected, or abandoned in light of changing outcomes. However, it's rare that abandonment of data (and the time and effort their collection represents) is necessary. As outcomes change, talk with colleagues and assessment staff about how to capitalize on existing data.

As departments and students change or new information becomes available, outcomes should be revisited. Like personal aspirations, outcomes are dynamic, shifting over time.

"If it's learning that happens while earning a degree, it must manifest within the degree-level outcomes." This isn't true. Degree-level outcomes are opportunities for departments to offer what's most important to them—their values. A finite set of statements will never capture everything, and while framing learning in a hierarchy captures most student learning, much learning, especially over an extended time, is tangential, incidental, or emergent. Students will, in fact, develop skills or knowledge that play a supporting role in the overall degree.

As students increasingly encounter degree-level outcomes and use them as guideposts on their educational journey, it would be worth keeping in mind that describing the learning that constitutes the degree is important, and any uncodified or supporting learning that is not communicated through DLOs can be conveyed other ways.

Again, degree outcomes are moments for departments to declare what they value, which usually but not always aligns with what they do. Additionally, discipline-specific accrediting bodies may require particular program outcomes. A result is that sometimes departments include outcomes related to ideals that are represented at a low level in the curriculum.

Confused? One example is ethics requirements in STEM courses. In very few instances do STEM students have a full course dedicated to theories of ethics and decision-making, yet ethics-centered degree-level outcomes often appear. Certainly, if outcomes represent a department's values for student learning, the department should have a mechanism—a module, a course, a case study, a reflection on extracurricular experiences, etc.—for students to demonstrate their facility. In this example, the department might identify one or (preferably) more curricular moments wherein students receive instruction in ethics and then work with those ideas, even if at a low level.

For curricularly de-emphasized outcomes, instructors still must be able to put eyes on student work and evaluate the type and level of learning that has occurred, i.e., the outcome must still be assessable. But even a learning ideal with low emphasis can be a degree outcome. "Students leave our program with a working knowledge of ethics." The job is to be able to answer: To what extent do they?

Capturing a broad array of cognitive activities naturally elicits the use of broad or all-encompassing terms:

  • "Students understand..."
  • "Students demonstrate..."
  • and more.

Faculty are commonly warned against such vague language. Indeed, warnings can be found on this very page, but caution is not prohibition. Terms like understand and demonstrate are permissible so long as supporting language that indicates what you mean specifically by these terms is also included.

  • "Students show their understanding of syntactic theories by dissecting passages across languages in terms of three theories..."
  • "Students demonstrate scientific thinking by designing, conducting, analyzing the results of, and reporting simple chemical syntheses..."

These two examples highlight that in some cases, vague language really means a single particular activity (analysis), while in other cases, a vague term may be an umbrella for several particular activities (design, action, analysis, written communication.) When thinking in broad terms, ask yourself: What do you really mean?

"Students will think critically about principles and practices in the discipline." Nearly all departments have some flavor of this outcome, which is problematic because:

  • This outcome doesn't differentiate a degree at a given institutions from the same degree offered at other institutions. Nay, this doesn't even differentiate a degree from degrees in other disciplines.
  • Every discipline has a different notion of what constitutes critical thinking. It's an interesting exercise to ask colleagues both within and without your department what, exactly, they think critical thinking is. What type of thinking is included? What type of thinking is excluded? If it means thinking like an expert, how, in real detail, do experts think? A related aside: most instructors also want to "teach students to think," which is a similarly noble but nebulous goal. There's little consensus as to meaning.

It will be better to work with colleagues to arrive at agreement on what is truly meant by "critical thinking" and to communicate that instead. Where departments inherit degree-level outcomes from, say, an accrediting body, is there an ability to alter or add to the statement for clarity? If not, through what other mechanisms can you clarify for students what is meant by "critical thinking?"

 

While non-technical language can be imperative for undergraduate degree outcomes, in which students or those supporting them may not be familiar with the disciplinary lexicon, this is much less true for graduate outcomes. Those applying to or in the midst of a graduate education typically have a background sufficient to make ready sense of weighty wording. But at that point, it may be worth asking: What does adding jargon afford that couldn't be achieved without it? Perhaps the department has established a good reason for writing graduate-level outcomes in an academic fashion—perfectly acceptable! While it will always be useful to keep external audiences in mind, at some level, using technical language will be a matter of taste when creating graduate degree outcomes.

Finding the right balance in DLOs: Tension between generality and specificity

Crafting a degree-level outcome is a balancing act between encapsulating myriad learning over significant time (suggesting general descriptions) and expanding upon the unique value the faculty and their offerings provide learners (suggesting specific descriptions.) This tension can be maddening, but the latter approach tends to be stronger, even as the common practice is the former. Often, degree-level outcomes are housed in intentional vagueness to create flexibility—good intent!—but in the way a bone housed in vinegar becomes flexible: All strength has been leached from it.

Read the following fictitious four examples of generic vs. specific degree-level outcomes, being mindful of the following:

  • How well could a student, colleague, or other audience get a sense of what will be expected of students in showing their learning?
  • To what extent can you envision how faculty might capture student progress toward the outcome, i.e., assess the learning taking place?
  • How compelling is the statement, resulting in appreciation, if not excitement?
  • For each, what questions remain about the the intended learning in the degree?

Generic degree-level outcomes

  • Students will communicate effectively orally and in writing, describing major theories in archaeology.



  • Students will understand ethical conventions in behavioral science and apply these conventions to their work.

  • Students demonstrate mastery of auditing and auditing processes.



  • Students earning a MFA in Film Making will produce a film in compliance with professional guidelines.

Specific degree-level outcomes

  • Students earning a B.A. in Archaeology can write well for various audiences, from children to academics, to inform them about how archaeologists write the prehistory of the world, as rooted in standard disciplinary notions, with special focus on "adaptationism," human archaeology, and the archaeology of gender.

  • Students in behavioral science critique the methods used in real behavioral science studies in light of both ethical theories and norms of the discipline, ultimately offering their findings in a poster presentation.

  • Accounting students in their last year will, in various courses, formally audit the taxes of each type of business (sole proprietorships, partnerships, limited liability companies, or corporations) and nominate the strongest of their audits to their undergraduate portfolio.

  • Students earning a MFA in Film Making will demonstrate their cinematography by filming and editing a 40-minute film that meets selected SMPTE guidelines, incorporates both standard and innovative style, and culminates in a live film exhibition with a panel of critics present at viewing (at least 3 of them department faculty).

Assessing broad, high-level degree-level outcomes

Assessment takes real student work and compares it to idealized learning (i.e., outcomes): What congruence is there?

When searching for evidence of student progress toward learning outcomes, two pieces of advice serve practitioners well:

  • Avoid temptation of trying to aggregate course grades. As already aggregated notions of learning, grades reflect multiple skills that may relate to multiple outcomes. One can imagine that further combining grades has the effect of obscuring progress toward the specific goal being assessed.
  • It is necessary to "drill down" into specific student work housed within courses. Best practice is to curate multiple "student artifacts" (student work that shows their thought or skill) from multiple courses to minimize different types of biases. Two or three sources of information will be better than one, in that multiple lines of evidence can corroborate (or refute) one another, providing a more robust sense of student progress. Similarly, the "learning story" that emerges from multiple lines of evidence is typically both more surefooted and interesting. One concern with this approach is added effort. Use of designated assessment events or committees can be useful here.

There's often some confusion as to what this last point about "drilling down" means. An illustration is offered below. Consider the assessment of the blue DLO. Where in the degree program do students show how far they progressed toward this outcome? A natural place to look is late in the curriculum. In those courses, are there instances of students working toward the blue outcome? Three pieces of student work might be collected in two different courses. In course B, a student reflection that relates to the blue outcome could be collected. In course C, there are two moments, including capstone work, that might also serve as evidence of progress toward the blue outcome. These three student artifacts together may be a compelling indication of student effort and mastery of this outcome. If, after review, faculty feel that the evidence doesn't match the outcome as hoped, they can, in a future iteration of assessment, identify and collect a different set of student artifacts related to the outcome. Alternatively, and often better, faculty may choose to adopt instructional approaches that better align with or support the outcome, thereby preserving their stated ideals (outcomes) while helping student reach them.

Hierarchy of learning activities: degree-, course-, and activity-level outcomes. There are 4 degree outcomes that relate directly to the course- and activity-level outcomes.
An example mini-degree consisting of three courses, each of which has several course activities. There are four color-coded degree outcomes. Course- and activity-level outcomes that relate directly to a degree outcome are the colored the same.