Promoting Critical Thinking and Analysis in a Graduate-Level Course—Meagan Patterson (2012)

Advanced Educational Psychology: Learning Processes in Education (PRE 704) (pdf) is a graduate-level course taken by students across the School of Education, with the largest group being from the Curriculum & Teaching department. Although it is a masters-level course, some doctoral students also enroll. In nearly all cases it is taken to fulfill either a School or program-level requirement.

PRE 704 has six major objectives; students will:

• Become familiar with the major theories of human learning.
• Demonstrate an understanding of the major tenets of the theories covered.
• Become familiar with current research on learning and instruction.
• Become familiar with current research on environmental, individual, and interpersonal factors influencing human cognition and learning.
• Understand the interactions among cognition, emotion, and motivation and how each influences learning and problem solving.
• Be able to apply learning theories to educational situations and discuss teaching and learning practices in the context of these theories.

I have taught this course several times, and although it has worked overall and seemed to satisfy the students, I was not satisfied that I had challenged them enough and that they were learning enough. Given students’ different backgrounds and interests, I struggled to find the right level for the class. I was concerned that students’ learning was not as deep and complex as I would like it to be. Students had done fairly well learning specific course content (course objectives 1–4), but they had not done as well with integration / evaluation / synthesis / application (objectives 5 & 6). Partly due to students’ experiences and goals, and partly due to the nature of course activities and assessments, students demonstrated stronger performance when asked to apply material than when asked to synthesize / evaluate. Therefore, for the Fall 2012 semester I revised PRE 704 in the hopes of making it more engaging (both for the students and for myself) and, more importantly, deepening student learning.

In past iterations of PRE 704 students had two major types of opportunities to demonstrate their learning: in-class discussions, which included student-generated discussion questions, and three take-home exams. Given my desire to promote higher-order thinking and learning, I reformulated the course to provide more and better opportunities for students to practice these skills and to demonstrate their skill acquisition.

The first step I made was to introduce empirical articles into the required reading. Previously, I assigned readings from a general textbook and supplemented it with scholarly texts. In this kind of reading, the majority of the work of synthesis, integration, and evaluation has already been done. Conversely, when reading an empirical article, the reader is required to interact more with the material.

I felt that including empirical articles might also help make in-class discussions more productive and more focused on analysis and critical examination of the readings. In previous iterations, I was not fully satisfied with the student-generated discussion questions. Although many students produced thoughtful questions that really facilitated discussion, too many questions were either closer in spirit to comprehension questions (demonstrating that a student had done the reading but not leading to discussion) or were aimed at getting more information about a particular topic, rather than at facilitating conversation and debate. I added a new, in-depth handout that provided examples of discussion questions of varying quality (i.e., excellent, good, acceptable).

Second, I reworked the course exams. Rather than having three take-home exams consisting of three prompts each, which accounted for the majority of the students’ grade, I implemented smaller response papers (pdf). Using many of the original exam prompts, students were asked to respond to only one prompt per paper. Students also were able to choose which papers to write; they received prompts six times during the semester but were required to turn in only five papers. The six prompts and their rubrics are available in the links below. Each response paper was also lower-risk, being worth 10 percent of the final grade rather than the heavier 25 percent per exam. This structure provided students with feedback earlier and more frequently, giving them greater opportunities to improve their work over the course of the five papers.

Response Paper 3

Response Paper 5

On average there are 20–25 students in the course. In the past, each student would turn in a 10- to 15-page response for each take-home exam. In order to grade this volume of work in a timely manner, I felt that I was not giving quality feedback. Following discussions during the May 2012 Best Practices Institute, I created rubrics to accompany each response paper, the goal of which was to allow me to provide information and guidance while reducing the time spent per paper.

Finally, I added a literature review (pdf) due at the end of the term, which not only required students to evaluate and synthesize their sources but also provided them a chance to practice a useful professional skill. While worth the largest percentage of the overall grade (pdf) (25 percent), I built in smaller assignments throughout the semester, including bringing in their topic, their bibliography, and a draft. These elements were low-risk, being graded on a credit/no credit basis, and allowed for any needed course corrections early in the process and for some peer review. By allotting some class time for these smaller assignments, I did lose some content-delivery time. However, the time lost was not extensive, giving preliminary and final topic and bibliography reviews about 15 minutes and rough draft reviews about 30 minutes.

Response papers

Overall, students did well on the response papers. I think that having an example rubric provided at the beginning of class helped students to understand how the papers would be evaluated. Student course evaluations confirmed this belief; the course received high ratings on the question regarding expectations being fair and well defined.

Many of the response paper prompts asked students to define or describe a particular concept, then apply that concept to a particular educational situation (for example, describe the characteristics of effective peer learning programs, then design a peer learning program that embodies those characteristics). Highly successful response papers (pdf) included examples that had sufficient detail for the reader to understand the people, situations, and strategies involved. Successful papers (pdf) also included a clear, explicit discussion of why the example presented was an example of the underlying construct or why the suggested strategy would work, with reference to theories, concepts, or research findings discussed in class. Less successful papers, in contrast, tended to have less detailed examples (pdf), or examples that addressed some (pdf) but not all relevant aspects of the underlying concept.

Literature review paper

A second major change in this course was the addition of a literature review paper, due at the end of the semester. This assignment allowed students to explore topics that were of interest to them but were not discussed extensively in the course itself (such as the effects of caffeine consumption on learning, or the role of conditioning in the development and treatment of phobias). Most students seemed to enjoy the opportunity to explore a topic of personal interest or relevance.

Highly successful literature review papers (pdf) described existing empirical research on the topic fully and clearly and integrated the findings of multiple studies to form a coherent picture of what is currently known regarding the research area. Less successful papers (pdf) tended to describe each article or study individually, with less evaluation and synthesis. Some less successful papers also included too few empirical articles and too many theoretical or review articles; I have revised the guidelines for this assignment for future classes to be more explicit about the types of articles that should be included.

Overall, I felt that the redesign improved the course in several ways. Having a greater number of small assignments and clear guidelines about how work would be assessed seemed to reduce student anxiety and uncertainty about their grades. Students seemed to enjoy the opportunity to research a topic of their choosing for the literature review paper, and several mentioned that they were using the assignment as an opportunity to explore a potential thesis topic.

One aspect of the course design with which I struggled was creating rubrics. During the Best Practices Institute at CTE, someone suggested that rubrics for upper-level courses should provide a “target” for each element, rather than examples of each element at different score levels. Given that this is a graduate-level course, I decided to utilize this format. I do think that the rubrics were an efficient form of providing feedback to students, and students seemed more aware of and comfortable with how their work would be assessed. I think the rubrics could be improved further, however. In some cases I had trouble quantifying “I know it when I see it” elements, such as whether an example included sufficient detail. I also want to work more on giving feedback that not only demonstrates how students could have improved their performance on the paper being graded, but also how they could apply this feedback to other assignments. In the future, I may also provide an annotated model of a high-level paper to provide another source of guidance on how response papers will be evaluated.

Although I think the literature review paper worked in general, I do plan to alter the assignment when I teach the course again. I surveyed the class, and nearly all students indicated that they knew how to find scholarly articles on a topic of interest. I also assumed that most students would have prior knowledge and/or experience with literature reviews. So while I provided some reference information via Blackboard, I did not spend much class time on the basics of conducting a literature review (e.g., finding articles). However, some students did not have as much prior knowledge as I had thought, and so they struggled with the assignment. In the future, I probably will spend some class time discussing how to approach the early stages of conducting a literature review.

In order to facilitate students working on the literature review throughout the semester, there were a series of preliminary assignments (i.e., topic, bibliography, rough draft) due at various points during the semester. One change that I will make for future versions of the course is adding a brief article summary as one of the preliminary assignments. I think this will ensure that students start reading the articles to be included in the literature review earlier in the semester (during this semester I had several students ask to change their paper topics shortly before the rough draft was due, suggesting to me that they had not done much reading prior to that point). Relatedly, I probably will also move the peer review process to begin earlier in the term. Some students, who misunderstood what a literature review required, only realized their mistake during the review process, which did not give them much time to alter course. To better utilize in-class review time, I plan to have students exchange drafts with their review partner prior to class.