Building KU's Teaching and Learning Community

Incorporating Research into an Upper-Division Undergraduate Course: Transferring Lessons from One Faculty Member to Another—Holly L. Storkel & Megan Blossom

Overview

Two faculty members in the Speech-Language-Hearing: Sciences and Disorders department collaborate from one semester to another to implement progressive enhancements changes to an undergraduate course. The goal was to foster skills related to reading, critiquing, and interpreting research articles, so that when students begin professional practice, they can quickly and efficiently read the articles and know how to apply findings to clients.



Background

SPLH 466 (Language Science) is a required course for undergraduate students majoring in Speech-Language-Hearing (SPLH) and focuses on the basic structure of the English language, as well as how language is stored and accessed in the mind. In addition, SPLH 466 has typically taught basic research methods that are used to investigate language structure and processing. However, this was accomplished through lecture about those methods rather than students reading primary sources. Thus, the goal of this portfolio is to document two related approaches to incorporating primary readings in this course. Holly Storkel implemented the first approach in the Fall of 2011. Megan Blossom implemented the second approach in the Spring of 2013, addressing challenges faced in the Fall 2011 approach.

Implementation

Fall 2011 – Holly Storkel
I made several changes to the course to incorporate reading of research articles. First, I provided information on how to find research articles and how to read a research article to ascertain what information was important in each section. Next, I incorporated two activities that allowed students to practice reading research articles: in-class article reports and independent article structured summaries. Students gained a relatively strong foundation in understanding the content of a research article. However, specific skills were more challenging to students, and the ability to think critically about research findings and relate them to class or to real-world scenarios varied across students. The approach implemented in the Spring of 2013 addressed these challenges.

Spring 2013 – Megan Blossom
The approach to incorporating research articles in the Spring 2013 semester was modified in several ways. First, the number of articles that students were required to read was reduced from nine to six, which allowed for more intensive instruction on each reading. For the first five articles, students completed a pre-discussion assignment that they brought to class where the instructor led an in-class activity and discussion. Students then wrote a post-discussion summary of the article using their notes from the in-class activity and discussion. For the final article, there was no in-class activity or discussion, and students were responsible for writing their summaries independently.

Student Work

Fall 2011– Holly Storkel
Most students showed emerging abilities in understanding the basic content of the research articles. However, students had difficulty identifying and describing the independent variable and the analysis. In terms of critical thinking about study findings, a range of skills were noted with 30% of the students showing basic skills, 40% demonstrating emerging skills, and 25% mastering these skills. In general, students showed the most difficulty relating the article to topics discussed in class. Overall, students may need more scaffolding and practice to relate articles to topics/theories discussed in class.

Spring 2013 – Megan Blossom
The central difference between the outcomes for Fall 2011 and Spring 2013 was that no students had scores in the Absent or Basic rubric categories for either understanding of article content or for critical thinking. Similar to the Fall 2011 outcomes, the majority of Spring 2013 students (86%) showed emerging knowledge of content. Once again, the main areas of difficulty for students were identifying independent variables and describing the analytical approach of the article. In terms of critical thinking skills, the sample of students was fairly evenly split between emerging skills (47%) and mastered skills (53%). This is a moderate improvement in skills over the Fall 2011 outcomes, where 25% of the students showed mastery in the critical thinking areas of their summaries.

Reflections

In general, students in SPLH 466 in Fall 2011 and Spring 2013 demonstrated a solid foundation of skills in reading and interpreting the content of research articles. Additionally, students in Spring 2013 showed stronger scores in the critical thinking areas in their final article summary than students in Fall 2011, suggesting that changes made in Spring 2013 were somewhat effective in providing the support needed for students to acquire this skill. Given that the analysis section and the independent variables section were the most challenging for students in both Fall 2011 and Spring 2013 classes, even more focus on the process of analysis during in-class activities and discussions might be a target area for further development.


^Back to top^

Background

SPLH 466 (Language Science) is a required course for undergraduate students majoring in Speech-Language-Hearing (SPLH). It also serves as an elective course for those majoring in Human Biology. Generally, SPLH majors take the course in their sophomore or junior years, whereas Human Biology majors or SPLH minors tend to take the course in their junior or senior year. There are no prerequisites to the course so students enter with a variety of experiences. This course typically enrolls 50-60 students and is lecture-based, with opportunities for in-class large group discussion and small-group interactive activities. Course content focuses on the basic structure of the English language, as well as how language is stored and accessed in the mind. This topic is addressed as it applies to normal fluent adult speakers of the language, as well as in children and adults with language impairments.

Additionally, SPLH 466 has typically taught basic research methods that are used to investigate language structure and processing. However, this was previously accomplished through lecture about those methods rather than students reading primary sources. Thus, in 2011, primary readings (i.e., research articles) were incorporated into the course. The goal for incorporating research articles was to teach students to quickly and efficiently read, critique, and apply findings from a research article to real-world clinical situations. Accomplishing this goal has two potential outcomes: (1) providing a foundation for reading research articles in more advanced classes and mentored research experiences, and (2) enabling lifelong learning through regular reading of research articles and regular application to clinical practice. Speech-language pathology and audiology are evidence-based disciplines, meaning that a clinician is expected to apply current research findings to meet his or her clients’ needs. To keep current with the evidence, practicing clinicians need to be able to quickly and efficiently read research articles and know how to apply the findings to the clients they see.

We describe two related approaches to incorporating primary readings in this course. Holly Storkel implemented the first approach in the Fall of 2011. Megan Blossom implemented the second approach in the Spring of 2013, addressing challenges faced in the Fall 2011 approach.


^Back to top^

Implementation

Fall 2011 – Holly Storkel
To incorporate the reading of research articles into the course, specific objectives were added (see objectives 4 and 5 in the syllabus), and content and activities were adjusted accordingly. In terms of content, I added two short lectures. On the first day of class, I provided information on how to find research articles (pdf). Because I wanted students to become independent in reading research articles, I wanted them to be familiar with how to find research articles using the library’s online resources. Although specific articles were selected for each assignment, students still had to find the article themselves through the library rather than simply download it from Blackboard. On the third day of class, I provided a tutorial on how to read a research article (pdf). The goal of this tutorial was to familiarize students with the various sections of a research article in this field and to help them ascertain what information is important in each section. To accomplish this, I provided an annotated research article (pdf), and I led the students through the article, commenting on what important information was located in each section (see sample lecture notes).

Students then gained practice reading research articles through two types of activities: (1) in-class article reports and (2) independent article structured summaries. Both used a similar standard format. There were two in-class article reports for each of the three units in the course, totaling six in-class article reports throughout the semester. In-class article reports gave students an opportunity to practice reading, critiquing, and interpreting a research article with minimal impact on their overall course grade. Students read the article and made short notes on the provided template. These reports also ensured that students read the article prior to class, providing a foundation for in-class discussion. The GTA and I graded article reports quickly and leniently at the beginning of each class (see Format/Instructions). While grading occurred, students discussed their responses in small groups of their choosing. After grading and small-group discussions were completed, students participated in an instructor-led discussion of the article. Students volunteered answers to each item on the template while I made notes on PowerPoint slides. I would then highlight the correct answer and explain why that answer was correct. Once the details of the article (e.g., methods, results) had been understood, the class discussed the relevance of the findings to topics previously discussed in class (e.g., theories of language structure or processing) as well as potential clinical implications of the findings. Understanding of basic methods, results, and implications were tested on unit exams.

While in-class article reports provided support and scaffolding for understanding and applying research findings, I also wanted students to gain independence in reading research articles. Thus, students completed independent article structured summaries (see sample structured summary that was provided to students). Two structured summaries were available in each unit, and students had to complete one of the two. Thus, all students completed three structured summaries over the course of the semester. In addition, students had the option of completing more than three structured summaries. If students pursued this option, then only the three highest graded summaries contributed to their final course grade. This allowed students to gain extra practice if they struggled with this task. Also, this tended to make the grading of structured summaries more feasible for the GTA. Students received feedback on structured summaries through a standard rubric (pdf).

As can be seen in the Student Work section, students gained a relatively strong foundation in research in terms of being able to understand the content of a research article. However, specific skills were more challenging to students (e.g., detecting the independent variable, dependent variable, describing the analyses reported), and the ability to think critically about research findings and relate them to class or to real-world scenarios varied across students. This latter aspect of the discussion tended to get truncated during class sessions because it was the last topic to be discussed. In addition, discussion typically concluded after only a small number of students shared their thoughts, making it difficult to assess whether all students were acquiring this skill. This suggests that future attempts to teach these skills may benefit from increased class time and wider class participation on this particular topic. These aspects were addressed in Megan Blossom’s implementation strategy in Spring 2013.

Spring 2013 – Megan Blossom
In the Spring 2013 semester I made several modifications to address some of the challenges that Holly Storkel observed in Fall 2011. These changes were meant to address two goals (see syllabus [pdf]):

  1. To provide students with more initial support in reading and interpreting the articles early in the semester while gradually decreasing support over the course of the semester.
  2. To make the in-class article discussions more interactive and meaningful for all students rather than just the subset of students who participated or were called on.

To achieve these goals, the number of articles that students were required to read was reduced from nine to six. Although this meant that students had fewer opportunities to practice reading research articles, and they were exposed to fewer studies, reducing the number of required articles allowed for more intensive instruction on each one. In previous semesters, students completed six in-class article reports, which were discussed as a class, and three separate independent structured summaries, for a total of nine. In Spring 2013, students were assigned six total articles. For articles one through five, students completed a pre-discussion assignment which they brought to class where I led an in-class activity and discussion. Students then completed a post-discussion assignment where they wrote a summary of the article using their notes from the in-class activity and discussion. The in-class activity varied somewhat for each article. Examples of in-class activities are described below. For the final article, there was no in-class activity or discussion, and students were responsible for writing their summaries independently. I provided some background on the methods used and offered students an opportunity to ask questions about the articles. Otherwise, students’ work was completely independent.

Pre-discussion assignments and in-class activities. Pre-discussion assignments were generally a series of short questions designed to prepare students for the in-class activity. The in-class activities varied widely depending on the article, but all involved some variety of group work with a focus on understanding the methods, results, and analysis of the articles. I had two reasons for focusing on these “nitty-gritty” aspects of the articles. First, in previous semesters, it was clear that students were generally adept at understanding and interpreting the “big picture” of articles – the central purpose and conclusions. However, details of the design and analysis were often more challenging. Additionally, some research has shown that focusing on the methodological details initially helps students gain a deeper understanding of a research article (Hoskins, Lopatto, & Stevens, 2011; Hoskins & Stevens, 2009).

At the beginning of the semester, the in-class activities involved concrete instructor support for understanding the article. For example, for the first article assignment, all students answered questions about the methods and results of the article prior to the in-class activity. The in-class activity was a group-based discussion. Each group of students was responsible for agreeing upon an answer for one of the pre-discussion questions and then presenting it to the class. I provided concrete feedback for students on their answers, gave them an opportunity to ask questions, and guided students in linking the design of the study with the results. In the middle of the semester, for article assignment three (pdf), the in-class discussion was a more hands-on activity designed to engage students with the article using a little less direct instructor support. The third article had been challenging for students in past semesters, in large part because of the somewhat unconventional methods used. To help illustrate the methods in this study, for the pre-discussion assignment, students had to design and prepare actual test items similar to what was used in the study. For the in-class activity, students worked in pairs administering the experimental tasks to each other using the test items they had prepared. I led the group in pooling the class results, comparing our findings to those reported in the article and discussing interpretations. In this way, I provided less concrete support in terms of telling students the correct answers for independent variable, dependent variable, etc. Rather, the focus shifted to students engaging more deeply in the details of the article through the activity. For the final article assignment, as mentioned, there was no in-class activity. Students were expected to use the skills they had acquired over the course of the semester to independently interpret and summarize an article that was not discussed in class.


^Back to top^

Student Work

Fall 2011 – Holly Storkel
The GTA and I scored a subset of unit 3 papers. This course was part of a larger initiative on documenting learner outcomes. Personnel in charge of that initiative pseudo-randomly selected a subset of students (approximately 50% of the students enrolled) for double scoring. The average rating of the two judges was used to examine the percent of students demonstrating each level of proficiency in each skill on the rubric (see outcome results). In terms of understanding the basic content of the research article, the majority of students1 (i.e., 80% of students) showed emerging abilities. Common patterns were noted across student work. In particular, students had the most difficulty identifying and describing the independent variable and the analysis, whereas students tended to be successful in identifying and describing purpose, participants, procedures, and results. Students were a bit split in their success identifying and describing the dependent variable. This is perhaps not surprising, given that many of these students had not yet taken a course devoted to research methods or statistics (which is typically taken later in the curriculum), but it does suggest areas that need greater focus or instruction during the course.

In terms of critical thinking about findings reported in the articles students read, a range of skills were noted, with 30% of the students showing basic skills, 40% demonstrating emerging skills, and 25% mastering these skills. In general, students showed the most difficulty relating the article to topics discussed in class. This tended to be the only difficulty for those with emerging skills, whereas students with basic skills also tended to struggle either with drawing conclusions from the article or applying the findings to clinical situations. Overall, students may need more scaffolding and practice to relate articles to topics/theories discussed in class.

Although not explicitly taught in this class, writing skills also were assessed. In general, students showed strong writing skills, with all students demonstrating emerging or mastered skills in the areas of clarity, organization, and mechanics. This is encouraging, given that most students would have completed the writing sequence by the time they enrolled in SPLH 466. Students were clearly able to transfer their writing skills to this structured assignment.

Spring 2013 – Megan Blossom
A subset of student papers for the final article assignment (~40%) was pseudo-randomly selected for double scoring by the GTA and me. Papers were pulled from a sample of students that represented the full range of final course grades. Double-scoring procedures were identical to what is described above in the Fall 2011 section.

The central difference between the outcomes for Fall 2011 (Storkel) and Spring 2013 (Blossom) was that no students had scores in the absent or basic categories in the rubric for either understanding of article content or for critical thinking. Similar to the Fall 2011 outcomes, the majority of Spring 2013 students (86%) showed emerging knowledge of content. Once again, the main content areas of difficulty for students were identifying independent variables and describing the analytical approach of the article.

In terms of critical thinking skills, the sample of students was fairly evenly split between emerging skills (47%) and mastered skills (53%). This is a moderate improvement in skills over the Fall 2011 outcomes, where only 25% of the students showed mastery in the critical thinking areas of their summaries (see Figure 1 to see the comparison in student performance on the dimensions of understanding article content and critical thinking between semesters).

Clarity of writing and mechanics were not directly assessed in Spring 2013, so these portions of the rubric were not part of this analysis.

1 Because only a subset of students’ work was selected for double scoring in both implementations, these and all other results reflect the subset of student work. However, given that the subset was randomly selected and that the selected work ranged the spectrum of final course grades, conclusions are made to generalize to the entire class.


^Back to top^

Reflections

In general, students in SPLH 466 in Fall 2011 and Spring 2013 demonstrated a solid foundation of skills in reading and interpreting the content of research articles. Additionally, students in Spring 2013 showed stronger scores in the critical thinking areas in their final article summary than students in Fall 2011, suggesting that changes made in Spring 2013 were somewhat effective in providing the support needed for students to acquire this skill.

Overall, students seemed to enjoy the addition of the in-class activities, particularly the ones that were hands-on. We noted multiple advantages of the modifications to the incorporation of research articles. First, the hands-on activities forced students to engage with the methods of the article and often helped students identify exactly what was confusing or challenging about an article, leading to meaningful and helpful in-class discussion. Second, having students work in groups allowed the instructor to walk through the room and interact with students one-on-one. This was particularly helpful for the struggling students who otherwise would have been reluctant to ask a question in class or approach the instructor. The most successful in-class activities were those where students conducted miniature versions of the experiments and then analyzed the class data as a whole. There were two out of five where this was possible. This gave students an opportunity to see the analytical approach in action, which seemed to be more enlightening than simply reading the descriptions of the statistical tests used in the article, particularly given the fact that most of these students had little or no statistical background. Given that the analysis section and the independent variables section were the most challenging for students in both Fall 2011 and Spring 2013 classes, even more focus on the process of analysis during in-class activities and discussions might be a target area for further development.

Our collaboration demonstrated to us the importance of sharing experiences among instructors. This is particularly helpful when instructors rotate teaching responsibilities for a course or when groups of instructors teach different sections of a course. For example, Blossom was able to learn from the prior experience of Storkel through discussion of learner outcomes for Storkel’s offering. This allowed Blossom to make additional changes to the course through brainstorming discussions with Storkel. This shared understanding of past learner outcomes and the teaching approach that generated those outcomes clearly benefited the students, as evidenced by an improvement in critical thinking skills by students in Blossom’s class. Ideally, this process of sharing teaching approaches and learner outcomes will continue into the future so that student learning continues to grow even when course instructors change.

Contact CTE with comments on this portfolio: cte@ku.edu.


^Back to top^
Author: 

Portfolio & Poster Search




List of all portfolios & posters

One of 34 U.S. public institutions in the prestigious Association of American Universities
Nearly $290 million in financial aid annually
44 nationally ranked graduate programs.
—U.S. News & World Report
Top 50 nationwide for size of library collection.
—ALA
23rd nationwide for service to veterans —"Best for Vets," Military Times
KU Today