Building KU's Teaching and Learning Community

Using Evidence-Based Principles in Clinical Practice—Holly Storkel (2005)

childrenOverview

To help students better understand how to use evidence-based principles in clinical practice, Holly Storkel revised her SPLH 880 course so that students can access evidence from clinical research and apply it to specific patients.

Background

By revising SPLH 880 Seminar in Speech Language Pathology: Clinical Treatment of Phonological Disorders, I increased students usage of evidence-based principles in clinical practice. In the past, I have been disappointed in the ability of students to adequately use academic literature without my intervention. They needed to increase their own ability to deeply understand professional scholarly research that forms the basis for phonological disorders treatment. In the future, I plan to again revise the course to further develop students abilities to judge the severity of a departure from standard research practices and integrate skills learned in the course. In addition, I will refine the grading rubric to define intermediate levels of performance for students case reports.

The available evidence is continually changing as research progresses and advances are made in our understanding of the nature of sound delays in young children. The most effective clinician is the one who has the most up-to-date evidence. In the 2004 version of this course, students appeared to understand course topics, but that assessment did not measure their ability to find the research evidence independently. Also, they had difficultly synthesizing, integrating, and applying the evidence to clinical practice, a skill that is essential if they are to keep up with the field.

Implementation

A series of student behaviors that I observed in the 2004 offering led me to re-evaluate my current teaching plan: I found that students were relying on me to walk them through the scholarly literature, and they had difficulty synthesizing, integrating, and applying the evidence. My goal for the 2005 offering was to increase student responsibility for seeking out information on their own and applying information to clinical cases by revising the course goals, student objectives, and course activities.

To accomplish this, I developed new materials in four areas that I believed would increase student learning: improve online reading quizzes, identify search terms for scholarly readings, adapt three clinical cases for in-class application of research evidence, and adapt 17 clinical cases for assessment of student performance.

Student Work

I established three criteria that I used to evaluate the impact of the new teaching methods on student learning. First, students completed reading quizzes for each assigned reading prior to class discussion to examine their independence in consuming research. Second, students independently created treatment plans for two different clinical cases. Finally, I compared the results from the revised 2005 course to that of the 2004 course.

Reading quiz comparisons showed higher performance in 2005 than in 2004, indicating that students ability to independently read and evaluate research improved. This translated into less class time devoted to understanding basic aspects of each article and more class time devoted to discussion of the merits of each article and application to clinical cases.

In 2004, even students earning an A had difficulty integrating and applying all the evidence discussed in class to their assigned clinical case. Students earning lower grades had even greater difficulty integrating and applying evidence and also demonstrated fundamental misunderstanding of some pieces of evidence.

In 2005, students had two opportunities to create treatment plans for different clinical cases. Their first attempt at creating plans yielded performance similar to the 2004 students. Students were still having difficulty integrating and applying evidence. This prompted a clinical grand rounds where possible treatment plans for each case were discussed with a specific focus on integrating evidence. The second and final attempt at creating treatment plans showed marked improvement in the ability to integrate and apply evidence to clinical cases.

Reflections

The change in the approach to the readings was valuable. Students benefited from explicit instruction on how to read and evaluate a research article, and the consistent format of the reading quizzes helped students internalize this framework. However, students still need greater experience judging the severity of departures from typical research practices. That is, they do not know when a particular weakness is fatal versus a minor flaw.

Greater emphasis on in-class application of evidence to clinical cases helped students appreciate the link between research and clinical practice, giving them a more in-depth understanding of how to apply specific pieces of evidence to clinical practice. However, integration was still problematic until specific feedback and discussion was provided through written case reports. The approach to in-class applications may need to be revised to continually demonstrate integration of old topics with new topics. Development of a grading rubric also may be helpful in communicating this goal to students.


^Back to top^

students working in classBackground

Evidence-based practice refers to a style of clinical practice where a clinician routinely consults the evidence from clinical research and applies this evidence to a specific patient. The clinician then closely monitors the patient experiences to evaluate whether or not the changes resemble those reported in the research literature.

The available evidence is continually changing as research progresses and advances are made in our understanding of the nature of sound delays in young children. For this reason, the most effective clinician is the one who has the most up-to-date evidence.

In the 2004 version (pdf) of the course, students reviewed the available evidence. They looked at treatment targets and treatment methods to promote the greatest change in sound learning for children with significant delays in speech sound acquisition. However, they did not appear to gain independence in their ability to locate, evaluate, and apply evidence to clinical cases.

Why I made changes
Students appeared to understand the course topics as 100% of them passed a comprehensive online exam. However, I noted problems beyond the exam’s coverage:

  • Students didn’t find the research evidence independently; instead, I provided the evidence and guided them through it.
  • Students had difficultly synthesizing, integrating, and applying the evidence to clinical practice.

Key course goals for 2005 course (pdf)

  • A thorough understanding of the current evidence.
  • The ability to locate new evidence through literature and Internet searches.
  • The ability to evaluate new evidence independently.
  • The ability to synthesize and apply new evidence independently to clinical cases.

For more information on the material studied in this course, please check out the KU Word and Sound Learning Lab.


^Back to top^

Professor Storkel lecturing during classImplementation

Revised goals:

  • Increase student independence in locating, evaluating, synthesizing, and applying current evidence in order to develop life-long learners.
  • Incorporate a greater emphasis on in-class application of evidence.

Revised student objectives:
Students will demonstrate that they can:

  • Perform effective search strategies to locate relevant evidence;
  • Efficiently identify key information in the obtained sources;
  • Critically evaluate the evidence; and
  • Apply evidence to clinical cases.

Revised course activities:

  • I devoted a class session to a discussion of how to find research evidence and created a supporting handout (pdf) with the help of Judith Emde, a KU librarian. For subsequent class sessions, students attempted to locate research articles on the topic being discussed in class. Students reported the search terms and databases they used and the number of relevant articles found via an online quiz prior to class. This gave students 12 attempts at locating research articles on topics related to treatment of children with phonological disorders.
  • I devoted a class session to instructing students on how to read and evaluate a research article with a supporting handout (pdf) and walked them through a research article to demonstrate where to find key pieces of information (e.g., the paragraph before the methods section usually summarizes the experimental questions). Students used this framework when completing online reading quizzes prior to class discussions of the course articles. This gave students 12 attempts at understanding and evaluating a research article independently.
  • To help students compare and contrast the different treatment packages reviewed in the second half of the course, I developed a handout (pdf) to help students know what components to look for when reading about a given treatment.
  • I adapted three clinical cases from my research program and used these to demonstrate application of the research evidence during each class. For the first case I demonstrated how the research evidence would influence my approach to the case, and then students participated in a discussion of how the evidence could be applied to the remaining two cases. The three cases represented a range of difficulty from clear diagnosis of sound delay to less clear diagnosis of sound delay. Likewise, treatment options ranged from only a few valid treatment options to a variety of valid treatment options.
  • I adapted an additional 17 clinical cases from my research program for assessment purposes. These cases represented a range of difficulty from clear diagnosis of sound delay to less clear diagnosis of sound delay. Likewise, treatment options ranged from only a few valid treatment options to a variety of valid treatment options.

CTE support
The time that I needed to develop the new materials was financially supported through the CTE Faculty Fellowship program. I had applied for and received a Fellowship in 2004 to work on the questions that had arisen about this course. I decided that the best use of the funds would be to hire a student who would develop the new materials. Through this pedagogical and financial support, I was able to enact the changes I believed would best improve students’ learning.

Revised course activities rationale
Tutorial on reading a research article.

  • To help students learn where to find key information in a research article.

Online reading quizzes prior to class discussion.

  • To ensure that students have completed the readings prior to class.
  • To decrease the amount of in-class time spent on first exposure to concepts in readings.
  • To increase student independence in locating key information in research articles.

Review of assigned readings based on quiz performance.

  • To ensure that students fully understand the evidence.

Generation of search terms used to find the assigned readings.

  • To teach students which search terms are useful.
  • To have students evaluate whether the instructor has presented a biased view of a particular topic.
  • To demonstrate the value of evidence-based practice by having students summarize prevailing views at different points in time. (For example, if it were 1990, would you have evidence about this topic? How does it differ from the evidence we have now?)

Clinical case application.

  • To ensure that students understand evidence well enough to apply it to a case that is highly similar to the research article cases.

^Back to top^

Students working in classStudent Work

Reading quizzes
In 2004 and 2005, students completed reading quizzes prior to class to prepare for in-class discussion. In 2005 two changes were made to the online reading quizzes. First, I provided a more consistent framework for understanding and evaluating research articles. Second, I reduced the number of articles read in detail to one article, rather than two – three articles in 2004.

As shown on the Reading Quiz Performance Graph the changes I made in 2005 appeared to improve student performance on reading quizzes. In 2004, 25% of students earned an A average on reading quizzes, 50% of students earned a B, and 25% earned a C. In 2005, 86% of students earned an A average with the majority of these being an A+. The remaining students earned a B+ average on reading quizzes. Moreover, student comments in class and on course evaluations suggested that they transferred these skills to other assignments beyond my course. Several students commented that they used the handout that I provided to read articles for their masters theses or for other courses.

This higher performance on reading quizzes translated into more in-depth in-class discussions in 2005. Students came to class with a firm understanding of the design and results of the research study, so this basic information rarely required review. Students also came to class with ideas about the strengths and weaknesses of the study. This resulted in more extensive discussion of the merits of each study as well as a more general discussion of what is “good” clinical research.

Clinical cases
In 2004, students orally presented a clinical case at the end of the semester, providing both a diagnosis and a treatment plan. In 2005, students provided a diagnosis and a treatment plan for a clinical case in written form at the midpoint of the semester. They were given written feedback and a class session was devoted to a group discussion of each case. In 2005, students also orally presented a second clinical case at the end of the semester, following the same format as the written case.

As shown on the Clinical Case Performance Graph the changes I made in 2005 appeared to affect student performance. In 2004, 38% of students earned As, 50% earned Bs, and 13% earned Cs. Performance in 2005 at the midterm written case was similar to 2004 with 29% As and 71% Bs. In contrast, final performance on the oral case in 2005 showed marked improvement with 86% As and 14% Bs. Completing the written case, obtaining specific feedback on that case, and the in-class discussion of the written cases seemed to help students understand how to integrate and apply research evidence to clinical cases, which they were then able to do independently on their final case oral presentations.

Closer examination of case discussions shows further qualitative differences between 2004 and 2005 performance. In 2004, even students earning an A had difficulty integrating and applying all the evidence discussed in class to their assigned clinical case. Students earning lower grades had even greater difficulty integrating and applying the evidence and also demonstrated fundamental misunderstanding of some pieces of evidence. In the 2005 written cases, students earning an A- were able to integrate and apply multiple pieces of evidence to their assigned clinical case. Small problems in accuracy were still noted for these students. Students earning Bs on the written cases showed similar problems as the 2004 students, demonstrating problems with integration and application as well as fundamental misunderstandings. In the 2005 oral cases, students earning an A considered all evidence reviewed in the course and accurately applied this evidence to their clinical case. These students may have shown small confusions that were resolved with questioning during their oral presentation but on the whole, their integration and application of evidence was solid. Not only did the number of students earning a grade of A increase from 2004 to 2005 but the quality of student understanding indexed by a grade of A also improved.

Samples of presentations with instructor feedback

2004:

2005:


^Back to top^

Professor Storkel lecturing during classReflections

Impact on student learning
The change in the approach to the readings was valuable. Students benefited from explicit instruction on how to read and evaluate a research article, and the consistent format of the reading quizzes helped students internalize this framework. The strong performance on the quizzes suggested that students mastered the ability to read and evaluate a research article by the end of the course, and anecdotal evidence hints that they may have even transferred this skill to other topics or courses. Not only is this greater independence in consuming research important for lifelong learning, but it also allowed for more class time to be devoted to discussion of the merits of the research and the application of the research to clinical cases. In focusing class discussion on the strengths and weaknesses of each study, I learned that students have difficulty evaluating the severity of a given weakness. Specifically, students had some ideas about what constitutes a “good” research study so they could identify potential shortcomings; however, they had trouble weighting these shortcomings. Is the problem so severe that we completely doubt the validity of the findings? Or can we look at the data and determine that the problem didn't compromise the findings? In the next offering of this course, I plan to spend more time discussing how to determine the severity of a given departure from “best practices” in research. In addition, I think this issue could be addressed further in undergraduate coursework. Although our undergraduate students complete a course on general research methods, it may be useful to incorporate discussion of research methods in content specific courses so that students can see how these general research issues play out in specific content areas.

Demonstrating application of the evidence to clinical cases in each class helped students see the link between research and clinical practice. It also gave them a more in-depth understanding of how to apply that specific piece of evidence to clinical practice. However, it did not improve their ability to integrate multiple pieces of evidence. On the midterm written case reports, 2005 students showed similar integration problems as the 2004 students. This midterm attempt at clinical application and the subsequent feedback and discussion did help students learn how to integrate multiple pieces of evidence to specific cases, yielding improved performance on the final oral case. My approach to in-class application of evidence to clinical cases may need revision. Specifically, in 2005, we applied each piece of evidence to the clinical cases independently, rather than integrating evidence from the previous class with the current class. Students may learn how to integrate evidence better if each piece of evidence were layered on top of the next in the in-class application demonstrations.

Another method that may help students understand that integration is expected is the development of a grading rubric that emphasizes this point and could be distributed prior to case report deadlines. Although I have an internal framework for judging the quality of student case reports, I have not solidified this framework, nor have I shared it with students. For the past two offerings of the course, I avoided creating a rubric because I wasn't sure what the students were capable of. Thus, the grading was always a relative scale with students differentiated relative to one another rather than compared to a gold standard of quality. Now that I have seen what students are capable of and have adjusted my teaching methods to steer them towards the outcome I want, it is time to develop a rubric to better communicate course standards to students. I have constructed a preliminary rubric (pdf) that lists the traits and what I would like to see for each trait. Thus, if every item were checked on this scale, the performance would be considered “perfect.” I need to refine this scale to determine intermediate levels of performance for each trait because it is unlikely that every performance from every student will meet this gold standard.

Next steps for 2006:

  1. Provide specific instruction in how to judge the severity of a departure from standard research practices.
  2. Change the approach to in-class examples to layer each new piece of evidence on top of the old evidence to foster the development of integration skills earlier in the course.
  3. Refine grading rubric by determining intermediate levels of performance for each trait.

Contact CTE with comments on this portfolio: cte@ku.edu.


^Back to top^

Portfolio & Poster Search



List of all portfolios & posters