DIcipher: Assessing Drug Information Skills Using Serious Gaming Methodology in a Pharmacy Course—Cambrey Nguyen (2020)
A pharmacy professor incorporates active learning into a drug information (DI) course by designing an escape-room themed challenge called DIcipher. The challenge was utilized to assess DI knowledge while offering a realistic view of the application of these skills in clinical practice.
PHPR 630 (Drug Information and Literature Evaluation) is a one-credit hour, lecture-based course taught in the third year of pharmacy school. One of the functions of a pharmacist in any pharmacy setting is to provide patients and healthcare providers information about all types of drugs available. The course prepares students to develop drug information skills through answering questions about drugs using various resources and databases. In order to develop these DI skills, students must understand when to use various drug information resources and databases. In addition, students must retrieve information quickly and accurately in order to provide a recommendation relevant to patient care. The course requires critical thinking and evaluation of biomedical literature, which may be challenging and cause students to become disengaged. To overcome this challenge, I designed an active learning challenge called DIcipher to assess student learning of drug information concepts in an interactive way while encouraging student engagement.
There have been three iterations of DIcipher since 2017. The original version was piloted with fourth-year students undergoing the drug information advanced pharmacy practice experiential (APPE) rotation in Summer 2017. The first implementation occurred in Spring 2018 in the PHPR 630 course and incorporated changes and improvements obtained from the pilot. The most recent iteration in PHPR 630 took place in Spring 2019. The objective of the challenge is for students to work in small groups to solve puzzles to obtain drug questions pertaining to a patient who has been diagnosed with a rare infection. The goal is to answer all of the questions correctly in order to unlock a safe and retrieve the antidote within a 45- to 60-minute timeframe.
While “winning” DIcipher is meant to be a major challenge that only a few groups would be able to achieve, all students were able to apply drug information concepts presented in the course, increase accuracy and efficiency skills, and learn to work in a team under fast-paced conditions. Results from pre- and post-DIcipher perception surveys showed that many students expressed a higher level of comfort with using drug information (DI) resources after participation in DIcipher. The mean scores on the knowledge based post-challenge assessment also increased compared to the pre-challenge assessment. The students found DIcipher to be fun and challenging as well as a useful team-building experience.
Incorporating active learning through the use of serious gaming into my drug information (DI) course has been successful in engaging students with materials that can be difficult, dry, or uninteresting. DIcipher improved learning by giving students the opportunity to apply drug information concepts taught in class in a simulated environment representing a real-world scenario. In addition, DIcipher offered a way for me to assess student learning through observation of their thought processes as they researched answers using various resources. I recommend incorporating serious gaming as an active learning strategy to foster student engagement in a variety of disciplines and topics.
^Back to top^
PHPR 630 (Drug Information and Literature Evaluation), is a one-credit hour, lecture-based course taught in the third year of pharmacy school and requires successful completion of PHPR 629 (Research Design and Biostatistics). The course reviews the fundamental tools used to identify drug information in primary, secondary, and tertiary resources. In addition, students learn to assess published biomedical literature and utilize electronic drug resources to formulate a response to drug information questions from patients and healthcare providers. Following completion of the course, students will be able to understand the strengths and weaknesses of the drug information resources and apply drug information skills to clinical practice relevant to patient care.
Drug Information is an important course in pharmacy education, and students must be well trained in this area prior to practicing in any clinical setting. The course largely requires critical thinking with a heavy focus on reading and evaluating literature in addition to reconciling conflicting drug information found in various electronic resources. These aspects have been great barriers in the course and have made it difficult for students to meet learning objectives and apply their knowledge past the timeline of course assessments.
Understanding the importance of connecting the students to the materials, I decided to pursue pedagogical alternatives to increase both engagement and retention in the classroom setting. Because PHPR 630 is a third-year course preceding the fourth-year pharmacy experiential rotations, these changes will help students to build critical skills for their rotations where they are expected to demonstrate satisfactory DI skills including fluency in electronic resources (e.g., PubMed).
To achieve my goal of increasing student participation and engagement in my PHPR 630 courses, I began incorporating active learning activities, in the form of team-based games, into the course curriculum. Specifically, I designed an escape-room game called DIcipher to assess DI skills. While serious gaming has been recommended by the Academic Affairs Committee of the American Association of Colleges of Pharmacy (AACP) as a novel pedagogy that is beneficial for student development of critical thinking skills (Cain et al. 2014), there are currently no published studies that look at the use of this method to teach drug information. I have created three iterations of DIcipher, and the original version was piloted with fourth-year students undergoing the drug information advanced pharmacy practice experiential (APPE) rotations in Summer 2017. The challenge was later adapted with improvements and changes based on feedback from the pilot to the PHPR 630 (Spring 2018) as the second implementation. The most recent iteration included a knowledge assessment component along with capturing efficiency of answering questions in PHPR 630 in Spring 2019.
^Back to top^
The idea for the challenge revolved around three overarching learning objectives: confidence in DI knowledge, efficiency, and teamwork. The challenge called on using DI skills to self-teach a condition or disease state not previously taught in pharmacotherapy sequences or other courses in the pharmacy program. The challenge tested students’ confidence in providing recommendations based on the electronic resources they used. This emulates real-life scenarios in pharmacy practice, where pharmacists may come across a drug or disease that is new or unfamiliar and must provide a recommendation quickly and accurately. A time limit applied in the challenge served as a measure of efficiency. Students were required to quickly and efficiently look up information for each of the five DI questions with time reserved to solve the puzzles to find the location of the DI questions. With more practice and greater familiarity with the resources, efficiency tends to increase when students understand the DI resources and when to appropriately use them for certain types of questions. Being efficient and quickly retrieving drug information is an important skill to have in practice. The challenge required students to work as a team through delegation of tasks (retrieving clues, solving clues, researching DI questions), communicating information to each other, and problem solving (resolving discrepancies if the combination did not open the lock box). This aspect of the challenge provides a realistic view of working with other pharmacists and when to ask for a second opinion.
Figure 1. Drug Information skills include multiple elements such as confidence in knowledge, problem solving, critical thinking, self-teaching; therefore, these skills were included in the challenge.
After developing the challenge, I piloted DIcipher in Summer 2017 with 15 fourth-year pharmacy students enrolled in a one-month summer DI rotation at the KU School of Pharmacy. The challenge measured whether the students could apply what they learned about DI resources to a new set of questions. In this initial implementation of DIcipher, students were provided with instructions for the challenge and the patient case that contained signs and symptoms, a list of current medications, and other health conditions. The challenge consisted of five puzzles or riddles, and solving each puzzle would lead to the location of the DI questions in the room (i.e., solve puzzle #1 to find DI Question #1 and so forth).
Figure 2. During DIcipher, students SOLVE puzzles to obtain the drug information questions pertinent to the patient case. The students use their drug information skills to RESEARCH the answers to RETRIEVE an antidote that will ease the symptoms of the fictional patient.
The drug information questions were centered on the patient case (preferred treatment, drug interactions, alternative treatment due to allergies, natural products, and vaccination) and obtaining the correct answer to the DI questions would reveal a number for the combination to the lock box (Table 1). The clues and DI questions were hidden under tables, chairs, books, and other decorative items staged in the room. The participants had 60 minutes to “complete” the challenge by unlocking the box to obtain the antidote. The challenge took place in a classroom that had live stream video and capability to talk to the instructor if hints were requested to solve the clues. I observed the group, kept track of the time, and recorded the number of DI questions the group answered correctly. The room was staged with items available such as books, posters, mortar and pestles, and other items already available in the classroom (gurney, table, chairs, bookshelf, clock, sharp containers, and trash cans). The students were not locked in the room, unlike actual escape room games (Figure 3, 4).
Figure 3. Students working to solve clues and answer questions during the second iteration of DIcipher (University of Kansas Lawrence campus, Spring 2018).
After a successful pilot, I decided to implement DIcipher as a core activity for my Spring 2018 PHPR 630 course (Figure 3). Based on the fourth-year students’ feedback, there were several changes implemented for adaptation of DIcipher in the drug information course. Students were allowed three attempts to “unlock” the box, and a safe with a 10-digit entry combination was used rather than a lock with five dials. These two changes discouraged students from attempting to open the box by physically adjusting the dial to find the correct combination as opposed to working through the DI questions. Additional changes included reducing the time limit from 60 minutes to 45 minutes, as the DI course is a one-credit hour class and the room was staged to resemble a quarantined hospital room to fit with the rare infection case. Class size was also much larger, with 136 students enrolled in the PHPR 630 course. The students were randomly assigned into small groups of five, and a total of 27 sessions of DIcipher were run with three rooms simultaneously occurring in the same hour. One session was composed of the 45-minute DIcipher challenge, followed by a 15-minute debriefing session to discuss and review answers to all of the DI questions in the challenge. I also drove to the Wichita campus to run the DIcipher challenge with 20 students in the course there. All sessions were conducted over the course of two weeks, and students were able to sign up for a time period that fit their schedules. One key aspect of this iteration was that I did not relay to students specifics of the challenge prior to it. The spontaneity of the activity fostered an environment where students would be open to taking part in something that was active and new and set the appropriate atmosphere for an emergency situation in a quarantined hospital room.
Figure 4. The items in the room are staged to represent items found in a hospital pharmacy setting that include a sharp’s container, gloves, IV bags, and drug references.
Similar to the pilot, students were given the same patient case, clues, and puzzles, and they had to use their DI skills to find an “antidote” for the patient. Students were allowed to access books and electronic DI resources on their laptops or iPads to research answers to the questions in the challenge. I watched the group remotely using a live video stream, which allowed me to observe students, record when questions were answered correctly, and offer help when students requested it for a certain puzzle. Video cameras, combination locks, safes, and staging props (e.g., mortar and pestles, posters, etc.) were funded through a Center for Teaching Excellence Course Transformation Grant. This setup is similar to a “real” escape room game, where participants must sort through and solve clues to “break out” of a room. There were no changes made to the setup of the DIcipher challenge for the Spring 2019 course.
^Back to top^
One of the key components of the challenge was to determine how many questions the students would be able to answer within the given timeframe. In Summer 2017 (pilot of DIcipher), only one group (five students out of 15) was able to complete the challenge by answering all five DI questions correctly and retrieve the antidote. During the challenge, the questions that students struggled with the most were the natural product and vaccination questions. All of the groups were able to find the correct answers for three out of the five DI questions featured in the challenge. In Spring 2018, no group (27 groups and 136 students) successfully completed the challenge. The students in the Spring 2018 course answered a maximum of three out of five questions correctly. In Spring 2019, no groups answered all five questions correctly; however, one group correctly answered four questions out of five. The majority of the groups also struggled with the natural product and vaccines questions featured in the challenge.
At the start of the semester in Spring 2018, students enrolled in the PHPR 630 course were asked to fill out a self-assessment to rate themselves on 13 specific core DI skills (Figure 5). The self-assessment results showed that the majority of the class believed that their degree of confidence relating to most of the 13 core objectives was “fair” prior to the first day of class and months before participating in DIcipher. The largest number of students felt most confident with skill 7 (knowing when to ask for a second opinion) and 9 (providing unbiased responses to patients), but least confident with skills 5 (having the skills to answer a DI question under time pressure) and 11 (discussing primary literature articles).
Figure 5. The following skills were self-assessed by the students in the PHPR 630 course: 1) Know the appropriate questions to ask patients when providing DI support? ; 2) Have the skills needed to answer DI questions?; 3) Can determine the best resource to use first?; 4) Have the skills to read and interpret the references you identify?; 5) Have the skills to assist those who are in a hurry?; 6) Have sufficient therapeutic knowledge to evaluate the appropriateness of the literature?; 7) Know when to ask a colleague for a second opinion?; 8) Can ask open ended questions to get the information you need to answer a DI question?; 9) Can provide unbiased responses?; 10) Are able to provide adequate DI responses when time is limited?; 11) Can discuss primary literature articles?; 12) Can provide complete and succinct responses?; 13) Can break down difficult drug information to patients?
In addition to the self-assessment on specific DI skills, students were given a five-point Likert-scale survey where they were asked to describe their overall comfort level in using DI resources to answer questions. In the pre-survey, most students reported moderate levels of comfort in their use of DI resources. When students took the survey following their participation in DIcipher, students reported higher overall comfort level in using DI resources, although this difference was not statistically significant (paired t-test, n=53, p=0.26; Figure 6). There was a 65% (88/136) response rate for the post-survey.
Figure 6. Student comfort level when using drug information (DI) resources, based on results from a pre-survey (light blue) and post-survey (dark blue). After participating in the DIcipher active learning activity, students assessed themselves as feeling overall more comfortable using DI resources.
In Spring 2019, the students were administered a knowledge-based assessment before and after the DIcipher challenge. The five questions in the assessments were similar to the questions featured in the challenge and evaluated students’ ability to follow a process in obtaining the correct answers for specific types of questions. Both assessments were “open-book” so that students could access resources and databases available through the Internet and research answers to questions. Of the 139 students enrolled in the class, 136 students took the pre-test, and the mean score was 68.1 + 25.9. A few weeks after the challenge, a total of 138 students took the post-test, and there was a statistically significant increase in the mean scores at 76.4+ 19.7 (P<0.05) with 65 students obtaining higher scores on the post-test. This may suggest that the DIcipher challenge improved student learning.
Efficiency is also a critical aspect of drug information skills and looks at the time spent to navigate through resources to retrieve answers to questions. This aspect is important in clinical practice settings, as pharmacists must work quickly to find information about a drug in order to make a recommendation. In the pre and post assessments administered in Examsoft, the software provided an average time to answer each question included in the assessments. It is expected for students to become more familiar with resources and find answers to specific types of DI questions, which would show an increase in efficiency. For the pre-test, the mean time to response ranged from 10:59 minutes to 18:54 minutes. As for the post-test, the mean time to response for each question ranged from 7 minutes to 14:41 minutes, with a total mean time of 9.5 minutes for all five questions. Based on the information provided by the Examsoft system, there was an increase in efficiency in answering the DI questions after the activity as shown by the decreased time spent on each question. The reason behind the increase in efficiency may be that students were more familiar with the resources through practice in the challenge so that they were able to work through the questions faster individually on the post-test.
All of the iterations of DIcipher gathered student perspectives regarding the design and implementation of the challenge which was separate from the knowledge-based assessments and self-assessment of DI skills. Surveys pertaining specifically to student opinions of DIcipher as an activity were distributed after both the Summer 2017 pilot and the PHPR 630 course in Spring 2018 and Spring 2019. Overall, students agreed with the statement that the challenge promoted active learning while also assessing knowledge of DI resources, and they considered the most difficult aspect of DIcipher was answering the DI questions (Table 3). Students reported that DIcipher was challenging because it tested DI knowledge in a novel way, required them to work under a time limit with a group of their peers, and it uncovered areas of their DI skills that needed improvement (Table 3). Student comments recorded as part of this post-survey were overwhelmingly positive, with one student reporting that “[the challenge] might be one of the most enjoyable activities I have done in pharmacy school!” Another student commented on the effectiveness of the game for attaining course learning objectives: “[this was] a better way to test my knowledge than an exam.” The challenge received positive student feedback because of the novelty in using the serious gaming methodology for the purpose of applying DI knowledge, critical thinking, and teamwork.
^Back to top^
The initial development of the patient case, clues, puzzles, and room staging for the DIcipher pilot in Summer 2017 required approximately 20 hours of preparation time. The adaptation of DIcipher for PHPR 630 was spread throughout a period of six months. The setup took six hours with the help of a fourth-year pharmacy student. DIcipher required some initial financial investment for the purchase of safes, combination locks, and hospital room themed decorations; all materials can now be re-used for future implementation of the challenge.
As an instructor, watching students research DI questions on an unfamiliar topic through a live video feed enabled me to better understand their learning processes. The challenge allowed for observation of how students worked with one another to solve the puzzles, discuss and compare answers to the DI questions, and work together to complete a task. In addition, the challenge was able to reveal students’ thought processes on how they go about approaching DI questions which traditional quizzes or exams may not be able to do. The challenge broke the monotony of traditional methods of assessments such as quizzes/exams and provided the students multidimensional learning opportunities such as identifying resources/types of DI questions they have or have not mastered, problem solving when they were unable to find the correct answers initially, communicating and collaborating with other students in the group, and efficiently finding answers to DI questions. DIcipher also gave the students an important opportunity to learn from their mistakes without harming a real patient.
The success of DIcipher has inspired me to continue to expand active and novel learning modules in the Drug Information class. I will continue to incorporate the challenge into the PHPR 630 curriculum, implementing minor tweaks each semester (e.g., writing new patient cases). In addition, the challenge has helped me shift and change materials to provide more focus on areas that students have struggled with and need improvement on throughout the semester.
In summary, my experience in the PHPR 630 course suggests that escape-room activities and other team-based games can be useful tools for increasing student enthusiasm about a variety of class topics that may not be initially interesting. Games offer students an engaging and novel learning environment where they are challenged to actively apply concepts in solving a tangible, real-world problem.
A Course Transformation Grant provided by the University of Kansas Center of Teaching Excellence to purchase materials for the challenge in the drug information course. IRB approval for both studies was granted by the University of Kansas Human Research Protection Program (00141070 and 00141872).
Contact CTE with comments on this portfolio: email@example.com.
^Back to top^