Using Collaborative Tools to Deepen Understanding of Research Methods—Mugur Geana (2010)

Research II: Methods (JOUR 802) is designed to introduce students to basic methods of marketing and media research. This course is required for graduate students in Journalism and is typically taken by students in their first year. At this stage, most students have no experience in research, and this course, along with Research I: Theories, is their first exposure to scientific research in journalism and strategic communication.

Through this course, I want students to not only develop a mastery of knowledge about research methods, but also to apply them in their own research, thus enabling them to solve pertinent research questions and analyze results. I want students to start thinking like researchers: to understand the importance of research, to think in terms of cause-and-effect in order to formulate research questions, and to learn how each research question can be answered through specific research methods.

At the end of the course students are expected to achieve the following goals:

Know-How: Learn concepts and methods of strategic communication research, and enhance understanding of how research fits within the broader framework of media studies.

Hands-on Experience: Hands-on experience in designing and conducting research, in interpreting research results, and in understanding how different research methods can work together to provide a more accurate image of the researched topic.

Professional Career: Prepare students with refined abilities to present and defend complex research-based strategic communication decisions in a clear and concise manner, and concomitantly help in developing appropriate research for a project or thesis.

One of the most challenging aspects of teaching this class has been to get students to understand and appreciate the value of research. Most students do not see a course on research methods as a prerequisite to being a successful mass communication graduate student or as a journalist. Students don’t see the course as being immediately applicable to their professional goals. Previous experience suggests that while they understand the theoretical components of research, they have trouble formulating research questions and applying their theoretical knowledge of research methods to answer those research questions. To get them to start thinking about the usefulness of this course and the relevance of research, I often give students an example of the process that an autoworker must undertake to fix a broken engine.

Step 1: Selecting the research question: Identification of the problem

Step 2: Identify appropriate research methods: Selecting the right set of tools

Step 3: Design application and analysis plan: Knowing how to use the tool(s) and what measurements to take to make sure the tool is working

Step 4: Data analysis: Analyzing and interpreting the results of applying the tool, e.g., How useful was this tool? Can I use it again? What was the outcome?

In my course structure I focus on both mastery of knowledge and application of research skills. Thus, each class is a combination of learning research methods and learning how to write a research proposal. I discuss different components of the proposal, bringing in good and bad examples and how such problems need to be approached. This is done on a continuum, starting with the literature review and ending with research methods.

To successfully complete the course, students must demonstrate knowledge of research methods, as well as apply these methods in their own research. Thus, assignments are designed to measure both theoretical understanding of and direct application to a research project. In my previous offering of this course, I assessed students’ knowledge of research methods and learning involvement pre and post course; the results showed an improvement in theoretical knowledge and increased confidence in using what they learned. However, in the first offering of this course, I found that students had a hard time applying research methods when designing the research for their final paper; there was a disconnect between perceived learning and actual learning. Students thought they understood the methods but had a hard time identifying and operationalizing research questions.

Unfortunately, students’ often think about a research method they like or understand best (content analysis, for example), and then try to build a research question around it. Nevertheless, when confronted directly with a research question, they stumble, and are not sure what the best method is to get the appropriate answer. Although it can be argued that an answer can be obtained through several research methods, there is always one that provides more comprehensive or accurate results. I realized that the difficulty students have with learning research methods is that they tend to think they have a preferred research method and usually stick with it, while discarding or giving minimal importance to other methods. Because of this, they often lack the skills that would allow them to compare different research outcomes from the perspective of a set research methods. Since the first offering of this course (Fall 2008), I have made many changes across two iterations—Fall 2009 and Fall 2010—in order to better scaffold, or support, the development of the research skills required to achieve the course goals.

To increase student engagement and learning, I introduced a wiki assignment in Fall 2009. Each week I assigned two students to serve as wiki editors for the specific topic. The editors were required to prepare a wiki entry based on the readings and class discussions. All other students were expected to read and contribute to this wiki entry. At the end of the week, the editors had to edit and prepare the wiki entry in its final form. The editors also needed to submit their reflections, stating what they learned while working on the wiki. Thus, students not only got an opportunity to apply their readings but also learned to incorporate multiple perspectives and collaborate with other students. This exercise also facilitated interpersonal and team building skills.

I found the wiki to be useful as an assignment, as well as a teaching resource. To further engage students, I changed the wiki assignment in Fall 2010. Besides summarizing class material, students had to find supplemental information that was not included in the readings or discussed in class. I wanted students to be responsible for their learning by using self-selected resources from Tier 1 journals and improve their information-seeking skills.

In addition to changing the wiki assignment, I also made some changes to the final project. Students had to come up with a research topic of interest in the beginning of the semester. During the course of the semester, they were expected to come up with a research question and identify the appropriate method to answer that question. The final project was the culmination of semester-long learning on research methods. The development of this research project was done through several drafts during the semester. I used rubrics to make my expectations clear to students and to assess their work.

Although I maintained this approach from the first iteration of this course (Fall 2008), to encourage students to participate in their own learning in Fall 2010 I changed the way I gave feedback to the students, from making suggestions to asking questions that challenged them to consider the validity of their approaches, as well as encouraging them to look at additional sources to justify their work. By posing challenging questions, I wanted students to take an active part in their learning process, turn to outside sources to find solutions to the questions, and defend their ideas in a critical and concise manner.

At the end of the course, I wanted students to not just understand the theoretical and practical components of the research process but also hone their critical thinking and team building skills. I have found that the peer review process affords an environment for students to apply their knowledge of research to critically appraise and evaluate their peers’ projects. Further, it enables students to work together and respect others’ ideas and work. To encourage students to review their peers’ projects, I increased the number of class sessions in which we discussed the projects. Students could intervene and make suggestions or ask questions in an exercise of peer review and enhance their abilities to present and defend their research ideas. Further, at the end of the semester, students were given a rough rubric to evaluate their peers’ research presentations. I found that the rubric also helped in the preparation of their presentations.

Finally, I also changed the general structure of the classes. In the previous offering of this course, students were expected to present the readings every class. The wiki editors were assigned to make class presentations on the weekly readings. However, I found the presentations to be inconsistent in terms of presentation skills, amount of information delivered, and the delivery method; in addition, students didn’t really like being “lectured” to by other students. To homogenize the discussions on the learned material, I replaced student presentations with class discussions in Fall 2010.

There were several indications that the latest course modifications were successful. These indicators included student performance on the final project, the wiki assignments, and student engagement in class.

Wiki assignments

By asking students to incorporate information from additional resources, students were now actively involved in their learning, thereby making them more involved and engaged. Furthermore, by looking up additional resources, students were participating in the research process: the wiki assignment served as a tool through which they could demonstrate their research knowledge and skills. Additionally, this helped students to consider various research methods in designing their own proposals and justifying their reasons for their chosen method.

Overall, I was very impressed with their ability to synthesize information from the course work and from outside sources. Students presented relevant work from outside sources, in addition to summarizing course work.

Wiki assignment—excellent (pdf)

Wiki assignment—good (pdf)

Further, students reported that the wiki served as a useful resource for referring to while designing their research proposal. As one student reported,

Sometimes I am not sure I completely understand the language of [book specified in class], so the wiki became a helpful, alternative tool of collective thinking. To my mind, such discussions were a good supplement to the main content.

Final project

I observed a much-improved ability to consider diverse research methods and justify their selection. Rather than relying on the instructor to provide all the answers, students turned to outside resources such as journals and books to find information. By posing more challenging questions in my feedback, students were forced to turn to different resources and justify their claims and arguments. Students were able to defend their choice of research method selection, selection of sample, and method of data analysis.

Final project—excellent

Instructor’s comments on final project—excellent

Final project—good

Instructor’s comments on final project—good

Class discussion

I always had a set of questions on the weekly readings that I asked to engage students in discussions. Being a relatively small class, it was easy to keep track of who was engaged with the topic and who was not, and I would make a habit of questioning those who preferred to keep silent. After the initial class meetings, everybody learned that there was no way of avoiding questioning, so they were more scrupulous in preparing for class. Students took an active role in discussing the readings and brought in perspectives from various sources. By devoting more class time to their final projects, students got several opportunities to present and defend their research ideas. This further strengthened their ability to justify their ideas and critically appraise their peers’ ideas.

I was very happy with students’ improved research and presentation skills as evidenced in their work. Therefore, this is an approach that I will continue to use. Students were able to compare research methods from the perspective of their research question, and they improved in their ability to consider the most appropriate method for the question at hand. Four students used these projects to develop their master’s theses.

However, there are still several areas that need further refinement and exploration. The structure of the wiki was not entirely consistent during the semester. Depending on which students were the weekly editors, sometimes there was greater emphasis given on the summary of the methods and less on the applications. Since one of the main goals was the application of theoretical research to real-world settings, this was a problem. In future offerings, I plan to decide on the wiki structure beforehand, rather than leaving it to the editors. At the same time, I want them to develop and sharpen their own problem solving skills so that they don’t need to rely and depend on the instructor for directions. Using a well-structured rubric will allow me to communicate my expectations to them without leading them to depend on me. I also plan on making the peer review process more structured. Students were not consistent in the ratings of their peers. This could largely be a result of the rubric, which was not very specific. In the future, I plan to divide students into groups of three and have peer review committees to assess each presentation. Further, I plan to use these peer reviews for computing the final grades.