This portfolio focuses on efforts by the Department of Spanish and Portuguese to systematically examine the development of students’ critical thinking and analytical skills across the curriculum. The Department of Spanish and Portuguese serves students who want to learn enough about the Spanish language and cultures to declare the major (we’ve averaged between 200 and 300 majors in the past five years), as well as students who are fulfilling their language requirements and/or pursuing individual interests in Spanish and Portuguese (during the academic year 2010 we had 1,500 students in our second-year Spanish classes alone). The growth of Spanish as the second language of the U.S., as well as the interest in Hispanic cultures and literatures, has increased dramatically in the last ten years, and our department has attempted to address this growing need in a number of ways.

A central goal of the department is the building of relationships that enhance our students’ ability to think critically. Classes in our department put a premium on participation and are usually small. This allows for an intensive classroom experience, although we also strive to promote a culture of learning (and learning of cultures) beyond the walls of the classroom. These practices include distance-learning opportunities and the use of a Blackboard interactive site that includes online discussion boards, video, audio clips, newspaper access, and online tutorials.

We have also been revamping our basic language curriculum to include more conscious engagement with culture, contributing to KU’s stated mission of preparing students for the “challenges educated citizens will encounter in an increasingly complex and diverse global community.” First-year courses introduce activities that encourage learners to reflect on their own cultural practices and become familiar with the cultural practices of Spanish speakers. Second-year courses are now content-based, which means that instead of focusing on the personal, students learn about the histories and civilizations of the 21 Spanish speaking countries of the world as they learn language. They also do extensive journal writing to increase their written communication skills. Analysis of these journal entries indicate that our second year language classes increase students’ language competence, cultural competence, and their awareness of different communities in the world.

After the first two years of study, we have a flexible class structure that includes variable topic courses. This allows students to increase their competency in the language in order to comprehend the perspectives and subjects that comprise the literatures, cultures, and languages of more than 20 countries across different time periods in Spanish and Spanish American cultures and traditions; there is little repetition of exact course material in classes at the 400 level. The major also includes a 500-level capstone course, offering students an intensive research experience and faculty the chance to use creative approaches to their subject areas.

Finally, study abroad opportunities provide a complement to the above-mentioned teaching approaches by focusing on engaging the Spanish and Portuguese-speaking communities, using technology to offer powerful learning tools to students, selecting context-embedded textbooks, and guiding students as they discover the interconnectedness of the world around them. Our students go to several locations in Spain, Argentina, Costa Rica, and Brazil, and these programs involve faculty and GTAs who coordinate, direct, teach, and regularly evaluate and modify programs—reviews that are essential in supporting the department’s high standards in teaching.

As a department, we have been interested in assessing how well these teaching approaches are meeting our goals for student learning. Previously, we garnered feedback about aspects of our undergraduate program and obtained student input about learning through focus groups, by periodically consulting senior survey information, and by distributing our own surveys to our graduates. Since Fall 2005, however, we have engaged in the anonymous, random sampling of student writing at different levels. We designated Spanish 216, 340, and 540 as appropriate classes for this evaluation, and faculty review copies of 10% of those classes’ final papers. Each of the papers is read by two faculty members and assessed using rubrics developed in consultation with all faculty members.

This portfolio focuses on data collected from SPAN 340 (Text Analysis and Critical Reading) and SPAN 540 (a capstone course for majors). Rubrics were developed for each of the writing samples collected from these classes. Seven criteria were identified for the 340 writing samples and five criteria were identified for the 540 writing samples. All criteria are assessed using a 4-point scale (0-3), where a score of 0 indicates an absence of that particular skill and a score of 3 indicates that the particular skill was strongly demonstrated. There was some overlap between the 340 and 540 rubrics.

After our first round of data collection in Fall 2005, we found that the rubrics were easy to implement and worked well. We used this initial data to determine that students would benefit from additional service learning courses, possible additional “bridge” courses that fill in gaps in student learning, and the redoubling of our efforts (supported by workshops at our brownbag lunches) to promote higher-order thinking skills. Since that time we have continued to collect data and have begun to analyze those results.

To examine the overall performance by students in the 340 courses relative to the overall performance of the students in the 540 courses, we first calculated percentages of points earned on the rubric for each course, since the number of points possible is different for each rubric. Looking at the breakdown of these percentages, we see that a larger percentage of students in the 540 course were performing at the highest levels (80–89% and 90–100% of the points earned) compared to the 340 students. It appears that the shift in understanding between the 340 and 540 students is such that the students who were at the 70–79% comprehension level in the 340 course have shifted into the 80–100% comprehension level when they reach the 540 course. However, it should also be noted that the same percentage of students who were “failing” as assessed on the rubric in the 340 courses are still failing at the 540 course level. Thus, students who “don’t get it” at the 300-level still “don’t get it” at the 500-level.

We were then interested in examining student performance on the different rubric components for each of the two course levels. For the 340 course, the majority of students earned a “2” on the rubric for the following skills: the presence of an interesting thesis, the use of solid examples, the close analysis of text, coherent shape and direction to the text, and the evidence of basic analytic vocabulary. Areas that were relatively weaker were the components that examined the variety of constructions, moods, and tenses, and the presence of good technical skills; the majority of students in the 340 courses earned a “1” on the rubric for these skills. There was no component on which the majority of students excelled (i.e., earned a “3”), nor were there particular components of the rubric in which a large number of students demonstrated a complete absence of understanding.

For the 540 course, the majority of students earned a “2” on the rubric for the following skills: formulates a thesis, evidences an ability to read critically, and uses a variety of grammatical constructions. For the other two components of the rubric, the analysis and synthesis of text and the use of good technical skills, a relatively equal number of “2’s” and “3’s” were earned by students in the 540 course. Thus, it appears that students in the capstone course exhibited relatively stronger performances in the areas of analysis/synthesis and technical writing skills than they did in the other areas assessed.

Finally, we were interested in looking at the distribution of scores between the two classes when the skills being assessed on the rubrics were similar. Looking at the distributions of scores for the rubric components that assessed the formulation of a thesis (Question 1 on both rubrics), the distributions appear similar, with most students earning a “2” on this component in both courses. We provide an example of a student paper (pdf) that was given a score of “2” on the formulation of a thesis. An examination of this paper reveals that while the paper does have a thesis, the thesis is simplistic in nature and states the obvious. It does not invite analytical representation.

When examining the rubric components that assess textual analysis (Question 3 on the 340 rubric and Question 2 on the 540 rubric), the distributions also appear similar across the two classes, with the greatest percentage of each class earning a “2”, and somewhat fewer students earning either a “1” or a “3.” Again, we provide an example of a student paper (pdf) on which a “2” was earned in contextual analysis. The analysis is good, but superficial. The student works with the basic ideas and the paper is well organized, but this student does not offer any critical or nuanced understanding of the texts being analyzed.

The most striking changes across similar components, however, are seen in the rubric components that assess the use of a variety of grammatical constructions (Question 6 on the 340 rubric and Question 4 on the 540 rubric) and the use of good technical skills (Question 7 on the 340 rubric and Question 5 on the 540 rubric). In both distributions, the 340 students mainly evidenced low levels of competence in these areas (with a majority earning a “1,” fewer earning a “2”, and the least number of students earning a “3”). This example of student work (pdf) is a paper that earned a “1” in grammatical and technical skills. The paper has basic grammatical errors (spelling, accentuation, moods and tenses, Anglicisms) which interfere with the clarity of the expression of ideas and textual analysis. The sentence construction is also basic and lacks sophistication. In the 540 course, however, we see drastic improvements in these skills, where most students earn a “2” on the variety of grammatical constructions component, and an almost equal number of students earning a “2” or a “3” on the use of good technical skills. Our final example of student work (pdf) is a paper that earned a higher performance in grammatical constructions and good technical skills. The student uses a variety of grammatical constructions in a sophisticated and accurate manner. The paper also evidences improved technical skills (spelling, accentuation, moods and tenses) and uses appropriate bibliographic references.

Overall, it appears that students’ understanding is increasing across the curriculum; in particular, we see improvements in students’ levels of grammatical and technical skills as they move from the 300- to the 500-level. We see less change in students’ ability to formulate a thesis or perform a textual analysis across the two courses; however, it should be noted that these skills were already fairly strong in the students at the 300-level, so perhaps there was less room for improvement.

When examining the endpoint skills that students in our capstone course exhibit, students are relatively weaker in the following areas: formulating a thesis, evidencing an ability to read critically, and using a variety of grammatical constructions. Thus, students may benefit from some additional instruction in these areas across the curriculum, so that these skills are strengthened by the time they complete their final writing projects for the major. Furthermore, the fact that a similar percentage of students in the 340 and 540 courses are demonstrating very low levels of comprehension deserves consideration by the department.

Over the course of the last year (2010–2011), the department has taken definite steps to target the weaker areas in student learning suggested by this study, namely, reading comprehension and critical thinking skills. The department determined that is was necessary to look at prerequisite courses for the major, focusing on Span 216 and Span 324 (the gateway course for the major that directly precedes Span 340). Through a mutual collaboration between Undergraduate Studies and Basic Languages, the department evaluated the transition between these two courses with the goal of better aligning the reading content taught in both Span 324 with Span 340, where students are expected to read longer and richer texts. It was determined that a new textbook and series of secondary readings would be selected for Span 324 and would be implemented in Fall 2011.

In terms of the data collection process, it would be beneficial to create one rubric that could be applied to the writing of students across multiple courses. This would allow more direct assessment of how student learning has changed through their time in the major. It would also be helpful to track additional demographic information, so that we could examine progress in terms of individual differences (e.g., whether students participated in a study-abroad experience or not, whether students were transfer students or if they completed the entire major at KU). This would be useful information in determining which factors of the major are more influential than others on impacting student learning outcomes.