Enhancing Students’ Professional Competencies in Urban Policy and Management—Kelly LeRoux (2007)

Course context

Public Policy and Urban Administration (PUAD 825) provides an introduction to urban public policy and the variety of policy tools used to address municipal, urban, and metropolitan problems. The first part of the course examines U.S. urban policy in the context of intergovernmental relations, highlighting the ways in which local governments are interdependent with one another, their states, and the federal government. The second part of the course looks at the basic differences in forms of local government in America and examines the consequences, or outcomes created by different local government political structures. Students are then exposed to selected topics of importance in urban policy including housing, economic development, and methods for improving the efficiency and effectiveness of local public services. Throughout the course, students are introduced to the logic and principles of microeconomics, and are expected to gain an understanding of how these principles apply to urban and metropolitan policy. Students also learn how these principles can only take public servants so far because they often overlook the complexities of public preferences, as well as values that are important to the public service profession, such as social equity.

This course is designed to enhance professional competencies in the following areas:

• Service management: service standards, service analysis, service delivery
• Administrative policy-making: policy formulation, policy expectations
• External awareness: policy trends, working in a political environment, external policy impact
• Strategic management: long-term outlook, external awareness
• Communication: verbal, written

This course is a required graduate-level course in the curriculum of our full-time "intern-option" Master's of Public Administration (MPA) students and is offered as an elective course for our "career-option" MPA students. "Intern option" students have typically just finished their undergraduate work, are from a variety of academic disciplines, and come to KU from all over the United States. The vast majority of intern option students aspire to have careers in the City Management profession. Mid-career or "career option" students have a minimum of three years full-time professional work experience. Most of them are currently working in local government, although some of them are employed in nonprofit organizations or government related industries. Career option students may be motivated to earn an MPA degree in order to advance within their organization and for professional growth and development. The class meets once a week and is structured as a discussion-based seminar course. Enrollment ranges from 18-22, approximately two thirds of which are intern-option students, and the remaining third are career option students.

My teaching questions related to this course include how well students are processing the information that we discuss in class, and how well this course, and my teaching of it, contributed to students' progress in developing the competencies required for professional public service and local governance. Reflections on my past teaching have led me to ask these questions because students have not been as talkative in class as I would like them to be. For example, students are unresponsive to questions I pose to them at times. Another factor that has led me to focus on this is student performance on exams. The first time I taught the class, it seemed to me that students did not demonstrate the depth or breadth of understanding of exam topics they should have, given the scope of material we covered in class.

There were several components that I changed in my course this semester, in order to be more explicit in my course goals and expectations, as well as to better assess levels of student understanding across the semester. These components are described below.

Topic presentations

Each student was responsible for doing a 20- to 30-minute (including Q&A time) in-class presentation on a topic/policy tool that corresponds to the week's material. I distributed the list of topics the second week of class, and students were allowed to sign up for the topic that they were interested in. It was hoped that allowing students to choose their topics would increase their level of motivation for learning about that particular subject. These presentations were intended to be case-based, examining the topic in broad context (e.g., How effective has research shown the policy, program, or management tool to be?). This assignment was designed to further professional competences in the area of verbal communications. To assess these presentations, I created a rubric to assess student performance using a scale that is based on the Public Administration Department's MPA competencies rubric. This presentation was worth 20% of students’ final grades in the course.

Urban issue research paper

Students were also required to write a 10- to 12-page research paper on an urban issue of their choice that we touched upon in the course. Paper topics could have included one of the presentation topics or some variation on one of the topics. Or, they could have elected another topic. Either way, students were asked to investigate the academic literature on the topic, and to speak to the following types of questions: What is known about the topic/policy tool you are examining? How effective has this policy tool been proven to be? Does it work better for some circumstances more than others (certain places, or for certain services, etc)? Early in the semester, I met with each student individually to review their topic and discuss appropriate sources and issues to focus on. Finally, students were told that papers should read as professional work products, as though they were writing for a government agency or policy think tank. This assignment was also assessed using a rubric that I developed, which included measures of professional competencies.

In this way, both the class presentation and the final paper served as measures of how well the course was contributing to students' development of skills needed in professional public service and local governance (e.g., being familiar with current research and being able to clearly explain public policy issues in a professional setting).

Exams

Finally, students were asked to complete both a midterm and a final exam. These exams were take-home and essay format; they covered the material from the required readings, lectures, and in-class discussions. These two assignments provide additional evidence of how well students are processing, integrating, and extending the course material, as well as assess how well students can discuss these issues in written format. Rubrics were also used to assess these assignments, and students were assessed on both their written/analytic competencies as well as their substantive knowledge of the course content. Together, the midterm and final made up 50% of the final grade.

Furthermore, I was also interested in examining potential changes in understanding across the semester by using a Pre-test and Post-test format. At the beginning of the semester, students were given a Pre-test of their familiarity and understanding of urban policy issues. At the end of the semester, on the final exam, students were again asked to answer a sub-set of the questions from the pre-test, drawing from their readings and the class discussion throughout the semester. The repeated questions were:

1) On the first day of class you were asked about your opinion on metropolitan fragmentation in this question: “Is having 116 local governments in the KC metro area mostly a good thing, or mostly a bad thing? Read your response to this question and reflect on what you have learned in this course.” Now, formulate a response to this question: Can local governments be cooperative and competitive at the same time? How do the self-interests of local governments shape their patterns of cooperation and competition?

2) You were also asked to respond to this question (pg. 3): “Given that (economic development) is the main way to increase local revenue, should cities try to expand their tax base as much as possible through development?” Throughout the class, we have basically established that cities cannot avoid economic development. Given this, what sort of things should city officials pay attention to when negotiating ED deals and public-private partnerships to ensure they create public value?

The changes in students’ answers were assessed using a modified competencies rubric, to determine whether there was significant advancement in student understanding of policy issues across the semester.

Samples of student presentations

In terms of the student presentations, overall performance on this assignment was excellent. Based on what we see from the grade distribution, student scores ranged from 88% to 100%. The grade for this assignment were as follows: A (12), A- (4), B’s (2). High-quality work exhibited analysis of both the benefits and shortcomings of the urban policy tools, effectively reviewed the literature related to the particular policy tool, and drew connections between the students’ own assessments and current theories on the value of a particular tool's value in public service. Presentations that were less successful failed to adequately address the implications of the student's topic for the broader public policy context, or they did not provide additional evidence of research outside of what was provided in the course textbook.

Presentations 1 and 2 are clear examples of “A” work. Both provide a case study to illustrate the subject matter in the context of a specific city or county. Presentations 1 and 2 also provide a slide with references, documenting the sources of their research and links to sites for additional information. In Presentation 1 (pdf), the student did an excellent job of presenting his information with engaging graphics and used several interaction slides to elicit class participation. This student provides a timeline illustrating the use of this innovation in local government, presents objective information on both the benefits and shortcomings of this tool, and provides his own assessment of how the tool rates in terms of the four core values of public service. In Presentation 2 (pdf), the student created an attractive presentation that shows evidence of thorough research, including information collected through personal interviews. In the slide discussing "Why Consolidation Worked" the student draws linkages between her own findings and current theories that explain city-county consolidation.

Presentations 3 and 4 are examples of “A-/B+” work. Presentation 3 (pdf) contains all the basic information I would expect in a student presentation, but it could have been improved in a number of ways. I would have liked to see the student address the issue of political feasibility of urban growth boundaries (UGBs), especially because it relates to a key competency in public management (external awareness/working in a political environment). One or more of the UGB examples provided by the student might have made a better illustration through the display of spatial maps or graphics to illustrate how UGBs contain urban development in more or more of these cities. On the slide highlighting “advantages of UGB’s” I would have like to see the student describe the implications of UGB’s for containing sprawl in his or her own words rather than copy a list of items from a source clearly designed to promote UGB’s.

Presentation 4 (pdf) provides a great deal of technical information, but it is disembodied from any context. A case study from a local city or county illustrating a special district would have helped to illustrate the subject matter more clearly. This presentation also contains a few too many slides and does not demonstrate evidence of research on the topic beyond that provided in the course textbook. Presentations 3 and 4 also could have been improved by adding a slide highlighting sources of their research and link to additional information on the topic.

Samples of student papers

Successful research papers were well-written, provided a thorough presentation of the relevant research, and evidenced connections to the course concepts. Less successful papers did not show a deep understanding of the topic, by either providing an inadequate review of the relevant research or failing to discuss the practical implications of what they discovered in the process of researching their topic. The class performance on this assignment, while still very good was less impressive than the effort students put forth on the presentations of these same topics. The grade distribution was as follows: A (9), A- (4), B+ (3), C (2). Overall, students did achieve the goals of the assignment, but a some students submitted papers of lesser quality than they were capable of, based on my observation of their performance on past assignments.

Research papers 1 and 2 are examples of paper that earned an “A”. Both of these papers are well-written, demonstrate thorough, relevant research, and draw upon theoretical concepts discussed throughout the course. In addition, these papers provide supplemental tables, figures, and illustrations to support arguments made in the paper (Example 1 (pdf), Example 2 (pdf)). Research paper 3 is an example of B paper. It is reasonably well-written, but the paper shows rushed thinking by the student. It misses many important sources of literature on the subject, and fails to link the discussion to recent federal legislation that has brought increased public attention and political salience to the topic (Example 3 (pdf)). Research paper 4 is an example of a “C” paper. It is written in simple, declarative statements and does not reflect the expected level of student understanding of the topic and the issues and debates it engenders. The material is too heavily reliant on the course textbook and largely fails to draw on the most relevant research and academic literature on the topic. Finally, the works cited are not properly integrated into the paper (Example 4 (pdf)).

Pre- and post-test analysis

The pre-test was an un-graded, short answer, in-class test given to students in the first week to establish a baseline measure of students’ understanding of the issues that would be covered in the course. In comparing the pre- and post-test responses, it is evident the majority of students have developed an appreciation for the complexity of major issues we covered in the class such as metropolitan fragmentation, and the costs and benefits of urban economic development (Pre- and Post-test Scores (pdf)).

The grade distribution for the post-test was as follows: A (13), B (3), C (2). The pre-test responses were very simplistic, often reflecting concrete thinking about these issues. For example, a common response to the question of whether having so many local governments in a metro area is mostly good or bad, was that it was bad because “local governments accomplish more by working together, because it just makes sense not to duplicate services in cities so close together.” The post-test responses tell me that students have acquired a much better understanding of the nuances involved in these issues. For example, post-tests offered evidence of student learning about the ways competing political values and conflicting citizen preferences shape the outcome of decisions about public service delivery, why cities are likely to cooperate for some functions, and destined to compete for others. For example, this quote from one student's post-test is typical of the responses provided throughout the class: “While regional cooperation promotes innovation, democracy, and community pride, as well or perhaps better than fragmentation and competition, I must conclude that there is a place in urban America for both. Rather than being mired in an ideological battle between regional versus local, the skilled public manager could be better served by analyzing the context of any particular situation and determining how either or both present value to the community.”

Finally, the class performance overall, as evidenced by the final grades, was excellent. The grade distribution for final grades was as follows: A (16), C (2). The class was comprised of really outstanding students to begin with, but I also think at least for the individual presentation assignment, students felt some pressure to perform well based on observing their peers. As the semester went on, the quality of the presentations in particular seemed to get increasingly better.

This analysis has helped me to discover that conducting a pre-test and post-test is a highly effective means of assessing student learning. This will become a permanent feature of the course, and I will continue to integrate the post-test questions into the final exam. This analysis also helped me realize that student learning is maximized through specific examples and case illustrations. Students seemed to learn the most by deeply engaging a topic and presenting their research to their classmates. Almost all of the students were able to build on the questions and discussions generated by their in-class presentation to further their thinking on the topic for the research paper. The cases presented by students in their presentations to illustrate their topics were so helpful in reinforcing theoretical arguments made in the readings and lectures, that a number of students cited other students’ cases as examples in responding to exam questions. Seeing how students grasped the material through cases, their own in-depth research and that of their peers, has made me re-think my approach which has tended to focus on breadth versus depth.

A next step for teaching this course is to re-asses the weekly readings. Knowing now that students learn better through more in-depth exposure to a subject suggests I should avoid readings that present many different ideas on the subject, but focusing more on readings that will reinforce the most important points. I can also use more in-class discussion to ensure students fully process these key points. I will also work on incorporating more case examples and illustrations into my lectures. I will also re-evaluate the goals of the urban research paper assignment, and would consider eliminating it given that the goals of this assignment do overlap somewhat with that of the presentation. Finally, I would also like to strengthen the link between the MPA competencies created by my Department and some of my assignments, such as the research paper.