Using Learning Goals to Guide Course Redesign—Michael Moody (2007)

Public Finance (PUAD 835) (pdf) is a graduate-level course that focuses on the role of the government in the economy. More specifically, we address the demand for government service, determinants of the level of provision of these services, and how governments fund these services. This course is required for students seeking a Master’s of Public Administration; enrollment has ranged from 8-20 students who have diverse backgrounds and varying career goals. The majority of the students enrolled in this course are enrolled in the MPA program, although there have been some students from the School of Education. There is much variation in the academic backgrounds of the students in this course. Some students are recent graduates and others have been out of school for over 25 years.  Each student brings unique perspectives from his or her working experience also.

The goals of the course are as follows:

• Appreciate the trade-offs that exist between equity, efficiency and feasibility in public finance.
• Understand how government policies and actions impact the behavior of individuals and firms.

In my first year at KU, I taught this public finance course three times. My experiences teaching it left me unsatisfied, because I was unsure what knowledge or understanding, if any, students were taking away from this course about core ideas related to efficiency and equity in public policy. The evidence of this problem came from the final papers. One of the criteria mentioned on the assignment was evidencing the ability to analyze a current issue/topic in public finance. A key component of this analysis was framing the issue in terms of the broader questions of efficient allocation of resources and equity issues. I looked at the final papers and was unsatisfied with the level of attention given to these core areas.

I believe that in the past, I started off on the wrong track when designing the course. In my initial approach, I began by thinking about which textbook to use and not about the end results of the course or what the student should take away after a semester together. The result was that the students were exposed to the concepts and terms, but they did not seem to move toward successful synthesis and application of these concepts to other new areas.

Therefore, my teaching question is: How can I assess whether students understand and can apply the core ideas from public finance?

The course consisted of several types of assignments—weekly journals, short assignments, and policy briefs—which were designed to scaffold students’ understandings of the core issues in public finance. Beginning with this end result in mind clarified the way to proceed.

Because I wanted students to be able to see how various topics fit into the big picture of public finance, I also presented a framework to the class in the beginning of the semester. I then returned back to the framework during each class, to help students place the current topic in relation to the whole.

Weekly journals

For this assignment students were asked to:

1. Record any questions about material presented in class or encountered in the readings,
2. Note what they felt was the most important issue/topic/point discussed,
3. Identify issues that they would like to discuss further, and
4. Make connections between different aspects of course material covered.

My hope was that this assignment would spur students to think more deeply about the topics discussed in class, and at the same time provide me feedback regarding how well they were understanding key points and how well I was introducing and teaching about these topics. The weekly journal was graded as either complete or incomplete.

Assignments

These short assignments consisted of either a problem set or a written memo and accompanying data sheet. In both cases these assignments revolved around specific topics such as property taxation, revenue forecasting, and debt management. The learning goals for these assignments were to help students write clear, concise memos regarding technical topics. These assignments were graded based on the quality of the writing and the coverage of the material.

Policy brief: written report

For the policy brief assignment (pdf), students were asked to do the following: describe a policy issue, explain why it was important, how they have analyzed it, what they have learned from the analysis, and why what they have learned is relevant for policymaking. The goal of this assignment was to get students to apply concepts from public finance to a policy issue of their choice. In particular, the concepts of equity and efficiency covered in class were to be the central focus of this project.

Policy brief: oral presentation

The oral presentation gave the students an opportunity to lay out the essential elements of the analysis and the most important insights. For the oral report, the goal was to require students to make coherent presentations in front of an audience.

Finally, I also created specific grading rubrics, which I distributed to students with the assignment descriptions. In this way, students were able to see what core concepts were being graded, which should have helped them focus their thinking and efforts toward the key ideas. It was also helpful for me as it made the grading process more transparent and consistent.

Student 1 (pdf) attempted to provide a summary of what was covered in class. This approach allowed me to see where this student missed key points and made major misstatements. This student failed to discuss why these questions were important or which was most important in his view. And most seriously, there appears to be little reflection on the topics covered in class. Student 2 (pdf) provided feedback on various aspects of the evening’s class and gave an “axiom” from the concept of taxation. Student 2 also made linkages to other courses that he was taking. Overall, this student signaled to me more reflection on the material in and out of class. Student 3 (pdf) did a good job of addressing key issues (e.g. equity) and recognized the complexity of these issues. He also thought about how the topic applied to him personally. This is another good example of a journal entry; it is more concise than Student 2’s work, but also illustrates his thoughtfulness.

Assignments

I have provided a couple examples of student work on the assignments. In particular, both of these examples address the Revenue Portfolio Assignment (pdf), which asks students to write a brief memo assessing a jurisdiction’s revenue structure, as well as provide data on the jurisdiction’s multiple revenue sources. The grade distribution of this assignment indicated that students did quite well, with 23 A’s and 5 B’s overall.

The assignment done by Student 4 (pdf) does a nice job covering the key points in a concise and coherent manner. There are good links between the diversity of revenue and the impacts on citizens.

My personal insight from this assignment is that I need to add questions to get students to focus beyond the numbers on the potential impacts on citizens and their personal tax burdens. A question such as: “Given the tax structure, which groups of people are most likely to have the largest tax burden?” would allow students to grapple with the equity aspects of revenue diversity.

Policy brief: written report

There was some variability in the overall performance on the written aspect of the policy brief. While 10 students exhibited A-level work, there were 15 B’s and 3 C’s on this assignment.

The policy brief examples show the difference in thinking about core concepts of public finance. Some students touched on some of the efficiency issues but glided over the important equity issues related to Tax Increment Financing. Student 5’s (pdf) written report, on the other hand, touches on both the equity and efficiency issues related to his topic. These discussions flow nicely in the context of the paper.

As I continue to modify this assignment, I am considering ways to get students to think about their topics earlier to give them more time to think about the equity, efficiency, and administrative and/or legal aspects of their issues.

Policy brief: oral presentation

The grades for the oral presentations were less variable than the written component of the policy brief assignment: 20 A’s and 9 B’s. I have provided two examples of these presentations, which were different in a number of ways. The accompanying slides and comments fail to fully reflect the differences in these presentations. Student 6 (pdf) was quite uncomfortable and rushed through the presentation as quickly as possible. There was almost no discussion of the core concepts such as equity and efficiency. It was difficult to see how this topic related to our course. My written comments to this student were: “The slides were very well done. Even though the slides looked good it is up to you as the presenter to make the points. You were so focused on the slides/notes that you were unable to really engage the class.”

Student 7 (pdf) did an admirable job making relevant linkages to the course material and covered core ideas. After seeing the presentations, I feel the need to emphasize the brief nature of the presentation and the importance of focusing on a few key insights. My written comments to this student were: “The presentation was very well done. You seemed very comfortable responding to questions and gave good answers. Certain slides were too full which creates small fonts that are difficult to read. The table was also quite difficult to read. I would have liked you to engage your classmates more in the question period.”

Making a few changes to my course had a significant impact on student understanding. By revising my course, with more thought given to my end objectives (preparing students to think deeply about public finance issues with respect to equity and efficiency), the thread between topics became much clearer. Thinking more deeply about how I was organizing and presenting material in my public finance course changed both what I did in the classroom and what I asked students to do outside the classroom

In the classroom I tried to remind students frequently about the fundamental questions that public finance addresses and the importance of approaching these questions with efficiency and equity in mind. This created an atmosphere where students could see clearer links between topics and how seemingly disparate topics fit together in a broader framework. I was able to see the links they were making through their journals.

Outside the classroom, I was able to tailor assignments to focus more on implications for equity and efficiency and get students started thinking about the trade-offs that can exist between the objectives from the beginning. Using grading rubrics improved my ability to focus students’ attention on the key areas. Compared to prior experiences teaching this course, there were clear improvements in the students’ abilities to understand the complexities of the public finance issues, as well as their ability to discuss these issues in their papers.

This first iteration in adapting my course was eye-opening for me. By focusing initially on the framework and the core questions of public finance, the students got a better sense of the unifying stream that runs through these issues and a more complete sense of why these issues matter. In the next iteration of this course, I plan to focus on the expansion of my assignment rubrics. In terms of the final papers, I didn’t have an explicit grading rubric for the final policy brief, although I had ideas about what should be emphasized (questions of equity and efficiency) in mind when I was grading them and made notes about some general trends in student performance. The components that should be included in a future rubric would include the organization and writing style of the paper, the legal and administrative aspects of the discussed policy, an analysis of the equity and efficiency issues, and a discussion of the real-world implications of this analysis. In this rubric, I will also be able to convey the relative weights that I place on these components, such that a clear investigation of the equity and efficiency issues are given more weight than the legal framework or organizational components of the paper. Furthermore, I will design the levels of the rubric to elaborate on what beginner, intermediate, and advanced levels of understanding would look like for each of these components.

While I had a rubric for the Memo assignment, I plan to elaborate upon its components in future iterations of the class as well. This rubric should describe the different levels of student performance, such as what constitutes excellent work in a skill compared to work that is not as advanced or complete. For instance, one component of the memo assignment rubric addresses the provision of supplementary materials. While the levels of the rubric currently are: Excellent, Very Good, Good, Fair, Poor, and Expand, it would be more useful to create fewer but more explanatory categories, such as: All supplementary materials are provided and are appropriate to the assignment, Some materials are missing or inappropriate to the assignment, or No supplementary materials are provided.

Through the revision of my rubrics, I will be able to not only provide increased guidance to students in terms of what each assignment requires, but I will also be able to assess the strengths and weaknesses of student performance within the class, to guide my future scaffolding of particular skills.