Rubric for Course-Specific Goals in PSYC 200 Report 3
1) Important factors to consider when designing an ethical study in order to test a new drug.
Absent: Did not provide any factors to consider or listed factors but did not explain them
Novice: Don't cause harm/awareness of risks; analyze cost & benefits; consent forms; follow IRB; misunderstanding of intermediate or expert concepts
*mention of level 2 without level 1 = level 1
Intermediate: Don't harm; consent/privacy; plus any of the following: accurate/full reporting of data; no unnecessary/egregious deception; debriefing
*mention of level 3 info without level 2 = level 2
Expert: Intermediate points plus mention of staggered drug treatments (Wait List Control Group); continued monitoring of outcomes; justice in participant selection or use of special populations; reporting funding sources; provide prior empirical evidence of drug effectiveness; data is only used for the purpose of the study, not for additional personal or professional gains; placebo group receives treatment after study if drug proves effective; appropriately trained researchers; data storing issues
2) What have you learned from having to think about ethics in research design?
Absent: Did not discuss a particular time when they had to think about ethics in research
Novice: Unethical to cause harm or disregard ethical concerns; importance of ethics; description of experience (Milgram or other) without ethical lesson learned; minimal mention of ethics or does not mention a particular experience
Intermediate: Ethical concerns/lessons are discussed in the context of the research only: Deception is sometimes justified, importance of debriefing, or another specific ethical issue
Expert: Broader ethical considerations are mentioned: Sometimes particular research questions cannot be answered in an ethical way; redesign of study may constrain the questions that can be asked while improving on the ethical qualities; discussion of how knowledge of ethics expands into other aspects of life (beyond class); the value of past experiments for future designs
3) How to design a phase 3 clinical trial to test effectiveness of new drug, Serenique.
Absent: Does not mention a specific study design; proposed design is unclear or experimentally flawed; rephrases answer to ethical question
Novice: Mentions a control group and experimental group or a single group pre-post test; post-test design
Intermediate: Novice plus discussion of at least one of the following: use of placebos, random assignment, sample size considerations; pre- and post-test; single- or double-blind study
Expert: Intermediate plus a clear discussion of at least one of the following: Test multiple treatment levels, placebo receives treatment after experiment if drug is effective, monitor results and stop study if depression levels become severe; possible confounds; converging measurements of depression symptoms; replication
4) Problems encountered in your own experience with research and what did you learn?
Absent: Did not clearly discuss an encountered problem
Novice: Describes problem (e.g., confounding or third variables) but fails to describes both how they reacted to it AND what lesson was learned
Intermediate: Describes problem (e.g., confounding or third variables) and describes either how they reacted to it OR what lesson was learned
Expert: Clearly explains the problem encountered, how they reacted to the problem, and what was learned about the research enterprise
NOTE: Coded as lesson whether it referred to a lesson about the research process, the study, or the individual's views about research/knowledge construction
Follow-up: tally types of problems identified
5) Ways that science may be limited for studies testing the effectiveness of antidepressant meds
Absent: Does not discuss limitations of science, or talks about study design issues rather than issues with the scientific way of knowing
Novice: There are so many variables, it's hard to measure them all (behavior is complicated); experimenter error; we can't accurately know what's happening in people's heads; self-reports aren't informative or useful
Intermediate: Individual differences exist, although science tests group mean differences; operational definition variability across cultures/groups; science is inductive and derived from individual observations, which can be flawed; clear discussion of the potential presence of confounds (e.g., history, maturation, instrument decay, unmeasured influential variables)
Expert: Does not prove, but contributes to a body of literature that demonstrate trends; Often limited in samples so generalizability is an issue; funding sources may direct study designs and tested hypotheses; ethical constraints limit the way in which questions can be investigated; difference between statistical significance and effect size; null results are not made public
6) Identify claims of fact made by Pristiq and the validity for those claims. Also identify why claims might not be true or why they may be questionable.
Absent: Does not identify a claim of fact
Novice: Identifies a claim of fact but does not discuss the basis (evidence) or the validity of that claim.
Intermediate: Identifies a claim of fact and either provides the evidence OR convincingly discusses the validity of the claim
Expert: Identifies a claim of fact and convincingly describes the evidence associated with the claim and discussed the validity of the claim
NOTE: May students challenge the "Pristiq is thought to work by affecting the two levels of neurotransmittersÉ" because they wonder how they can be wishy-washy in their understanding of their drug. Either it works or it doesn't.
Most are receiving 2s either because they completely missed the report of empirical evidence or because they do not evaluate the claims critically, given the research they just reviewed.