Psychology 757

Advanced Topics in Quantitative Methods: An Introduction to Bayesian Statistics

Spring 2010

Instructor of record Patrick E. McKnight, Ph.D.
co-Instructor Jim Thompson, Ph.D.
Office David King 2064/2065
Office Hours TBA and by appointment
Phone (703) 993–8292
Class Location DK 2072
Class Date/Time
Important Dates Please see GMU academic calendar
Syllabus PDF format

Overview

The Advanced Topics in Quantitative Methods (PSYC 757) covers fundamental probability theory, Bayesian statistics, and Bayesian applications using common social science statistics. Jim Thompson and I co-teach the course and alternate primary responsibilities each year. We review probability theory using the Cartoon Guide to Statistics (don’t laugh - it provides the best introduction and refresher on probability theory). After that introductory material, we intend to cover Bayesian analyses and their applications in social science. The only pre-requisites we expect are 1) successful completion of 611 and 612 and 2) an interest in learning new statistical techniques. We intend to have a very “hands on” approach whereby each student will be running analyses every class period; thus, each student ought to bring a laptop to class. We also plan to distribute supplementary readings that offer both theoretical and practical discussions of Bayesian statistics in social science.

We welcome any and all students interested in expanding their skills beyond contemporary frequentist (i.e., null-hypothesis significance testing) procedures.

Prerequisites

Due to the nature of the material and the relevance to research, I assume all students will have successfully completed the introductory graduate course sequence in statistics (PSYC 611/612 or its equivalent). I do not intend to cover in great detail the statistical models underlying frequentist models so you may want to reread some material on ANOVA and regression if you feel weak in those areas.

Course Requirements and Grading

The course covers two primary topics - fundamental probability theory and Bayesian statistics. These two topics overlap substantially but, more importantly, the latter requires the former. I intend to ensure that everyone understands probability theory before jumping into Bayesian statistics. I expect all students to attend every class, complete all the readings prior to the class meeting, and come prepared to discuss the topic as outlined. In exchange for these requirements, I do not require written papers nor exams. One brief assignment will be discussed the first day of the course and will require some work outside class. Grades, therefore, are determined based upon class discussion and this brief assignment.

Readings and Required Texts

The readings will be made available in electronic format. Each article is scanned into an Adobe Acrobat file (i.e., pdf file). The quality of some readings is not great but all articles are readable in the format they are distributed. Some students may prefer to get the original articles from the source journals but that is left to each student to decide. The electronic versions are distributed at no charge to students enrolled in the course.

The following two books are required for the course. They may be purchased either at the bookstore or online. I would urge you not to rely on the library for your course materials. Other students and faculty members may recall these books at any time and it might interfere with you completing the readings for the course.

Gonick, L. and Smith, W. (1993). The cartoon guide to statistics. New York: Harper Collins.

Albert, J. (2009). Bayesian computation with R (2nd Ed.). New York: Springer.

The following two books are HIGHLY recommended since I intend to shape the course based upon their content and presentation order.

Berry, D.A. (1995). Statistics: A Bayesian Perspective. Belmont, CA: Duxbury Press.

Lynch, S.M. (2007). Introduction to Applied Bayesian Statistics and Estimation for Social Scientists. New York: Springer.

Topic Outline and Readings

Part 1: Introduction to Bayesian statistics

Part 2: Introduction to Probability

  • Readings: Cartoon guide to statistics (probability chapter; optional) and Berry (Chapter 4)
  • Purpose: Introduce students to fundamental probability theory and emphasize its importance in all statistical procedures
  • Goals: Explain each probability principle in detail with examples
  • Single point worth remembering: Probability is based upon logic; use your head and pull apart all probabilistic statements and the language will be much easier to grasp.
  • Points to cover
    • set theory
    • proportions
    • notation

Part 3: Conditional Probability and Bayes Theorem

  • Readings: Berry (Chapter 5) and Albert (Chapter 8; to be discussed later but I want you to read it for this week)
  • Purpose: Extend fundamental probability theory by including conditional statements and moving into Bayes theorem - the basic theorem underlying all Bayesian statistics.
  • Goals: Carefully present all conditional probability principles so that every student understands how Bayes theorem works.
  • Single point worth remembering: Mastering basic conditional probability leads to a mastery of all statistics - frequentist and subjectivist alike.
  • Points to cover
    • Joint probability (intersection and union)
    • Independence of events and experiments
    • Conditional probability
      • Multiplication Rule
      • Law of total probability
      • Bayes’ Rule

Part 4: Bayesian Inference - proportions

  • Readings: Berry (Chapter 6) and Albert (Chapter 3)
  • Purpose: Move directly from theory into application starting with some simple problems with direct applicability to social science.
  • Goals: Provide concrete examples with R code to demonstrate that Bayesian inference follows a similar path as frequentist inferential models.
  • Single point worth remembering: Bayesian statistics can be applied with any statistical software and proportions are the best way to sharpen your skills.
  • Points to cover
    • Models for Proportions
      • Population sampling
      • Priors
      • Likelihood
      • Posterior Probability
      • Prediction
    • Single Sample Studies (Albert Chapter 3)
  • R notes for Part 4

Part 5: Bayesian Inference - density functions

  • Readings: Berry (Chapter 7)
  • Purpose: Expand the material from proportions to probability density functions - an essential element in Bayesian and Frequentist statsistics
  • Goals: Walk through the specifics of observed data translated into distributions that reflect underlying probability density functions.
  • Single point worth remembering: Most of you know the concepts but not the language; don’t let Bayesian terminology confuse you.
  • Points to cover
    • Gradients
    • Beta density
    • Updating rule
    • Consistency checks
    • AUC and intervals
    • Percentiles and probabilities
    • Prediction
  • R notes for Part 5

Part 6: Bayesian Inference - proportion comparisons

  • Readings: Berry (Chapter 8) and Albert (Chapter 4)
  • Purpose: Tie Bayesian reasoning to standard hypothesis testing via the simple proportion comparison models.
  • Goals: Move the student form single sample, single outcome models to traditional research comparison models where null hypotheses dominate.
  • Single point worth remembering: Model testing relies on probabilities regardless of your statistical orientation; Bayesians just make it explicit whereas Frequentists make you work for what you want.
  • Points to cover
    • Two population models
    • Rule of independent models - critically important
    • Probability differences (PdALx) to be covered on 3/23
    • NHST to be covered on 3/23
  • R notes for Part 6 (To be continued…)

Part 7: Bayesian Inference - Two proportion densities material for 3/23

  • Readings: Berry (Chapter 9) and Albert (Chapter 5)
  • Purpose: Move the discussion from portions to density functions so we can appreciate how distributions of differences fit with Bayesian inference.
  • Goals: Discuss the relevance of multiple density functions within the framework of inference and hypothesis testing.
  • Single point worth remembering: Distributions matter; Bayesian inference is very similar to Frequentist inference but where the two sides differ is in their attention paid to the distribution.
  • Points to cover
    • Multiplying independent likelihoods
    • PdALx for densities
    • Probability intervals for d

Part 8: Bayesian Inference - Continuous models material for 3/30

  • Readings: Berry (Chapter 10)
  • Purpose: Introduce Bayesian inference in the standard context of continuous data
  • Goals: Begin with simple examples and carefully bridge the material from proportions to continuous data - note the similarity of the sampling distributions you learned in basic statistics?
  • Single point worth remembering: Proportions di ffer very little from continuous models; the key is the density function

(see next part for more)

  • Points to cover

Part 9: Bayesian Inference - Single mean densities material for 4/6

  • Readings: Berry (Chapter 11)
  • Purpose: Continue our discussion of continuous models but delve deeper into the density functions for these models
  • Goals: Extend what we learned with proportion densities to density functions of continuous data
  • Single point worth remembering: The density function turns out to be the most important element in all statistics; Bayesians rely on it while Frequentists hide it.
  • Points to cover
    • Prior densities
    • Updating rule and Prediction for Normal models
    • Sample size dependency (Law of Large Numbers and the Central Limit Theorem)
    • Rule of means
    • Precision vs. Accuracy
    • Probability intervals

Part 10: Bayesian Inference - Multiple mean comparisons material for 4/13

  • Readings: Berry (Chapter 12)
  • Purpose: Move the discussion to the level of statistical inference most are familiar - means comparison
  • Goals: Begin with simple examples using Frequentist models and apply Bayesian inferential methods
  • Single point worth remembering: Often the results of Bayesian and Frequentist models agree but the methods to arrive at them di er substantially
  • Points to cover
    • Multiple means testing
  • Exercise for today

Part 11: Bayesian Inference - Regression modeling material for 4/20

  • Readings: Berry (Chapter 14); Albert (Chapter 9)
  • Purpose: Extend our discussion from means comparisons to individual predictive models such as regression models.
  • Goals: Move away from means comparison toward parameter estimation
  • Single point worth remembering: Bayesian models tend to produce similar results to Frequentist models but have different implications
  • Points to cover
    • TBA

Part 12: Markov Chain Monte Carlo (MCMC) material for 4/27

  • Readings: Albert (Chapter 6)
  • Purpose: Discuss the details underlying Bayesian estimation algorithms - in particular the fundamental process that falls under the heading Markov Chain Monte Carlo methods
  • Goals: Familiarize the students with the diff erent estimation algorithms underlying MCMC
  • Single point worth remembering: There is nothing magical about these fancy terms; they all pertain to basic assumptions that help us make sense out of great uncertainty
  • Points to cover
    • TBA

LAST DAY OF CLASS May 4th STUDENT PRESENTATIONS