Skip Navigation
UT wordmark
College of Liberal Arts wordmark
llilas masthead
Charles R. Hale, Director SRH 1.310, 2300 Red River Street D0800, Austin, TX 78712 • 512.471.5551

Joseph E. Potter

Professor Ph.D., Princeton University

Professor, Department of Sociology and Population Research Center
Joseph E. Potter

Contact

Interests

Reproductive health; population; development

LAS 381 • Human Fertility

40345 • Fall 2011
Meets T 300pm-600pm MAI 1704
(also listed as SOC 389K )
show description

This course is intended to provide a broad and in-depth exposure to substantive, theoretical and methodological issues in the study of human fertility.  The emphasis will be on attempts to explain the dramatic declines in fertility that have taken place in the last three decades in countries such as Brazil, Mexico, Thailand, Spain, Italy, and the countries of the former Soviet Union, as well as the relative stability of fertility in the countries of Northern Europe and the United States.  It will also address the policy issues that are now at the forefront of national and international debates regarding reproductive health and population, as well as the likely course of fertility in various settings.   Participants interested in an empirical exercise will have the opportunity to analyze a fertility and contraceptive prevalence survey recently conducted in a country of their choosing, or in the analysis of a prospective survey of oral contraceptive use along the US-Mexico border in El Paso, Texas.

 There is no prerequisite for this seminar.  Participants without previous training in, or exposure to demographic measurement or techniques will, however, need to spend some time on their own gaining familiarity with basic measures such as the Total Fertility Rate.  Evaluation will be based on several empirical exercises (10 percent), a preliminary bibliographic paper due about half way through the semester (30 percent), a presentation/discussion leadership related to one of the course topics (20 percent), and a final “paper” that builds on the preliminary paper and could add to it either an empirical application or a proposal for future research (40 percent).   More details on these assignments will be forthcoming as the semester proceeds. 

 Everyone will be expected to have read the assigned papers before class, and to participate in the discussion.  Often, but not always, some questions related to the readings will be distributed in advance.  

LAS 381 • Evaluation Of Social Policy

40726 • Spring 2011
Meets F 900am-1200pm MAI 1704
(also listed as SOC 384M )
show description

coming soon

LAS 381 • Eval Of Social Pol In Lat Amer

40732 • Spring 2010
Meets F 900-1200 MAI 1704
(also listed as SOC 384M )
show description

SOCIAL POLICY EVALUATION

SOC 384M (46520)

Spring 2010

 

Prof. Joseph E. Potter                                                      Population Research Center

   (joe@prc.utexas.edu)                                                  BUR 520

_________________________________________________________________

 

            This PhD-level methods course offers an introduction to the practical application of microeconomic principles and cutting-edge statistical techniques to evaluating social policy, program, and treatment effects.  The substantive focus will be on programs in health, education, welfare, and workforce training.  Students will be invited to work through a series of concrete program evaluations conducted for a number of international and national organizations ranging from the World Bank and the Inter-American Development Bank, to the governments of Brazil, Mexico, Kenya, Indonesia, and more.  While the course does touch on cost-benefit analysis (prospective evaluation before a program is in place), the primary focus is on the design and execution of program evaluations (the assessment of on-going programs or of programs after the fact).  Key features of the course include the following:

 

  • Treatment of the formal logic of experimental and quasi-experimental research design
  • Multivariate statistical tools (distance measures, cluster analysis, factor analysis)
  • Managing large databases
  • Acquiring some mastery of the Stata statistical analysis package
  • Statistical techniques for dealing with selection bias and other commonly encountered threats to internal validity
  • And much more…

 

            Readings draw from the literature on program evaluation, quasi-experimental design, and the emerging field of microeconometrics.  While oftentimes challenging, these developments in statistical methodology have important implications on the way we conduct public policy analysis.  Students will write occasional commentaries on real-world studies and analyze large real-world datasets.

 

            The core textbook for this course is the newly published:

 

S. Khandker, G. Koolwal, & H. Samad (2010), Handbook on Impact Evaluation: Quantitative Methods and Practices, The World Bank

 

            Students willl be evaluated in terms of the quality of class participation, in-class presentations, fortnightly take-home exercises, and performance on two take-home examinations.

 

 

Topical Overview

 

  • Central theme of the course
    • The microeconometrics of evaluating policy, program, and treatment effects

 

  • Substantive foci of the course
    • The formal principles of social science research design
    • Statistical analysis of non-random samples
      • Selection biases and other endogeneities
      • Problems of missing data
      • Truncated or censored data
  • Advanced tools for exploiting the power of panel datasets
  • Working with large, complex, and messy datasets
  • Appreciating the power and sophistication of the Stata statistical software package
  • Conditional cash transfer (CCT) programs in Latin America

 

  • Methods and tools
    • Computer-intensive statistical techniques
    • Robust statistical estimators
    • Difference-in-differences estimators
    • Matching estimators
    • Non-parametric and semi-parametric regression
    • Instrumental variable methods
    • Maximum likelihood estimation
      • Logit and probit models (binary, multinomial, and ordinal)
      • Incidental truncation estimators
      • Switching regressions
      • Fixed and random effects models
      • Hierarchical linear models

 

  • Real-world case studies and datasets
    • National Supported Work Demonstration project (USA)
    • Balsakhi school tutoring program (India)
    • PROGRESA education, nutrition, and health program (Mexico)
    • Red de Protección Social program (Nicaragua)
    • Bolsa Familia program (Brazil)
    • The Job Training Partnership Act (USA)
    • Indonesian Family Life Survey on midwifery (Indonesia)
    • Seguro Popular de Salud program (Mexico)

 

COURSE OUTLINE AND READINGS

 

Week 1 (Jan 22):  Introduction and Overview

 

  • Random sampling, random assignment, and random variables
  • Experimental vs quasi-experimental research
  • The critical question of ex ante versus ex post control over the selection of observations and assignment to treatment
  • The "Fundamental Evaluation Problem" and the challenges it poses to statisical analysis
  • Testing the difference between two means: Review
  • Seven ways to measure the standard error of the difference in sample means
  • Introduction to the Stata statistical  package

 

  • KEY READINGS:
    • W. Shadish, T. Cook, D. Campbell (2002), "Experiments and Generalized Causal Inference", Ch 1 in Experimental and Quasi-Experimental Designs for Generalized Causal Inference
    • H. Bloom (2005).  Learning More from Social Experiments: Evolving Analytical Approaches, Ch. 1
    • Handbook on Impact Evaluation, “Introduction to Stata”, skim Ch 11

 

Week 2 (Jan 29):  Research Design in Randomized Experiments

 

  • Internal and external validity: Hallmark issues in social policy research
  • Standard notation for depicting research designs
  • Bias due to selection, missing data, and failing to properly control for unobservables
  • Translating the difference in means into the language of linear regression
  • The "Difference in Differences" (DDIF) method for evaluating pretest/posttest control group designs
  • Introduction to the National Supported Work (NSW) case

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, Chs 2-3, Ch 5, and Ch 12
    • B. Meyer (1995), "Natural and Quasi-Experiments in Economics", Journal of Business and Economic Statistics, 13(2):151-161
    • D. Campbell and J. Stanley (1963), Experimental and Quasi-Experimental Designs for Research (skim for highlights)
    • Stata Lecture Packets 1-4 (browse as needed or desired)

 


Week 3 (Feb 5):  The Econometrics of Program Evaluation

 

  • Discussion: The NSW case
  • Heteroskedasticity-robust and bootstrapped standard errors
  • The regression discontinuity design
  • Introduction to the Balsakhi case and dataset
  • "Long" vs "Wide" dataset formats in Stata

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, “Regression Discontinuity and Pipeline Methods”, Ch 7
    • Stata User's Guide (Release 9), "Obtaining Robust Variance Estimates", pp.275-280
    • W. Trochim (2002), "The Regression Discontinuity Design" and related material available at: www.socialresearchmethods.net/research/RD/RD%20Intro.pdf
    • The MIT Poverty Action Lab Policy Briefcase (2005), "From Schools to Learning: Meeting the Needs of Marginalized Children"
    • A. Bannerjee, S. Cole, E. Duflo, & L. Linden (2004), "Remedying Education: Evidence from Two Randomized Experiments in India" (the "Balsakhi Case")

 

Week 4 (Feb 12):  Randomized Experiments in the Developing World

 

  • Discussion: The Balsakhi case
  • Working with difference-in-difference models
  • Randomizing by groups rather than by individuals
  • Adjusting standard errors for clustered observations
  • Special practical challenges posed by social policy evaluation in the developing world
  • More on the regression discontinuity design

 

  • KEY READINGS:
    • H. Bloom (2005), "Randomizing Groups to Evaluate Place-Based Programs", Ch 4 in Learning More from Social Experiments
    • E. Duflo & M. Kremer (2003), "Use of Randomization in the Evaluation of Development Effectiveness", MIT Department of Economics, prepared for the World Bank Operations Evaluation Department
    • R. Wooldridge (2002), "Single-Equation Models under Other Sampling Schemes", pp. 128-132 in Econometric Analysis of Cross Section and Panel Data
    • G. Burtless (1995), "The Case for Randomized Field Trials in Economic and Policy Research", Journal of Economic Perspectives, 9(2):63-84  [Supplemental Reading Only]
    • M. Hudgens & M.E. Halloran (2008), "Toward Causal Inference With Interference", Journal of the American Statistical Association, 103(482):832-842   [Supplemental Reading Only]

 

Week 5 (Feb 19):  The PROGRESA Case, Part 1

 

  • Wrap-Up: The Balsakhi case
  • Introduction to the Progresa case
  • Conditional cash transfer (CCT) programs
  • The pragmatics of designing and implementing a large-scale randomized social experiment in a development setting
  • Politics and bureaucracy versus good social science

 

  • KEY READINGS:
    • E. Rios-Neto (2008), "Pocket Book Poverty Alleviation", Americas Quarterly, pp. 68-75
    • S. Levy (2006), Progress Against Poverty: Sustaining Mexico's Progresa-Oportunidades Program, Chs 1-2
    • P. Bate (2008), "The Story Behind Oportunidades", IDB America at www.iadb.org/idbamerica/index.cfm?thisid=3049
    • E. Skoufias (2001), " PROGRESA and its Impacts on the Human Capital and Welfare of Households in Rural Mexico: A Synthesis of the Results of an Evaluation by IFPRI", International Food Policy Research Institute (IFPRI)
    • T.P. Schultz (2001), "School Subsidies for the Poor: Evaluating the Mexican PROGRESA Poverty Program", Yale University Economic Growth Center Discussion Paper No. 834

 

Week 6 (Feb 26):  The PROGRESA Case, Part 2

 

  • Discussion: The PROGRESA case
  • The selection of treatments and controls
  • The econometrics of evaluating the PROGRESA program
  • Multinomial logit and probit modeling of PROGRESA outcomes

 

  • KEY READINGS:
    • E. Skoufias, B. Davis, & S. de la Vega (2001), "Targeting the Poor in Mexico: An Evaluation of the Selection of Households into PROGRESA", World Development, 29(10):1769-1784
    • J. Hoddinott & E. Skoufias (2004), "The Impact of Progresa on Food Consumption", Economic Development and Cultural Change, 53(1):37-60
    • T. Liao (1994), “Multinomial Logit Models”, pp.48-59 in Interpreting Probability Models

 


Week 7 (Mar 5):  The Red de Protección Social (RPS) Case

 

  • Wrap-Up: The Progresa case
  • Did the greatest gains go to the poorest of the poor?
  • The transferability of the PROGRESA program to other settings
  • Introduction to the Red de Protección Social case
  • Heterogeneity of program impacts: Introduction to quantile regression

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, “Measuring Distributional Program Effects”, Ch 8
    • J. Maluccio, et al (2005), “Red de Protección Social: Breaking the Cycle of Poverty”, International Food Policy Research Institute
    • A. Dammert (2009), “Heterogeneous Impacts of Conditional Cast Transfers: Evidence from Nicaragua”, Economic Development and Cultural Change, 58(1):53-83

 

Week 8:  (Mar 12):  Day before Spring Break 

 

  • This is a “slack time-slot” for our yet-to-be scheduled discussion of Brazil’s Bolsa Familia program.  All other dates in the syllabus are likely to shift depending on the timing of our guest lecturer’s visit to UT.
  • RPS assignment due

 

Week 9 (Mar 26):  The Challenges of Quasi-Experimental Research

                                 and Introduction to Non-Parametric Matching Techniques

 

  • Wrap-up: Red de Protección Social case
  • Assessing causality in quasi-experimental research: Creating counterfactuals to impute the missing potential outcomes for treatment and control
  • Sidebar on cluster analysis and factor analysis
  • Matching treatment observations to controls with multivariate distance measures: One-to-one matching, One-to-many matching
  • Introduction to propensity scores and propensity score matching
  • The contribution of Rosenbaum & Rubin (1983) to the “curse of dimensionality”
  • The question of "common support" of the covariates associated with the treatment and control groups
  • The average treatment effect on the population (ATE) versus the average treatment effect on the treated (ATT)
  • Introduction to Stata's NNMATCH and PSMATCH2 routines
  • Revisiting the National Supported Work (NSW) case

 

KEY READINGS:

  • J. Currie (2003), "When Do We Really Know What We Think We Know?:  Determining Causality", UCLA Department of Economics Working Paper
  • R. Moffitt (2005), "Remarks on Causal Relationships in Population Research", Demography, 42(1):91-108
  • M. Ravallion (2001), "The Mystery of the Vanashing Benefits: Mr Speedy Analyst's Introduction to Evaluation", World Bank Economic Review, 15(1):115-140
  • Human Resource Development Canada [HRDC] (1998), "Quasi-Experimental Evaluation"
  • Handbook on Impact Evaluation, “Propensity Score Matching”, Ch 4
  • A. Abadie, D. Drukker, J. Herr, &  G. Imbens (2001), "Implementing Matching Estimators for Average Treatment Effects in Stata", The Stata Journal, 1(1):1-18

 

Week 10 (Apr 2):  Virtues and Limitations of Matching Methods

 

  • Discussion: The National Supported Work (NSW) case
  • Risks in using propensity score matching
  • Blocking on the propensity score
  • Sensitivity analysis for matching estimators
  • Introduction to the Bangladesh Microcredit case

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, “Propensity Score Matching Technique”, Ch 13
    • D. Peikes, L. Moreno, & S. Orzol (2008), "Propensity Score Matching: A Note of Caution for Evaluators of Social Programs", The American Statistician, 62(3):222-232
    • T. Nannicini (2007), "Simulation-Based Sensitivity Analysis for Matching Estimators", The Stata Journal, 7(3):334-348
    • T. Cook, W. Shadish, & V. Wong (2008), “Three Conditions under Which Experiments and Observational Studies Produce Comparable Causal Estimates”, Journal of Policy Analysis and Management, 27(4):724-750
    • M. Caliendo and S. Kopeinig (2008), "Some Practical Guidance for the Implementation of Propensity Score Matching", Journal of Economic Surveys, 22(1):1:31-72  (good reference; skim for highlights)

 

Week 11 (Apr 9):  Further Ways to Exploit Propensity Scores

 

  • Discussion: The Bangladesh Microcredit case
  • Using the propensity score in place of the covariates in regression analysis
  • Weighting outcomes by the propensity score
  • Combining regression and matching methods
  • Non-parametric and semi-parametric regression estimators

 

  • KEY READINGS:
    • G. Imbens and J. Wooldridge (2009), “Recent Developments in the Econometrics of Program Evaluation”, Journal of Economic Literature, 47(1):5-86
      • A comprehensive and authoritative (but fairly dense) review of the state of the art in the microeconometrics of program evaluation.  Read pp. 32-42 and skim the rest as desired.

 

Week 12 (Apr 16):  Fixed/Random Effects Models in Program Evaluation

 

  • Discussion:  The NSW case
  • One-way and two-way fixed effects models: Controlling for the level effects of unobservables
  • The relationship betweeen fixed effects and difference in differences
  • Random effects models and empirical Bayes estimators
  • Relationship between random effects models and robust/cluster adjustments for variance estimates
  • Introduction to the Job Training Partnership Act (JTPA) case

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, “Double-Difference Method”, Ch 14
    • J. Brüderl (2005), "Panel Data Analysis", Universität Mannheim Working Paper
    • J. Frain (2008), “Stata Commands for Unobserved Effects Panel Data”, unpublished manuscript, Trinity College, Dublin
    • [Review the logic of fixed-effects and random-effects models in your favorite standard econometrics textbook]

 

Week 13 (Apr 23):  More on Selection on Unobservables

 

  • Discussion:  The JTPA case
  • Review of truncated and censored regression
  • The Heckman procedure
  • Switching regressions to control for selection bias in quasi-experiments: An extension of the Heckman procedure
  • Application of the Heckman procedure to the NSW dataset
  • A sidelight on dealing with missing data in evaluation studies
  • Introduction to the Indonesian Family Live Survey case

 

  • KEY READINGS:
    • R. Berk (1983), "An Introduction to Sample Selection Bias in Sociological Data, American Sociological Review, 48(3):386-398
    • D. Powers (2005), "Censored Regression, Sample Selection, Endogenous Switching, and Treatment-Effect Regression", UT Dept of Sociology Working Paper
    • J. Angrist & G. Imbens (1995), "Two-Stage Least Squares Estimation of Average Causal Effects in Models with Variable Treatment Intensity", Journal of the American Statistical Association, 90(430):431-442 [Supplementary reading only]

 

Week 14 (Apr 30):  Instrumental Variable Solutions to Problems of Bias

 

  • Discussion: The Indonesian Midwives case
  • Review of instrumental variables and two-stage least squares regression in econometrics

 

  • KEY READINGS:
    • Handbook on Impact Evaluation, “Instrumental Variable Estimation”, Ch 6 and Ch 15
    • E. Frankenberg and D. Thomas (2001), "Women's Health and Pregnancy Outcomes: Do Services Make a Difference?", Demography, 38(2):253-265
    • P. Ender (2004), "Instrumental Variables Regression", a tutorial using Stata, UCLA School of Education at www.gseis.ucla.edu/courses/ed231c/notes3/instrumental.html
    • S. Black (1999), "Do Better Schools Matter? Parental Valuation of Elementary Education", Quarterly Journal of Economics, 114(2):577-599
    • W. Evans and D. Lien (2005), "Does Prenatal Care Improve Birth Outcomes? Evidence from the PAT Bus Strike", Journal of Econometrics, 125(1-2):207-239
    • J. Potter, C. Schmertmann, & S. Cavenaghi (2002), "Fertility and Development: Evidence from Brazil", Demography, 39(4):739-761

 

Week 15 (May 7):  Wrap-Up

 

  • Introduction/Discussion: Mexico’s Seguro Popular de Salud case
  • “Triple robustness” to minimize threats to internal validity
  • The fuzzy regression discontinuity design

 

  • KEY READINGS:
    • G. King, et al (2007), “A ‘Politically Robust’ Experimental Design for Public Policy Evaluation, with Application to the Mexican Universal Health Insurance Program”, Journal of Policy Analysis and Management, 26(3):479-506
    • Handbook on Impact Evaluation, “Regression Discontinuity Design”, Ch 16

LAS 381 • Human Fertility

41020 • Fall 2009
Meets TH 300pm-600pm MAI 1704
(also listed as SOC 389K )
show description

See attachment

bottom border