Syndicate content

Add new comment

A Curated List of Our Postings on Technical Topics – Your One-Stop Shop for Methodology

David McKenzie's picture
This is a curated list of our technical postings, to serve as a one-stop shop for your technical reading. I’ve focused here on our posts on methodological issues in impact evaluation – we also have a whole lot of posts on how to conduct surveys and measure certain concepts that I’ll leave for another time. Updated February 17, 2017.
Random Assignment
Allocating treatment and control with multiple applications per applicant and ranked choices
Is optimization just re-randomization redux? Thoughts on the "don't randomize, optimize" papers
Be an optimista, not a randomista, when you have small samples
Tips for randomization in the wild: adding a waitlist
How to randomize in the field
Stratified randomization and the FIFA world cup
Doing stratified randomization with uneven numbers in the Strata
How to randomize using many baseline variables
Public randomization ceremonies
Designing experiments to measure spillover effects
Mechanism experiments and opening up the black box
Sample weights and RCT design
Response to the policymaker complaint that randomized experiments take too much time
Have RCTs taken over development economics?
Definitions in RCTs with interference
Pre-analysis plans and reporting
Pre-registration of studies to avoid fishing and allow transparent discovery
A joint test of orthogonality when testing for baseline balance
A pre-analysis plan check-list
The New Trial Registries
What isn’t reported in impact evaluations but maybe should be
Randomization checks: testing for joint orthogonality
An addendum to pre-analysis plans: pre-specifying when not to use data
Propensity Score Matching
Guido Imbens on clustering standard errors with matching
Testing different matching estimators as applied to job training programs
The covariate balanced propensity score
The often unspoken assumptions behind diff-in-diff
Regression Discontinuity
Curves in all the wrong places: Gelman and Imbens on why not to use higher-order polynomials in RD
Regression discontinuity with an implicit index
Tools of the trade: the regression kink design
Other Evaluation Methods
Evaluating an Argentine tourism policy using synthetic controls: tan linda que enamora?
Impact as narrative
The synthetic control method, as applied to regulatory reforms
Using spatial variation in program performance to identify impacts
Small n impact evaluation methods
Can we trust shoestring evaluations?
Using case studies to explore and explain complex interventions
Another reason to prefer Ancova: dealing with measurement changes between baseline and follow-up
Endogenous stratification: the surprisingly easy way to bias your heterogeneous treatment effects and what to do instead
Why is difference-in-difference estimation still so popular in experimental analysis?
Regression adjustment in randomized experiments (part one, part two)
When to use survey weights in analysis
Adjustments for multiple hypothesis testing
Bounding approaches to deal with attrition
Linear probability models versus probits
Dealing with multiple lotteries
Estimating standard errors with small clusters (part one, part two)
Decomposition methods
Estimation of treatment effects with incomplete compliance
What does Alwyn Young's paper mean for analysis of experiments?
Winsorizing, testing balance, and dealing with incomplete take-up
You ran a field experiment, should you then run a regression?
Power Calculations and Improving Power
Should I work with only a subsample of my control group if I have take-up problems?
Power calculations: what software should I use?
Does the intra-cluster correlation matter for power calculations if I am going to cluster my standard errors?
Power calculations for propensity score matching
Power calculations 101: dealing with incomplete take-up
Collecting more rounds of data to boost power
Improving power in small samples
Did you do your power calculations using standard deviations? Do them again.
Power calculations for regression discontinuity (part 1, part 2, part 3)
Power calculation software for randomized saturation experiments
On External Validity
Getting beyond the mirage of external validity
All those external validity issues with impacts? They apply to costs too
External validity as seen from other quantitative social sciences and the gaps in our practices
Towards a more systematic approach to external validity: understanding site selection bias
Weighting for external validity
Will that successful intervention over there get results here?
Learn to live without external validity
Why the external validity of regression estimates can be less than you think
Why similarity is the wrong concept for external validity
A rant on the external validity double standard
Towards policy irrelevance: on the experimental arms race
What's wrong in how we do impact evaluation?
What's in a title? Signaling external validity through titles in development economics
Jargony Terms in Impact Evaluations
A proposed taxonomy of behavioral responses to evaluation
Quantifying the Hawthorne effect
The Hawthorne Effect
The John Henry Effect
Placebo effects
Clinical Equipoise
Stata Tricks
Generating regression and summary statistics tables in Stata
Graphing impacts with Standard Error Bars
Calculating the intra-cluster correlation
Generating regression and summary statistics tables in Stata: A checklist and code
Worm wars: the anthology
Worm wars: a review of the reanalysis of the Miguel and Kremer deworming study
Response to Brown and Wood's response
Brown and Woods response on "how scientific are scientific replications"
how scientific are scientific replications?
The infinite loop of failure of replication in economics
More replication in economics
Systematic reviews and meta-analysis
how systematic is that systematic review? The case of learning outcomes
How standard is a standard deviation? A cautionary note on using SDs to compare across impact evaluations
should we give up on SDs for measuring effect size?
What do 600 papers on 20 types of interventions tell us about what types of interventions generalize?
If you want your study included in a systematic review, this is what you should report

Book Reviews for Books on Impact Evaluation
Review of Imbens and Rubin's Causal Inference
Review of Glennerster and Takavarasha's Running Randomized Experiments
Review of Gerber and Green's Field Experiments
Review of Karlan and Appel's Failing in the Field
Review of Tim Ogden's Experiments in Development from Every Angle

Getting Published
11 tips for making a short presentation based on your research