Syndicate content

Recent comments

  • Reply to: Finally, a way to do easy randomization inference in Stata!   15 hours 14 min ago

    Hi! I wondered if you ever solved this? I have a similar problem where I have aggregate state-level data from 2000-2012 & a policy that was implemented in some states in 2007 and others in 2009. I have a similar dichotomous variable - treatment - which is 1 during treatment years in treated states, and 0 otherwise). The code for my regression is:

    xtreg outcome treatment i.year i.state, vce(robust)

    For the ritest, I understand I need an interaction variable so that I'm evaluating _b[c.treatment#c.postperiod]. But my problem is, given the postperiod differs between states (sometimes it is 2007+ and sometimes 2009+) & we don't know what it would have been for the control countries, how do I model this? I think I need to program a two-stage randomization process (randomized to control/treatment & randomized to 2007/2009) - but I am struggling with writing this program. Specifically, I can't tie the second randomization (randomization of years in the postperiod variable) into the program properly so that this part of randomization is implemented.

    Thanks for any advice!

  • Reply to: Finally, a way to do easy randomization inference in Stata!   1 day 11 hours ago

    In the past months I received several emails asking about how to implement the pairwise tests for experiments with more than two treatment groups (Point 7 above). I thus added a new option to -ritest- that makes this easy.

    Using the option fixlevels(list of levels), one can restrict the re-randomization to hold all observations fixed that have the specified values for the treatment variable. In terms of the example David gave in this post, this could be simply done by specifying:

    ritest treatment (_b[0.treatment]-_b[1.treatment]), cluster(clustervar) strata(stratavar) fixlevels(2): reg y i.treatment

    The way the option is implemented internally is fairly straightforward, by creating a temporary stratum for each of the specified levels during the re-randomization (Jason Kerwins's idea). The new version of -ritest- can be obtained from my GitHub. Via Stata:

    net describe ritest, from(https://raw.githubusercontent.com/simonheb/ritest/master/)

    Please don't hesitate to contact me for bug reports or if you have questions.

  • Reply to: Make Your Research Known – 10 Tools to Increase Consumption of Your Research   2 days 14 hours ago

    Very useful the information

  • Reply to: Tools of the Trade: a joint test of orthogonality when testing for balance   1 week 1 day ago
    Hi Andrew,
    I don't think you want a joint orthogonality test here - you aren't trying to test if none of the adjectives are related to gender. Instead you are testing if your treatment reduces the use of gendered adjectives. So there would be two approaches I would take to doing this:
    1. Just define a count of the number of male adjectives used on male profiles (call this M20), and a count of the number of female adjectives used on female profiles (call this F20), and then run regressions like:
    M20 = a+b*Treat + e
    F20 = a+b*Treat + e
    This will show whether your treatment succeeds in getting males to use the male adjectives less, and females to use the female adjectives less. (I would run these regressions separately by gender, but you could also pool together males and females and just create a variable that is the number of gendered adjectives of your gender you use).

    2. If you are particularly interested in whether the treatment reduces the use of particular adjectives, then you can run the 40 regressions of the form:
    X1 = a + b*Treat + e
    or X1 = a+b*Treat + c*Female + d*Treat*Female + e
    and so on up to X40
    and then use a multiple testing correction to account for the fact you are doing 40 different tests.
  • Reply to: Tools of the Trade: a joint test of orthogonality when testing for balance   1 week 2 days ago

    Long time reader, first time poster.

    I have a slightly off-the-wall question about using a joint test of orthogonality.

    Say I’m looking at dating website profiles. I note 20 adjective that are much more likely to be used on females’ profiles than males’ profiles. I note another 20 adjective that are much more likely to be used on males’ profiles than females’ profiles.

    I then roll out a design change across the site that I hypothesise will reduce the use of ‘gendered language’. I want to test whether it has done so. Imagine that we pushed the redesign to only half of our users, so this is a proper randomised A/B test.

    Should I use a joint test of orthogonality?

    Here’s how I would envision it working:
    - Using the same list of words that the exploratory analysis has already found, we would make each word an indicator variable which takes the value of 1 if the word is used in a profile and 0 if it is not. We would then run a joint test of orthogonality:
    - We take our set of 40 words (X1, X2, …, X40) and run the following regression:
    - Female = a + b1*X1 + b2*X2 + b3*X3 + ….+b40*X40 +u
    - We then test the joint hypothesis b1=b2=b3=…=b40=0 as a linear regression, with an F-test.

    But how should I use the indicator variable for whether the user has been ‘treated’ or not? Interacted with each word-indicator variable?

    Many thanks in advance,
    Andrew