Syndicate content

Add new comment

Submitted by Andrew Schein on

Long time reader, first time poster.

I have a slightly off-the-wall question about using a joint test of orthogonality.

Say I’m looking at dating website profiles. I note 20 adjective that are much more likely to be used on females’ profiles than males’ profiles. I note another 20 adjective that are much more likely to be used on males’ profiles than females’ profiles.

I then roll out a design change across the site that I hypothesise will reduce the use of ‘gendered language’. I want to test whether it has done so. Imagine that we pushed the redesign to only half of our users, so this is a proper randomised A/B test.

Should I use a joint test of orthogonality?

Here’s how I would envision it working:
- Using the same list of words that the exploratory analysis has already found, we would make each word an indicator variable which takes the value of 1 if the word is used in a profile and 0 if it is not. We would then run a joint test of orthogonality:
- We take our set of 40 words (X1, X2, …, X40) and run the following regression:
- Female = a + b1*X1 + b2*X2 + b3*X3 + ….+b40*X40 +u
- We then test the joint hypothesis b1=b2=b3=…=b40=0 as a linear regression, with an F-test.

But how should I use the indicator variable for whether the user has been ‘treated’ or not? Interacted with each word-indicator variable?

Many thanks in advance,
Andrew