mbietz January 17th, 2007
The subject pool might be a bigger problem than I thought at first. It seems from what I’ve been hearing that the Michigan Paid Experiments pool isn’t being maintained. The software is still the same home-grown stuff that has been unreliable in the past (and lacks many of the features I want). I also worry about the composition of the pool. I don’t know the last time that there was any recruitment into the pool, so the addresses seem to be getting stale. The sign-up functions don’t have any potential for any level of pre-screening.
I talked to Yan today about getting access to the ISR pool, but my study doesn’t qualify. To preserve the purity of the pool, they don’t allow any studies that use “induced incentives” or deception.
I’m thinking I will probably have to do my own recruitment and scheduling without the pool. I don’t think it will be hard to generate interest, but I might end up having to write my own code to do the scheduling (and maybe a little bit of pre-screening). Not hard, but it would be one more thing to do. Would I need to go through IRB with something like this?
mbietz January 17th, 2007
I’m thinking today about incentives. In my first experiment, subjects were given the incentive that the PPT they created would be graded on a scale of 1 to 10, and if they received a 10, they would get a $5 bonus, a 9 would earn $4, down to a 6 would earn $1 bonus. The point of the incentive is to give participants a financial as well as intrisic motivation to do well at the task. Essentially, I want to give them a reason to think about the feedback and the process. This also gives them motivation to spend time editing the document before they turn it in.
I’ve been wondering whether I want to change the incentive. I could tell subjects that anyone whose graded essays are in the top X% of all the essays would get a $5 bonus. There are two changes here. First, it’s an all or nothing bonus, rather than having gradations. Second, the bonus would also include an element of competition among the participants. I’m not sure if this would be good or bad–in some ways I could see it happening that subjects would start monitoring each other for cues about how much work to give (especially since I’m planning on running several subjects at a time in parallel).
I talked to Yan Chen about this a bit today, and she didn’t have a strong opinion about which option I should use. She pointed me to a review article:
- Camerer, Colin F., and Robin Hogarth. 1999. “The Effects of Financial Incentives in Economics Experiments: A Review and Capital-Labor-Production Framework.” Journal of Risk and Uncertainty, 7-42.
I think for the time being I’m going to keep the incentives the same (not the least so that I have as much commonality as possible with the first experiment).