Matthew J. Bietz

Yesterday I passed my dissertation defense!

Already it feels like a distant blurry memory. It was great having so many friends there, and my parents were able to drive in from Peoria (although when Dad raised his hand to ask a question, I wondered for a second why I’d invited them!). This was the last major milestone. I’ll be making revisions and dealing with the mountain of paperwork for a little while longer, but soon I’ll be Dr. Bietz. (Which, as dad reminded me, has a much better ring to it than “Master Bietz.”)

Two weeks from today. October 16. 1:00pm (eastern). I’ll be defending my dissertation. It’s hard to believe it’s almost here. If all goes well (and even if it doesn’t), soon I won’t be a student any more. If you count pre-school, I’ve been a student for the past 32 years. Yeah, there was that year working in the library at Stony Brook, but even then I was taking classes, it was a university library, and I knew that I was going back to school. I didn’t stop being a student, I just took a sabbatical.

I’m looking forward to being grown up. But it still feels weird.

I got this graph today. I’m not going to go into what it means right now. Suffice it to say that it’s real data from my experiments. It’s preliminary analysis, but I think there’s something there.

I started working on this study ages ago. It’s been months of planning and pilot testing and running subjects and data crunching to get to this point. Months of not knowing whether or not there was anything to see. Was I barking up the wrong tree? Does my argument make any sense? Am I even measuring what I think I’m measuring?

But today I typed a few commands into my trusty R Console, and saw those magic asterisks that said that my model was significant. Kind of anti-climactic, in a way. But still way better than not seeing the magic asterisks.

Of course, I’m not done yet. I still have to run all kinds of error checking and double-check my work. And I have to go over all of the lab procedures to make sure that everything worked the way it was intended. And then there’s that little step of writing it all up. But today I have a graph that I can explain using my theory. Yeah!

The subject pool might be a bigger problem than I thought at first. It seems from what I’ve been hearing that the Michigan Paid Experiments pool isn’t being maintained. The software is still the same home-grown stuff that has been unreliable in the past (and lacks many of the features I want). I also worry about the composition of the pool. I don’t know the last time that there was any recruitment into the pool, so the addresses seem to be getting stale. The sign-up functions don’t have any potential for any level of pre-screening.

I talked to Yan today about getting access to the ISR pool, but my study doesn’t qualify. To preserve the purity of the pool, they don’t allow any studies that use “induced incentives” or deception.

I’m thinking I will probably have to do my own recruitment and scheduling without the pool. I don’t think it will be hard to generate interest, but I might end up having to write my own code to do the scheduling (and maybe a little bit of pre-screening). Not hard, but it would be one more thing to do. Would I need to go through IRB with something like this?

I’m thinking today about incentives. In my first experiment, subjects were given the incentive that the PPT they created would be graded on a scale of 1 to 10, and if they received a 10, they would get a $5 bonus, a 9 would earn $4, down to a 6 would earn $1 bonus. The point of the incentive is to give participants a financial as well as intrisic motivation to do well at the task. Essentially, I want to give them a reason to think about the feedback and the process. This also gives them motivation to spend time editing the document before they turn it in.

I’ve been wondering whether I want to change the incentive. I could tell subjects that anyone whose graded essays are in the top X% of all the essays would get a $5 bonus. There are two changes here. First, it’s an all or nothing bonus, rather than having gradations. Second, the bonus would also include an element of competition among the participants. I’m not sure if this would be good or bad–in some ways I could see it happening that subjects would start monitoring each other for cues about how much work to give (especially since I’m planning on running several subjects at a time in parallel).

I talked to Yan Chen about this a bit today, and she didn’t have a strong opinion about which option I should use. She pointed me to a review article:

• Camerer, Colin F., and Robin Hogarth. 1999. “The Effects of Financial Incentives in Economics Experiments: A Review and Capital-Labor-Production Framework.” Journal of Risk and Uncertainty, 7-42.

I think for the time being I’m going to keep the incentives the same (not the least so that I have as much commonality as possible with the first experiment).

Today was mostly spent working on the procedure/overview document for my 2nd experiment. I need to get this done tomorrow so that I can get the IRB application in by next Tuesday (the due date for full reviews if I’m going to do the study in September). I don’t think I’ll need full board review, but I don’t want to chance it. I also want to get this document (along with my analyses from Exp.#1) to my committee ASAP so I can get their input/approval.

Good news today – poster got accepted to CSCW. Now I need to get all the funding lined up.

Overall, an OK day for writing.