_{Photo Credit: lunch note on flickr photosharing}

In my first part, I discussed how students could under-perform on a high stakes test, like the California High School Exit Exam ,in a way that has nothing to do with their true ability or knowledge. In the second, I discussed the perversion of incentive based systems. Now, we move on to cheating…

Have you ever worked in a place where food was being stolen out of the common refrigerator? Ever wonder how they justify that behavior? Ariely discusses how folks in some of our finest institutions of higher learning think nothing of taking a can of coke that he planted in the staff fridge, but will leave behind dollar bills he left there. He supposes this is because taking money is stealing, and taking a coke is something else in the thief’s mind. He then expanded the study to see how folks did when being incentivized for correctly doing math problem and “self-reporting” their answers, and concluded that folks cheat. Leavitt and company at Freakonomics already explored how cheating occurred in Chicago Public schools testing by teachers. Ariely is looking more at the why, and Leavitt at the how.

In the first two parts I talked about how imbuing tests with too much value could pervert behavior, but what about if the people most affected by a test score, and the most incentivized, did not see value in it (not far-fetched given stereotype threat)? Most of Ariely’s work is geared toward why traders in exotic financial instruments that are far removed from real cash may start treating them like monopoly money. Ariely concludes that the farther removed from money (something of a perceived real value), the more likely folks are to cheat and steal.

Looking at Ariely, I think we can see that if a teacher thinks, “Well these scores don’t mean a darn thing,” they could justify cheating in a situation where they felt their performance is being judged on it. I don’t think it would be everyone, but it would only take a few to undermine confidence in the system, which is not very high in many quarters already.

Links

Testing: Stereotype Threat and the Perversion of Incentives, Part I

Testing: Stereotype Thread and the Perversion of Incentives, Part II

Testing…Testing, testing…TESTING | Reflections on Teaching

Inside School Research: California’s Exit-Exam Policy: A Study in Inequity

Predictably Irrational Blog

Op-Ed Contributor – What’s the Value of a Big Bonus? – NYTimes.com

Dan Ariely Takes on ‘Irrational’ Economic Impulses : NPR

Freakonomics Blog at NYTimes

In my first part, I discussed how students could under-perform on a high stakes test, like the California High School Exit Exam, in a way that has nothing to do with their true ability or knowledge.

That’s students, it was an article on excessive bonuses that caught my eye with regard to teachers. Economist Dan Ariely, in Predictably Irrational: The Hidden Forces That Shape Our Decisions takes a look at how excessive bonuses can actually pervert effects.  Here’s a quote from a NY Times guest op-ed piece:

What would you expect the results to be? When we posed this question to a group of business students, they said they expected performance to improve with the amount of the reward. But this was not what we found. The people offered medium bonuses performed no better, or worse, than those offered low bonuses. But what was most interesting was that the group offered the biggest bonus did worse than the other two groups across all the tasks.

So, this definitely has implications for teacher incentive pay. If you make it small it’ll help, if you make it medium, it’s the same, and if you make it big, it’ll make it worse. In other talks he’s discussed how stressed out the folks in the India version of his experiments got when they were working towards the highest bonus (equivalent to 6 months pay in India).

He points out that this anger, resentment, worry, and stress, was echoed in financial institutions that relied heavily on bonus incentives during the last quarter of the year. I think it’s the same feeling I’ve had at during previous testing seasons.

So it turns out that social pressure has the same effect that money has. It motivates people, especially when the tasks at hand require only effort and no skill. But it can provide stress, too, and at some point that stress overwhelms the motivating influence.

When I recently presented these results to a group of banking executives, they assured me that their own work and that of their employees would not follow this pattern. (I pointed out that with the right research budget, and their participation, we could examine this assertion. They weren’t that interested.) But I suspect that they were too quick to discount our results. For most bankers, a multimillion-dollar compensation package could easily be counterproductive. Maybe that will be some comfort to the boards at UBS and Goldman Sachs.

I believe that offers like the D.C.P.S. for $100,000 salaries are the equivalent to the million dollar compensation packages that have become the norm in financial institutions, and will likely have the same effect, which is not the one we are all looking for.

Another idea he touches on is how monetary incentives can undermine social relationships. That comes from an experiment on paying late fees at an Israeli daycare center. When fines were implemented, parents started to “pay for the privilege” of a late pickup. Even after the policy reverted back to “no-fee” which it had been at the start, parents were late more often than they had been before the rules changed. Incentives perverted the social contract.

I think we need to think about this in terms of our students and high stakes testing, and how it might be intersecting with “stereotype threat”. The implications for what might happen when we implement any more incentive based pay systems, especially for teachers, are pretty clear, you won’t get what you pay for.

Links

Testing: Stereotype Threat and the Perversion of Incentives, Part I

Testing…Testing, testing…TESTING | Reflections on Teaching

Predictably Irrational Blog

Op-Ed Contributor – What’s the Value of a Big Bonus? – NYTimes.com

Dan Ariely Takes on ‘Irrational’ Economic Impulses : NPR

Freakonomics Blog at NYTimes

As I’ve shared already, it’s testing season. This is the second year I’ve been a prep teacher, and therefore not had a specific group of students whose scores I was “tied” to. I’ve also noted that I’m not feeling as angry, stressed, annoyed this testing season, and I think the two are definitely connected. Lots of interesting stuff floating out there about high stakes testing and incentives, so I thought this would be a great time to look at this…

First up, for those of you who missed it, Stanford University released a study on California’s High School Exit Exam (CAHSEE), which found it was increasing dropping out among minorities and females as they felt they couldn’t pass the test.  Here’s some interesting background from Ed Week’s new blog, Inside School Research:

Just as intriguing, though, is the researchers’ explanation for why the effects hit some groups of students harder than others: They chalk it up to “stereotype threat.”

…when the research team examined students’ previous scores on other state tests, they turned up some evidence that minority students and women had underperformed on particular sections of the state exit exam. Women fared worse than their earlier performance might have predicted, for example, on the math portion. Asian students did worse-than-expected on English-language arts.

Think about the implication for our student’s test scores. If they think the test is critical for their future, then in aggregate, they are likely to perform at a level lower than past performance would indicate. That means if they have internalized the test as being “high stakes” the more likely it is to be inaccurate in actually measuring knowledge. The implications are troubling. I think that students didn’t feel the “threat” until CASHEE because while the stakes are high for educators, testing doesn’t have a real direct link for kids. It is used in some cases to assign double blocks of Language Arts and/or Mathematics in middle school, but CASHEE Is where the rubber hits the road for students.

Links

Testing…Testing, testing…TESTING | Reflections on Teaching

Inside School Research: California’s Exit-Exam Policy: A Study in Inequity