Value-Added, NY Schools and What's Next

The Edusphere has recently been abuzz with concerns regarding a pilot program that will take place in New York City schools in which some teachers will be adjudged by test scores. Kevin Carey of The Quick and the Ed talks about the experiment and some of the reaction.

What got my attention about Carey's post about the value added methodology that will be used in New York was his reference to Moneyball, the book about the Oakland A's Billy Beane, in which Beane used different statistical measurements (and past performance) to find value in overlooked ball players. Carey writes:

And at some point I realized that the underlying premise of Moneyball and the promise of value-added were the same: using empirical data to fundamentally change and improve a labor market. Instead of relying on human observations of characteristics, with all the biases and errors that result, focus on outcomes instead.

. Carey has caught some flack for his posting. I had pretty much the same thought about teacher evaluations which led to a project that I have started and not gotten very far on. (See Teacher Quality Stats topic on the left). To be honest, I have had to do a lot more research on the matter and I have been looking at some data.

But like I noted in my last post, we have to look beyond how we currently do things and quite frankly, every other industry and yes, even govenrmental departments, look long and hard at outcomes now. Education seems to be the last bastion where outcomes don't take precedence in evaluating the performance of the labor market.

Just as Billy Beane's theories and practices have changed the labor market in Major League Baseball (as has steroids), looking at value-added statistical analysis has the potential to radically alter not only the labor market, but also the manner in which we train, prepare, recruit and pay teachers. Those who are most successful will get more pay, those who are not will at least have a clue where to look. Instead of gut reactions and a good show, there will be hard data to look at. As Carey writes:

One of the biggest problems with the teacher labor market is that the top teachers--the ones who are one or more standard deviations above the mean in terms of effectiveness--are criminally underpaid, and have no way of demonstrating their real value to the labor market. Their unions, however, are totally aghast at the prospect. Randi Weingarten, head of the United Federation of Teachers (and rumoured to be next head the national AFT) said:

“Any real educator can know within five minutes of walking into a classroom if a teacher is effective."

This is the equivalent of the scouts and general managers in Moneyball who were always on the lookout for the "good body," the "five-tool guy," the player who just looked like a major leaguer. As everyone now knows, they were profoundly mistaken, and people like the Oakland A's Billy Beane were able to exploit the market distortions that resulted.

What we're seeing in New York City today is all the major challenges of 21st century K-12 teacher policy being played out in real time. Value-added methods are still very much in development, subject to limitations of standardized tests, among many things. But in the long run, there will only be more, better information about student performance, along with newer, faster ways of analyzing that information and drawing increasingly accurate conclusions about how well teachers are doing their jobs. At some point the methodological debates will be resolved and the margins of error whittled down the satisfaction of reasonable people.

In reality, the only debate that is worth having and that will be important is the methodology argument. Yes, there will be the obstructionists who argue that we shouldn't do value-added at all, but we are already on that path, and it is a path that is accepted by just about everyone but the teachers' unions, including many really good teachers.

The fact is that just as in any other labor sub-pool, the newest practitioners will need help to get started and to be fair a little bit of leeway. But the strong performers will be evident quickly and their performance will be backed up by hard data. The poor performers, those who are in teh bottom five percent of their cohort, will be quickly identified so that they can either A) improve or B) move into another career more suited to their talents.

Make no mistake though, value-added is coming and the teachers had better start coming to grips with it. The unions would be far better served to be a partner in developing an adequate methodology for computing value-added than being an obstructionist.