Tuesday, January 03, 2006

Evaluating Teachers

Last week, I posted a piece called the Variable Value of Teachers which, according to Site Meter, got a lot of hits. The piece also garnered some thoughtful comments from a couple of people. One comment from Mike included this:
I can't imagine a school where merit pay did not quickly degenerate into a cesspool of corruption--in other words, those who were rewarded would be among the worst teachers.

And how do we measure such merit? Standardized test scores? Fine with me if the scores measured are my GT kids. But I also have "regular" classes, wherein enough kids (and parents--do they actually have them? Hard to tell) just don't care to screw up any hope of ever getting a merit raise. In such a system, a teacher's ability and effort matter not at all in any case. I teach up a storm in every class, but my GT kids are also involved in their educations, hence, results. In the other classes, well, some will do well, others won't, and it's nothing more difficult to define than the luck of the draw--class personality.

After I responded that other white collar work and teaching are very similar in that no easy metrics exist to determine teacher quality and merit, I recieved this very thoughtful comment from Dean Pence:
The difference between white collar private sector work and teaching is that the consumer is in a good position to evaluate the quality of the output. Of course, this is not as easy with accounting as it is with say, canned soup. But still, an accountant whose work consistently passes review will help bring in new clients and add luster to the firm's reputation.

You have a much more difficult time determining the quality of your child's education DURING CHILDHOOD.

An adult is obviously learned or not most of the time, but a child is a work in progress. Rather than using test scores or the ability to regurgitate facts, many would judge the quality of a child's education by the level of UNDERSTANDING achieved.

I thought I would take the time to explain the kind of evaluation scenario I am talking about when judging merit pay and bonuses for teachers. I hope this post may address also the issues raised by commenters.

First, my understanding of current teacher pay scales is that they are based largely, although by no means solely, on ojective criteria, such as time in service, academic degrees of the teacher, certification status, and some others. Objective criteria are important, but my question is whether these criteria lead to student achievement. Certainly a teacher with 5 years of experience will make a more significant impact on student achievement than a 2 year teacher. But does a teacher with 25 years of experience make a significantly measurable difference in student achievement than a 15 year teacher? In other words, does not the law of diminishing returns apply? Some people, like Jay Greene in his book, Education Myths, point to studies showing exactly that.

Mike made the point that he worries that merit pay would become a "cesspool of corruption." I am not sure exactly what Mike means, but I can extrapolate from arguments I have heard before, i.e. that evaluations to determine merit pay will be too subjective, based solely on one or two people's opinion, and woe to the person on the wrong side of those people. A truly valid concern that can be addressed by making the subjective evaluation of a principal or department head merely one part of an evaluative process.

The best analogy I can give is to the performance and advancement system of the U.S. Navy. In the Navy, I was subjected to an advancement system (which is pay raises with promotion) that included

  1. objective criteria such as my time in the Navy and my time in my current paygrade
  2. academic preparation, that is passing a test on the subject material of my speciality
  3. leadership preparation, a combination of a test and evaluation by my superiors
  4. points for additional knowledge, such as becomeing surface warfare, air warfare or submarine service qualified (a lot of practical knowlege about shipboard operations)
  5. points for medals and commendations


These criteria all went to an advancement score and depending on how many positions the Navy had to fill for the next paygrade up in my speciality, I was either advanced or not. In many ways, the system is like a merit pay, bonus structure I envision.

So how does a school system embrace merit pay and bonuses (discussed in a different post) without degenerating into a cesspool of popularity? First, by acknowledging that some aspects of a popularity contest are likely and there is little that can be done about it. But you can limit the "beauty contest" aspects by spreading the subjective evaulation material around. Not only would the principal and perhaps department head be involved, but also parents, reasonably aged students, the teachers' peers and those teachers who get the students the next year.

Objective criteria, such as time in service and academic credentials can, and probably should, still matter, but on a lesser scale. But some subjective criteria can be measured, such as a teacher's responsiveness to parental inquiries, demonstrating the ability to tailor lessons to the students in that class or within classes, etc. These subjective criteria can be judged on a scale referencing a set norm. Thus on a 1 to 5 scale, three would be average performance of all teachers. Thus the question is more like, "compared to other teachers, how effective is this teacher in responding to parental inquiries?" Again, the matter is subjective to the teacher in question, but scaled in an objective fashion.

But if you ask a pool of stakeholders, parents, colleagues, successor teachers, students, etc. about the quality and merit of a teacher, you are likely to get a much better idea of the value and worth of that teacher to the community at large, rather than just the opinion of one or two people. You also have the ability for the consumers of education, children and their parents to have some input on personnel matters, something that is likely to increase parental involvement in teh education of their children.

Of course, critics will no doubt say my ideas are expensive--and they are initially. It will take time and money to develop an evaluative process dependent on input from a large body of people. But on the other hand, once in place, the system can be adjusted as necessary with little expense.

Critics will also say that some stakeholders will not respond, i.e. disengaged parents. Also true, but does the education establishment not have a vested interest in getting all stakeholders involved. I won't accept the cop out excuse that just because parents aren't invovled now they won't be involved later if given an opportunity to make a real difference. As we have seen in voucher programs across the country, previously indifferent or disengaged parents become quite involved when presented with an opportunity to make a difference in their child's and other children's education.

Finally, as Dean Pence pointed out, critics will charge that you cannot evaluate a child's education like other consumer products and services. To this I say, Hogwash!!! Just because a child's education is ongoing does not mean there are points in the process in which evaluations can be made. Afterall, we have standardized testing to do just that, measure a child's minimal education achievement at some given point in time. Why can't we do the same for a teacher's performance as an educator at a given point in time. We routinely evaluate all sorts of professions relative to a given time frame or at a given point in time. The argument that you cannot evaluate a teacher's effectiveness because you can't evaluate the end product, the educated child, until later belies the common understanding of an annual performance review. Why do we not evaluate teachers in the same manner that we evaluate other professions?

The natural response will be, following on the heels of my last post, that a teacher has no control over the quality of the inputs into the system, i.e. that children have varying motivations and skills and the teacher has no control over who shows up in their classroom. True, but many other professionals don't have that much discretion either. Does a public defender, a lawyer, have any control over the quality of her clients? Does a doctor? A private practice lawyer? An accountant, a police officer, a firefighter. No, they have little practical control over the inputs into their professional lives, yet we have no trouble paying an excellent lawyer for the value of his services. Nor a doctor. Those prices for services are set by the marketplace.

A teacher's value to the educational community can, likewise be measure by the marketplace. Teacher quality will improve, and with growing professionalism, so too will the value of a merit based system of pay.

No comments: