Moneyball and Money Teachers

A few years ago, I had a boss who was constantly hammering into those of us working for him that numbers matter, statistics matter and we can make more money if we pay attention to metrics and past performance. Because we were in a political business, that of managing software and political action committees, I thought he was nuts. But, trying to keep an open mind, I went along with a couple of his experiments, examining costs to achieve certain results, time spent on activities, and other ideas that led to some hard numbers about our performance. The results were impressive, we were able to cut costs, cut time on common tasks by spending the time and money to build tools and processes to cut time and reduce error. As a result, we were able to take more clients and make more money.

I have been revisiting the idea by reading Moneyball, the fabulous book by Michael Lewis about how the Oakland A's baseball team that was able to win so many games with a payroll that was a fraction of the big teams' salaries. Billy Beane, the general manager of the A's instituted a program where he studied stats and examined the data behind the most successful players and games. With his system of hard data examination, he was able to achieve practical miracles that countered everything "baseball experts," including his own staff, thought would happen.

Moneyball, this post by a new blog, The Common School, and this post by Brett Pawlowski at the DeHaviland Blog have gotten me thinking about what metrics we can use in schools to measure teacher effectiveness.

Right now we have lots of metrics and measurements. The NCLB proficiency ratings, state school assessments, individual test scores, and on and on and on. But do we really have stats on teachers? If we as a society do, we certainly don't talk about them and we certainly use them to determine what works and what does not.

In Moneyball, Billy Beane and his team learned that among the most important factors in winning baseball games was on-base percentage (that is how likely a person is going to get on base every time they come to the plate--this includes walks) and slugging percentage, (that is the number of bases a person is likely to get for each hit). Teams and players that had high percentages were more likely to win games because they would score runs, you can't score runs if you don't have players on base and if players are on base they are not adding to the out count.

So if the goal is the successful education of children, then we need to determine what factors increase the likelihood of a well-educated child. As the Common Room recently noted (and many others) the single most important factor is the quality of the teacher.

First of all, teacher experience and licensure rank near the top of the list while class size reduction and teacher's having master's degrees rank at the bottom. But more important than the ranking is how much smaller of an impact class-size has than some of these other factors. For example, having a teacher who is not a novice (7.2% - 9.1% SD) exerts an influence 3½ to 7 times greater than the impact of reducing class size by 5 students (1.0% -2.5% SD).

So that is one factor, but experience is hardly the only determinant of quality teachers. So what are some of the other factors, i.e. statistics, that can be determined and applied.

Well, some obvious ones would be proficiency on state/federally mandated tests, i.e. are the teachers students below, at or above grade level. Another would be the difference in these numbers relative to their peers with the same task/experience level. Thus you would be comparing 6th grade Math teachers with 4 years of experience with other 6th grade math teachers with 4 years of experience.

Keeping with that theme, how many students changed their proficience level relative to last year. For example, how many (what percentage of students) went from basic to proficient or vice versa? Now these may be a little broad, so if the criteria were narrowed to look at test scores themselve, i.e. the raw scale change in scores.

Of course, of the the most common complaints from teachers is that they have little control over the assingment of students to their classroom. So measurements like those above may be skewed unfairly by poor student assignment luck. But such concerns can be alleviated by pre-and post testing, that is test given at teh start and end of the year on matters of curricula. You test students at teh start of the year to see what they know and the same thing at the end of the year. Good teachers should be able to increase the knowledge base better than their peers. These kinds of stats would allow for each teacher to be judged on what they accomplished with what they had. Plus as an added bonus the teacher can use the results to help tailor curricula.

There have been a number of times where I discussed the need for treating teachers as professional and having teachers act like professionals. One of the most important things for us to do is find a way to empiracally study teachers, i.e. come up with a way of measuring their effeciveness. Teachers who are effective should be examined to see what can be replicated and I refuse to believe that successful teachers are "magical" in some indefinable way that cannot be replicated.

But we don't have any data.

Any other suggestions for measurement?