Value-Added, NY Schools and What's Next

The Edusphere has recently been abuzz with concerns regarding a pilot program that will take place in New York City schools in which some teachers will be adjudged by test scores. Kevin Carey of The Quick and the Ed talks about the experiment and some of the reaction.

What got my attention about Carey's post about the value added methodology that will be used in New York was his reference to Moneyball, the book about the Oakland A's Billy Beane, in which Beane used different statistical measurements (and past performance) to find value in overlooked ball players. Carey writes:

And at some point I realized that the underlying premise of Moneyball and the promise of value-added were the same: using empirical data to fundamentally change and improve a labor market. Instead of relying on human observations of characteristics, with all the biases and errors that result, focus on outcomes instead.

. Carey has caught some flack for his posting. I had pretty much the same thought about teacher evaluations which led to a project that I have started and not gotten very far on. (See Teacher Quality Stats topic on the left). To be honest, I have had to do a lot more research on the matter and I have been looking at some data.

But like I noted in my last post, we have to look beyond how we currently do things and quite frankly, every other industry and yes, even govenrmental departments, look long and hard at outcomes now. Education seems to be the last bastion where outcomes don't take precedence in evaluating the performance of the labor market.

Just as Billy Beane's theories and practices have changed the labor market in Major League Baseball (as has steroids), looking at value-added statistical analysis has the potential to radically alter not only the labor market, but also the manner in which we train, prepare, recruit and pay teachers. Those who are most successful will get more pay, those who are not will at least have a clue where to look. Instead of gut reactions and a good show, there will be hard data to look at. As Carey writes:

One of the biggest problems with the teacher labor market is that the top teachers--the ones who are one or more standard deviations above the mean in terms of effectiveness--are criminally underpaid, and have no way of demonstrating their real value to the labor market. Their unions, however, are totally aghast at the prospect. Randi Weingarten, head of the United Federation of Teachers (and rumoured to be next head the national AFT) said:

“Any real educator can know within five minutes of walking into a classroom if a teacher is effective."

This is the equivalent of the scouts and general managers in Moneyball who were always on the lookout for the "good body," the "five-tool guy," the player who just looked like a major leaguer. As everyone now knows, they were profoundly mistaken, and people like the Oakland A's Billy Beane were able to exploit the market distortions that resulted.

What we're seeing in New York City today is all the major challenges of 21st century K-12 teacher policy being played out in real time. Value-added methods are still very much in development, subject to limitations of standardized tests, among many things. But in the long run, there will only be more, better information about student performance, along with newer, faster ways of analyzing that information and drawing increasingly accurate conclusions about how well teachers are doing their jobs. At some point the methodological debates will be resolved and the margins of error whittled down the satisfaction of reasonable people.

In reality, the only debate that is worth having and that will be important is the methodology argument. Yes, there will be the obstructionists who argue that we shouldn't do value-added at all, but we are already on that path, and it is a path that is accepted by just about everyone but the teachers' unions, including many really good teachers.

The fact is that just as in any other labor sub-pool, the newest practitioners will need help to get started and to be fair a little bit of leeway. But the strong performers will be evident quickly and their performance will be backed up by hard data. The poor performers, those who are in teh bottom five percent of their cohort, will be quickly identified so that they can either A) improve or B) move into another career more suited to their talents.

Make no mistake though, value-added is coming and the teachers had better start coming to grips with it. The unions would be far better served to be a partner in developing an adequate methodology for computing value-added than being an obstructionist.

Teacher Quality Stats: Experience Matters

It has been a bit longer to post on this subject than I had intended, but personal and professional matters have gotten in the way of what is becoming my hobby on finding new ways to measure teachers' effectiveness. If you are looking for more posts on this subject, please see the Topic Teacher Quality Stats on the left.

In this post I would like to talk about something fairly common sensical--teacher experience. I hope to add some additional ideas into the mix about how we as consumers of education look at matters of teacher experience and how those of us looking to improve the quality of education look at experience as a factor in teacher effectiveness. In this post, I will talk about time in service, i.e., the number of years of experience a teacher has; time in position, that is the number of years a teacher as worked at a given school, and time in class, which I define as the number of years working in a grade level for elementary teachers and/or subject matter for secondary teachers.

Time In Service.
When looking at teacher experience, we generally look at one number--the number of years a teacher has been doing their job. After all, this is a relatively easy number to figure out--simply count the number of years since that teacher started teaching and then subtract out any gaps for say a sabbatical or maternity/paternity leave. Voila, there you have an experience statistic--number of years on the job. Given that everyone improves in their job over time as they become more confident and more experienced, measuring the length of service of a teacher is a reasonable, and respectable, measure.

This is not an irrelevant statistic, just an incomplete one. Studies have shown that most teacher rapidly gain effectiveness in teaching in the late second to the fifth year of experience. Thus, one would reasonably predict a teacher in their fifth year to be more effective than a teacher in their second year of teaching. One could also assume that a teacher with 20 years of experience would be more effective than a teacher with 10 years of experience. However, that has not proven to be the case, as teacher effectiveness tends to level off after the fifth year of experience.

But can we break down teacher experience a little deeper to find some methods, manner and or procedures that can lengthen the learning and effectiveness curve of teachers beyond five years. From a procedural standpoint, one way to increase teacher effectiveness over a longer period of time might be to alter than school operations to involve a more collegial atmosphere. Susan J. Rosenholtz studied the effects of teacher quality and the school's organization, arguing many times that an atmosphere that encourages interaction and learning from each other produces teachers more likely to continue to develop their effectiveness.

So one measure of teacher experience would be to also look at the relative collegiality of the school in which the teacher works. While it may be hard to fully quantify something like collegiality you could develop an arbitrary scale of 1 to 5, where 1 is the least collegial--say barely any talking in the teacher's lounge and 5 is the most collegial, using team teaching, fully developed mentorship programs, regular peer appraisals outside of performance appraisals, lesson plan review and critique, regular departmental and interdepartmental conferences on teaching techniques, etc. In short, how much support and input from fellow teachers of all experience levels do the teachers in a school get. If a collegial atmosphere is fostered by the school administrator, it is possible that the teachers themselves will produce more experience beyond the simple count of years in the classroom.

Time in service is, in reality, a limited measure. After five years, studies seem to point to little statistical significance in effectiveness between a five year teacher and a ten year teacher. If the goal is to extend the time in service for teachers, then we must look to other factors that can be used to judge teacher quality. The degree of collegiality and how it impacts a teachers effectiveness and quality after the first five years hints at something that many people who have studied teachers and teaching have guessed as an impact. External stimuli and challenges often create additional experiences which lead to greater quality in teachers.

Given that teaching is, by and large, a solitary experience for the teacher, finding ways to stimulate the teacher would seem to be a plus for school systems. Increasing the collegiality of a school certainly seems to help and should be implemented as best as possible. But collegiality may only go so far and other measures must be examined.

Time in Position.
If collegiality is a factor in teacher experience and quality, what about a negative factor as well, complacency. When a person has worked in the same place of for a number of years, in the same job, it is easy to get complacent; to take certain things for granted. While the influx of new children each year may lessen the impact or extend the period before complacency takes hold, it is a natural progression. However, unlike collegiality, complacency has something of a built-in indicator, the number of years in a particular school, or time in position.

If the number of years of teaching can be referred to as time in service, time in position would be simply the number of years a teacher has been at the school where she works. So far, in my research (which admittedly is not exhaustive), I have found little on this question of time in position on teacher quality. Hypothetically, the time in position would add to teacher effectiveness in the first few years in much the same way the first five years of time in service works, quality and effectiveness would increase significantly over the first five year in position as the teacher learns and adapts to the school, the administration, her peers, and the neighborhood and its students. As more and more understanding of the people and their neighborhood increases so too does that teacher's effectiveness at that school. But as the five year mark passes, the teacher's effectiveness may begin to slow in its growth.

One implication of this hypothesis would be that some slow teacher mobility may actually be a good thing both for the teacher and the school system. In general, society and parents have looked at the longer time a teacher has at the school, the better or more effective their teaching is. However, if the hypothesis of time in position being similar to time in service holds true, the after five or six years, a teacher may actually stop growing in terms of effectiveness at one school. If time in position approached five, six or seven years, it may behoove both the teacher and the school to transfer to another school for a period of five or six years. Since the population of the school's students turns over every five or six years, there can be some continuity for students and teachers, but with a transfer every five or six year, the teacher is "refreshed" by the challenges of a new environment and colleagues in much the same way as a new teacher is challenged during the first few years in service.

The primary difference, however between a new teacher and a transferred teacher is that a transferred teacher comes to the school with skills and knowledge of an experienced teacher. They may be able to make an immediate impact in the new environment, an impact that a new teacher may not be able to make.

Time in Class.
If time in service and time in position argue for regular, if long term, changes in teacher assignments, time in class would arguably work in the opposite direction. Time in class is the time in which a given teacher has been teaching at a particular grade level for elementary students or a particular subject area for secondary teachers. Again, this is a simply calculation of time.

Time in class is predicated upon a teacher subject area knowledge, more than his or her skill as a teacher. Thus, an elementary school teacher that works with second grade students would be expected to understand the physical, emotional and educational development of seven and eight year olds (those children most likely to be in her class). The more time a teacher spends with such students, the more experience she would gain as a result of the regular and repeated interactions, even thought the class roster might change from year to year. Similarly a secondary teacher who teaches say, American History, would be expected to develop a deeper and broader understanding of the subject, as well as how to teach that subject as she gains more experience in the class.

If time in service and time in position can be graphed in a curvilinear fashion, that is a steep increase in effectiveness in the early years followed by a steady decrease in the rate of growth after year five or six, time in grade would, ideally, have a relatively constant growth pattern, with perhaps some spikes based upon a master's degree or other professional development instruction.

However, in order to achieve a time in class chart of steady growth, teachers would have to have a professional development program based upon subject matter expertise rather than on the current fad in education or pedagogy. Thus, elementary teachers would have to have a professional development program based upon the psychology and physiology of their students at a given age, as well the latest changes in curricula and pedagogy. Secondary teachers would need professional development strongly related to their subject matter. Ideally, subject matter seminars presenting the latest information in their fields would form the basis of the a secondary professional development.

In addition to primary subject matter professional development, schools should look to expand teacher skills and knowledge into other subject matters. According to the NCES many teachers are not teaching in their field. Given the push by NCLB and the general public for highly qualified teachers, why not embrace diversity of assignment and allow teachers to explore other fields in which they may have an interest and aptitude. Would the teacher have a degree in the field? Perhaps, if schools and school systems would equate a second bachelor's degree on par with or even more valuable than a master's degree.

Imagine, a science teacher who teaches chemistry and/or physics must have a solid grounding in mathematics. What if that teacher were to obtain a second bachelor's degree in mathematics? Given that obtaining a second bachelor's degree would take roughly the same time as a master's degree without all that attendant "general education" requirements most colleges have, the economics are not all that different for schools or the teachers. Now that teacher is qualified to teach both science and math. The resulting diversity in subject matter and the ability of the teacher to meet multiple needs for the school keeps that teacher engaged, and effective. Just as adding to a subject matter knowledge is important for secondary teachers, having different knowledge allows for a deeper understanding of both knowledge fields.

Time in class may be different for most teachers than their time in service. Indeed, elementary school teachers may start teaching first graders, but a few years later move to third or fifth graders. Each time, a new time in class clock begins. Similarly, secondary education teachers may have five classes of history and one class of government, each with a different time in class "clock." Time in class is a bit more precise than simply time in service and allows for other comparison and measurement. Indeed as the NCES study points out, many of the measurements of teacher assignments and "out-of-field" teaching are either over or under inclusive. Some provide a bit more precision, such as percentage of course and percentage of students measurements. (see discussion on pages 11-12) When combining the later two, in particular the percentage of students measurements, with a time in class clock, you can get an accurate measure of how much time a student spends with a "out-of-field" teacher and perhaps measure the impact upon their learning.

Conclusion.
Time in a classroom and experience as a teacher are undoubtedly important measurements of teacher quality. As we have learned in multiple manners, the quality of education a child receives is greatly dependent upon the quality of that child's teachers. While it would be great to have nothing but experienced teachers with impeccable academic credentials, such a Stepford school system is impossible. But if schools could look beyond simple measures of seniority and experience found in a time in service measure, much could be learned about teacher effectiveness.

In large part, much of this post is somewhat hypothetical and based upon conjecture. Right now the only measure of experience regularly used is time in service and as we have learned, time in service as a predictor of teacher quality tends to diminish after year five in the classroom. While school conditions such as a collegial atmosphere or other outside inputs may enhance more senior teacher effectiveness, it may not be enough to truly make an impact.

But delving deeper, if we studied effectiveness of teachers based on their time in position as well, we may see a similar pattern. A steep increase in the first five years followed by a reduction in rate of growth after a teacher has spent time in a particular school. But since a teacher with five or six years of time in service may but transfers to a new school and thus resets her time in position clock has the advantage of time in service.

Time in class, however, is different because as a teacher spends more time in a given grade level or subject matter their actual effective would, one would hope, increase or at the very least grow on a geometric progression. While subject matter knowledge may be based in large part upon the prior education of the teacher, the key to successfully using time in class measurement will be found in the effective use of professional development. A poor professional development program for teachers, particularly secondary school teachers, will likely negate any effectiveness benefit gained from a teacher with significant time in class.

Regardless, more research on the impact of time in position and time in class needs to be done. Simply measuring time in service loses its relevance after too short a period and runs counter to the effort to identify and retain truly effective teachers. If we can find out where and how teachers lose the the effectiveness increases of their early careers, school systems can fashion policies, such as transfer policies and professional development programs that can keep the effectiveness curve as steep as possible much to the benefit of our students.

Teacher Quality Stats--The Basic Measurement

This is the fourth in a series of posts about teacher quality statistical measurements. For more of these posts, click on the Teacher Quality Stats category on the left.

Student Achievement--The Basic Measurement.
In order to build a statistical view of a teacher we have to start with a few basics building blocks. Although I have noting more than anecdotal data to support this hypothesis fully, most teacher evaluations are done with little regard to actual student performance. I assume that most techer evaluations consider student performance somewhat, but in the end, the evaluation is based upon other criteria, such as attendence at work, evaluations of lesson plans, once or twice in-class evaluations by the reviewer and other factors such as continuing education credits and perhaps even a subjective feeling about a teacher. Indeed many blogging teachers have complained that evaluation by principals engenders feelings of favoritism or other politics.

Traditionally, many of the measurement of teacher quality would be experience, education, certifications, personal attitudes and other such things that we think of when we think about quality teachers. Many of those qualities were enshrined in the "High Quality Teacher" sections of NCLB. But I believe that those concepts may not have as much relevance as we believe.

In Moneyball terms, Michael Lewis might describe the manner in which teachers are currently evaluated as "old knowledge." In baseball, old knowledge was represented by some baseball scouts and the traditional thinking about the future of baseball players. What Oakland A's General Manager Billy Beane learned was that past peformance from a player is a much better indicator of future performance. So Beane started gathering data, and not just traditional data about players, like their batting average or hits or strikeouts. But other data such as slugging percentage, on-base percentage, number of walks, number of pitches seen by a batter against pitchers. Baseball, of course, lends itself to this kind of statistical knowledge and there are literally dozens if not hundreds of people at every game gathering that basic information. Beane was after the "new knowledge."

But the "new knowledge" in gauging teacher knowledge may not be new, but it is going to a different criteria for judging teacher quality. Admittedly, much of what is discussed in this post is not new idea. But having said that the most basic building block of that knowledge is the advancement of students in the current measurement scheme. Simply put, a value-added model.

The current measurement scheme breaks down the student into grades and proficiency levels. Thus, there are 12 grades from elementary high school and four proficiency levels, below basic, basic, proficient, and advanced. The proficiency levels in my scheme are worth .25 points, depending on their relation, with proficient being equal to the grade level. Thus every student has a grade level and a proficiency level, such a Grade 3 Proficient (3.0) or Grade 5 Below basic (4.5--two levels below grade five). We have tests, such as they are, that measure this grade and proficiency level. (The quality of the test is another variable that will have to be addressed, but there are more than a few policy concerns about the tests).

Thus, at the beginning of an academic year, we must know the grade level and proficiency of each student in a teachers class. That is our starting point for measuing student achievement and therefore teacher quality.

At a minimum performance level, we can and should expect each teacher to increase the achievement level of each student by one grade level per academic year. Thus a fourth grade teacher should raise each student from a third grade level to a fourth grade level. Thus a teacher who achieves this minimal level of success would be judged as having accomplished 1 improvement point in her student. Grade 3 proficient to grade 4 proficient is the increase of 1. In a class of 25, a teacher would garner 25 points if she did just what was expected of her.

For every change in proficiency level, the value added is .25, that is one quarter of a grade level for each of the four profieincy levels. So, for example, a teacher is able to advance a student from grade 3 basic to grade 4 proficient will have added 1.25 improvement points for that student. This simple value added measurement looks solely at the student achievement subjectively and without regard to any external factors, such as the student's race, socioeconomic status or even ESL status. Each student is then a given data point for that teacher.

For example, let us take two 4th grade teachers, each with 25 students. Here are their class breakdowns of before and after an academic year.
Teacher ATeacher B

If we had aggregated the data for each teacher, we would have come to the conclusion that each teacher was a quality teacher, in that overall each did a good job advancing their students. Indeed, looking just as the summary data, one could easily think that Teacher A did a better job that B. Although the average increase was 1.05 for Teacher A versus 1.09 for Teacher B, Teacher A had a higher median increase than B, but also had a wider range of success. Teacher B didn't have any students loose ground, but also didn't experience a dramatic turnaround like A did.

So the question is which techer is better? Each teacher has apparently improved the lot of their students. By the old measures, we would say that each of these teachers is a quality teacher. We might even be right, so aggregating the data might lead us to believe that Teacher B did a better job that teacher A, but is that necessarily the case?

Of course, looking at only one year is not particularly conclusive. Over time, several years of performance may yeild better results as to which of these two teachers provides the most value added for their students.

The value-added method serves two important protections. First, as stated above, all student centered information aisde from proficiency level is completely outside the picture. No information on race, socio-economic status, parentage, residence or ESL status or special ed status enters the equation. Each student-teacher relationship is measured solely on successes of that relationship.

Second, teacher centered information such as where the teacher works, the education and background of the teacher, years of experience, everything is isolated in favor of looking only at what the teacher does with each student in hard terms. In pure statistics, this is the cleanest you can get. In terms of pure accountability, there can be no other base measurement. Either a student is advancing on par with expectations or the student is not.

The value added measurement also allows for large numbers of data collection points. Each student-teacher relationship is measured. The more data points you get, the more reliable a measure of the teacher's quality you get.

This measurement is not new. In fact, as Brett Pawlowski noted to me, the Education Consumers Foundation has used these kinds of measurements in a pilot project on school quality in Tennessee. See his posts here, here, and here.

However, suffice to say at this stage, the value-added data would have to be collected on a student-by-student, teacher-by-teacher basis so that each teacher-student relationship can be measured. That is already the case in Tennessee although student and teacher level data is kept private, school by school aggregate data is available.

Future Postings
Of course, there are aspects on teaching data collection that I have not addressed and I expect a certain amount of criticism along those lines. In future posts, I intend to talk about class size, the issue of resources in the schools (that is the per pupil spending), the issue of subjective evaluation-type subjects, accounting for student turnover, ESL and special ed status, socio-economic status, and a whole range of information. There are other teacher-centered measurements that need to be addressed, such as certifications, licensure, education, years of experience, years in subject, years in school, years in grade, etc.

I suspect that once such data were actually collected on a wide scale, properly evaluated and analyzed, we might find that the old knowledge of what makes a good teacher may be completely debunked.

Teacher Quality Stats--The Why.

In my quest for finding out a new knowledge about teachers and teacher quality, I have spent some time reviewing books and information about statistics gathering, information that is available and trying to figure out a good way to use statistical data to gauge the performance, and therefore quality, of teachers. But first, I think I need to answer a question first. My initial post on the matter (found here) garnered a comment from an anonymous poster:

Better question- what is this data FOR? Considering NCLB wants to use it, if it's created, to classify the bottom 25% of teachers based on such scores as a quota to allow them to be fired?

This is a valid question. My goal is fairly simple, increase the quality of our teaching corps. Nothing more, nothing less.

But to understand what a quality teacher is, I need to know what makes a quality teacher. The traditional knowledge about a quality teacher is based on things like education, experience, certifications, the types of degrees held by a teacher, etc. These are often important, but I don't know, and I mean KNOW for a statistical fact, if these traits which have been traditionally the definition of a quality teacher, actually translate into knowledge and learning in students. In the end, the only thing that matters about a teacher's quality is the success of her students. I cannot think of any rational argument for measuring teacher quality in any other way.

Once we know who is a statistically quality teacher, we can study that teacher, looking for the common traits that lend themselves to success in the classroom for all students. Then, armed with that knowledge, we can begin to build teacher education programs that attempt to replicate those qualities. Now, the identification of poor teachers is going to happen, that is the nature of statistical measurements as opposed to gut feelings about people. I don't doubt the passion and dedication of any teacher, but poor teachers either need to be retrained or dismissed. Their value and impact are likely a negative upon the students and if we as a nation truly care about the education of our children, is it fair to employ a sub-par teacher just because that teacher cares?

Performance is all that matter. I am sorry if that sounds cold and callous, but that is life. I want schools to be centered on the students and only the students. Any adult that inhibits learning for the students needs to be retrained, reassigned or relegated to the sidelines.

The problem we face is that much of the data we collect and know about teachers doesn't actually measure the success of students in the classroom. Everyone throws out all kinds of reasons and anomalies that must be accounted for. For example, the above mentioned commenter noted:

How will you compare 3rd grade at an underperforming school at 30-to-1 with an experienced teacher but 75% turnover, vs. a novice teacher, "average" school, 20-to-1? You can't collect data to evaluate a teacher without at least a minimum statistical percentage of the classroom, what happens when turnover exceeds that?

Are you going to incorporate ELL test scores to equalize that status? What if tests like the CELDT are testing everyday language, not academic language, and what happens when some students are say, a 3 across the board and others are like 2-5-1?

What I am attempting to do is build a model for measuring student success that can account for some of these variables and statistical differences in a classroom. Over time, I am hoping to discuss how to account for these variables. I realize that some of my ideas may not be new, but I want to present some of these ideas in such a way as to perhaps shed a little different light on measure teacher quality.

Moneyball and Money Teachers

A few years ago, I had a boss who was constantly hammering into those of us working for him that numbers matter, statistics matter and we can make more money if we pay attention to metrics and past performance. Because we were in a political business, that of managing software and political action committees, I thought he was nuts. But, trying to keep an open mind, I went along with a couple of his experiments, examining costs to achieve certain results, time spent on activities, and other ideas that led to some hard numbers about our performance. The results were impressive, we were able to cut costs, cut time on common tasks by spending the time and money to build tools and processes to cut time and reduce error. As a result, we were able to take more clients and make more money.

I have been revisiting the idea by reading Moneyball, the fabulous book by Michael Lewis about how the Oakland A's baseball team that was able to win so many games with a payroll that was a fraction of the big teams' salaries. Billy Beane, the general manager of the A's instituted a program where he studied stats and examined the data behind the most successful players and games. With his system of hard data examination, he was able to achieve practical miracles that countered everything "baseball experts," including his own staff, thought would happen.

Moneyball, this post by a new blog, The Common School, and this post by Brett Pawlowski at the DeHaviland Blog have gotten me thinking about what metrics we can use in schools to measure teacher effectiveness.

Right now we have lots of metrics and measurements. The NCLB proficiency ratings, state school assessments, individual test scores, and on and on and on. But do we really have stats on teachers? If we as a society do, we certainly don't talk about them and we certainly use them to determine what works and what does not.

In Moneyball, Billy Beane and his team learned that among the most important factors in winning baseball games was on-base percentage (that is how likely a person is going to get on base every time they come to the plate--this includes walks) and slugging percentage, (that is the number of bases a person is likely to get for each hit). Teams and players that had high percentages were more likely to win games because they would score runs, you can't score runs if you don't have players on base and if players are on base they are not adding to the out count.

So if the goal is the successful education of children, then we need to determine what factors increase the likelihood of a well-educated child. As the Common Room recently noted (and many others) the single most important factor is the quality of the teacher.

First of all, teacher experience and licensure rank near the top of the list while class size reduction and teacher's having master's degrees rank at the bottom. But more important than the ranking is how much smaller of an impact class-size has than some of these other factors. For example, having a teacher who is not a novice (7.2% - 9.1% SD) exerts an influence 3½ to 7 times greater than the impact of reducing class size by 5 students (1.0% -2.5% SD).

So that is one factor, but experience is hardly the only determinant of quality teachers. So what are some of the other factors, i.e. statistics, that can be determined and applied.

Well, some obvious ones would be proficiency on state/federally mandated tests, i.e. are the teachers students below, at or above grade level. Another would be the difference in these numbers relative to their peers with the same task/experience level. Thus you would be comparing 6th grade Math teachers with 4 years of experience with other 6th grade math teachers with 4 years of experience.

Keeping with that theme, how many students changed their proficience level relative to last year. For example, how many (what percentage of students) went from basic to proficient or vice versa? Now these may be a little broad, so if the criteria were narrowed to look at test scores themselve, i.e. the raw scale change in scores.

Of course, of the the most common complaints from teachers is that they have little control over the assingment of students to their classroom. So measurements like those above may be skewed unfairly by poor student assignment luck. But such concerns can be alleviated by pre-and post testing, that is test given at teh start and end of the year on matters of curricula. You test students at teh start of the year to see what they know and the same thing at the end of the year. Good teachers should be able to increase the knowledge base better than their peers. These kinds of stats would allow for each teacher to be judged on what they accomplished with what they had. Plus as an added bonus the teacher can use the results to help tailor curricula.

There have been a number of times where I discussed the need for treating teachers as professional and having teachers act like professionals. One of the most important things for us to do is find a way to empiracally study teachers, i.e. come up with a way of measuring their effeciveness. Teachers who are effective should be examined to see what can be replicated and I refuse to believe that successful teachers are "magical" in some indefinable way that cannot be replicated.

But we don't have any data.

Any other suggestions for measurement?