My last post on the subject of teaching involved references to Michael Lewis' Moneyball, that the same kinds of techniques that helped a much poorer baseball team compete in the Major Leagues, can probably be used in education to help schools, parents, policy makers and teachers better understand what makes a better teacher. If you have better teachers, you have better schools and you have a better educated child and ultimately happier parents and policy makers. So the goal is to find some statistics and data that will help us achive that goal.
Allow me to set the stage a little. The current regime of school data is almost entirely student centered, very little data beyond some very basic information is captured about teachers specifically. As a result, the data that is generated about school quality and teacher quality is at best derivative and doesn't tell a particularly detailed story of teacher quality and effectiveness. I don't mean this as a criticims of the student data currently gathered, but if we as a nation are to dramatically improve our schools, we need to look beyond the temporary, albeit, important denziens of the system and look to the most important factor in student success--the teacher.
The current school data, being student centered and currently gathered under NCLB and other federal programs is pretty easy to game. We have seen the results, where states announce X% of schools being quality schools, but NAEP scores telling a different story. Here is the best way to game a system.
Let us take a typical school in a working class/middle class neighborhood. A given percentage of students are going to be ranked proficient or otherwise pass the mandated exams (the quality of the exams is an altogether different post). Likewise, whether we like it or not, a smaller percentage of students will not pass. Thus, schools will focus efforts on that percentage of students who are borderline. While we may reel in shock about this "triage" it is natural given the stakes invovled.
This gaming relates to teachers in the manner in which a school deploys its teaching resources. Let us assume that 10 percent of teachers that our fictious school are truly outstanding, gifted teachers. It would behoove teh principal to assign our border line students to these teachers, the ones most likely to help their students make the greatest strides. The result is a much higher percentage of students passing the standardized tests, which in turn creates the public impression of a quality school, when in fact the schoold is just ordinary. In other, a small cadre of excellent teachers, if properly employed, raises a school's "quality" ranking, despite the fact that their teaching corps in general is nothing spectacular.
But my interest in this scenario is that cadre of excellent teachers. Assume that each teacher has between 100 and 150 students in their charge during a given year. We now have thousands of interactions and data points that can be measured. In Moneyball, Michael Lewis noted that everything that happens in baseball has happened thousands of times before. Similarly, everything that has happened in a clasroom has happened thousands, if not millions of times before. There is simply not enough variation in behavior for this observation to not be fact. So the challenge if figuring out not only what, but how to measure teacher performance and therefore make deductions about future performance.The question, of course, is how?
Heisenberg's principal states that the mere act of observing a thing changes that thing. Unlike baseball players or the stock market or other matters that can be quantitatively analyzed, teachers operate largely in private. Baseball games have hundreds of people watching for the only purpose of gathering data. Sabermetrics is an advanced statistical science that is remarkably good at analyzing past performance as a predictor (not a 100% accurate predictor, but a pretty good one) of future performance. But because teachers work largely on their own without a great deal of observation by anyone other than their students, we have a massive data collection problem. Students may not be perceptive enough to make the necessary observations. Self-reporting by teachers presents a conflict of interest matter that cannot be resolved in order to accumulate data.
Some information can be gleaned, by derivative, from student test scores, but that is only part of the equation, a results part of the equation. Another part is the day to day interactions with students, the pacing and completeness of curriculum, even the level and amount of homework given by a given teacher, i.e. the process part of the equation. So that the end result of the teacher's quality would be something like this:
Teacher Quality=Intrinsic qualities+Results data+process data+historical performance
Intrinsic qualities would be things like years of experience, years in current school, years in current job (like teaching English, etc.), personal educational degrees, certifications, continuing education credits, other experiences, etc. These would be matters apart from their interactions with students. This data would be easy to gather and while it counts toward teacher quality, it is not the most important. these are also data points we already collect
Results data would have to be derivative of and/or based upon student performance on tests. These data points would have to be weighted somewhat less because there is no or very little control as teacher has once the student begins a test. But this data would have some more bulk to it since each student will have multiple data points, data points that can be plotted and examined over time and should be tied to individual teachers.
Process Data is far and away, at least in my current hypothesis, to be the most important. Data collection would be largest for this cohort of data, but it is the most difficult to collect and right now has very little definition.
Historical performance is related to that teacher's personal perfomance from prior years. Solid professional teachers should experience a dramatic rise in year to year performance in their early careers followed by a gradual leveling as they gain more experience. These same professionals will also seek to keep the skill slope as steep as possible over time.
So right now we have a data definition problem and a data collection problem. Anyone have any suggestions?
No comments:
Post a Comment