Studies, Statistics, Spin and How to Get Around It

Hat Tip: The Carnival of Education

Dr. P has a great post on studies, statistics and spin:

stakeholders will selectively use data that bolsters their theory and suppress other data that doesn’t.

Of course, having spent nearly a decade in politics and public policy, such spin of studies doesn't suprise me at all. But what bothers me, and Dr. P., is that data from public institutions, about public employees, is not made available. This carries profound implications when dealing with teachers, schools and other matters that are public services, paid for with public dollars. As Dr. P. noted in his experience:

One of the repeating themes of my posts is the plea to make as much data public as possible. For example, state boards of education and state colleges and universities have a wealth of data on how prospective teacher candidates perform on their licensure exams. Examination of this data could help explain why some states can set cut-scores 30 points higher (on a 100 point test) than others. But since this data might also be embarrassing as well as revealing, it is not available.

When I was soliciting data from the Educational Testing Service (ETS) for my investigations, it was made very clear that I could not have any disaggregated state level data. This restriction was a contractual obligation ETS had with the individual states that contracted for ETS’s services. Otherwise, ETS could hardly have been more gracious or cooperative.

Now, when dealing with state data like this, one can file a Freedom of Information Ac request and supposedly get such data free of personally identifiable information. So Dr. P. and others should be able to get data on the licensing exam scores of teachers on a statewide level, to see the pass rate, average and median scores, and other information they may find relevant. ETS may be contractually forbidden from disclosing the data, but the state is probably bound by law to give it up, if asked. So we as parents and advocates should be asking more frequently, and publishing more frequently, this kind of information.