Public schools in my area have developed a new karma over the last decade: data central.  It was student central for a while, but alas, like many other movements in education, it was replaced by something perceived to be bigger and better. What could be more scientific and certain than basing decisions on data? And which data are the most important?  Student performance data, of course. We measure our schools by how their students perform.

To be more specific: In North Carolina we measure our schools by how their students perform on a state-wide multiple choice test at the end of the year that is designed to measure school-wide mastery of the state’s grade level educational objectives for reading, math, and science. This type of measurement does not take into account other student performance data, such as how well students can talk about what they have learned, what projects or portfolios they have completed, how their writing has improved, or whether they can think critically.

It also means that it is the combination of all the different versions of the test that has meaning for school-wide achievement, but the individual versions of the test vary greatly. For example, Form A of the test might be much thicker than Form B. Form C might have several technical terms, and Form D might have none. Taken collectively they purportedly measure the chosen learning objectives for the grade level, but individually there is no such validity. (One wonders if the Common Core Assessment will be of the same type.)

The test does not take into account students who get very nervous when their most significant measurement is a one-shot deal that measures an extremely narrow range of knowledge and skills. It does not take into account that schools don’t always modify for students who may need it but don’t have an IEP (Individual Education Plan). It doesn’t take into account that the test covers such a small percentage of what most teachers cover in their classroom that it cannot possibly be a reliable measurement of student learning. It measures, for example, certain kinds of reading skills but not English or Language Arts curricula in general.

More important, however, is whether the performance data is achievement data or growth data. Since the NC test was designed to measure school achievement rather than individual achievement, how meaningful are the individual student growth figures that are extrapolated from these test results? How many students could have passed the test on the first day of school that year? Since there is no pretest given on the grade level objectives for the coming year, we in fact don’t measure student growth at all. Yet the only meaningful data for measuring the school or its students is individual student growth data.

Despite all these problems, the latest trend in education is to use the all-important student performance data to measure our teachers as well as our schools and students. Since the tests don’t measure student growth, how can they possibly measure teacher competence?

Additionally, learning is affected by many variables of which teacher quality is only one. We might conclude that a plant died because it didn’t get sunshine when it really died from drought.

Proponents of measuring teacher quality by students’ test data say they will make the test data only a part of a teacher’s evaluation, so drought will not be too much of a discrediting factor. No one has  heard yet what part it will be or how much it will weigh, however, and since the tests are not measuring student growth, it is a moot point anyway. Until we offer testing that measures pre- and post- performance, and until we include broader kinds of pre- and post- data—such as writing, speaking, thinking, acting, creating—we can’t say we are even coming close to measuring student growth.

What does this mean for measuring teacher competence? Quite a bit, since the two are inextricably related. Student growth should be a critical factor in teacher competence. Assuming we had such data, should that be the only kind of data we use? Should there also be qualitative considerations?

Qualitative data require much more work for administrators. It is much easier simply to rest everything on test scores and not do the hard work of evaluating a human being in his or her work setting. Qualitative data require knowledge of how the teacher’s classroom operates on a daily basis, how well the teacher knows and communicates his or her subject, how the teacher thinks about lesson planning, how well the teacher relates to all kinds of students, and much more. It requires extensive observation and communication with the teacher, which many administrators forego due to being pressured to favor data that fits their district’s agenda.

The bottom line is that if you want to measure teachers in their interactions with students and their learning, there is no way to do it objectively; you must tell the teacher’s story based on observation and evidence, starting with data such as how many hours did the administrator spend in the teacher’s classroom? How many minutes did the administrator spend talking to the teacher about his or her practice?

During my last year of teaching my new principal (he did not know me since he was new to the school that year) spent 37 minutes total in my classroom over three visits. He spent 0 minutes talking to me about my teaching (although he did talk at me at length about his interpretation of the data he chose to record during my 30-minute observation and the research on which he based it).

Performance data has its place in education, but it should not be our central operating principal. People and their learning are not subjects that can be represented best by numbers and test scores. Our goal in education is not to train students to take tests, after all; it is to help students develop into good citizens who can deal productively, effectively, and creatively with the complexities of modern society.

We need to break out of the web of data central karma; and since we are capable of freedom and thought, we can.