Testing Wags the Dog and Other Tales of Unintended Consequence

Summary: Testing in schools has taken its impetus from corporate measurement, and has a place. But the steps taken to assess skills have altered classroom chemistry and in the end may have retarded the very progress the tools have been designed to measure.

Imagine a classroom at some future date coming to a school district near you. The science of quality teaching has progressed to a stage described by complex algorithms, which in turn pivot around quantifiable interaction between teacher and student, student to student, and so forth. In the classroom of an experienced teacher we shall call Marsha there sits in an upper front corner a miniature computer, complete with video eye and audio record that tracks over the course of the day the behavioral minutiae of the on-going interactions between Marsha and her students.

The subject matter of the computerized observations harbors no surprises. Broken down into segments of fifteen minutes, the sophisticated software records the number of individual interactions with students, instances of positive feedback, the number of times Marsha checks for her charges’ understanding of the material, the amount of time she spends in direct instruction, the dispersion of teacher questions to every student, the amount of times spent in disruptive outbursts, the length of off task behavior and so on – fill in with your own favorite checklist.

Intelligent software is able to transform the massive amount of data from one day’s instruction into categories that give both the teacher and her supervisor an easily digestible snap shot of what has occurred during that day’s instruction. The supervisor could in fact monitor the computer feed, and check in real time the progress on a particular lesson, while also keeping tabs on other teachers.

Promoters of the technology argue the tool allows supervisors an in depth look at the quality of teaching, and can guide improvement by increasing the rate at which the teacher “checks for understanding,” among a variety of measures.

Lamentably, the human element enters. Teachers, under pressure to improve their data, are reported to have given positive feedback where none was warranted; the algorithm cannot detect the fabrication. Supervisors under fire to improve teaching and learning have been known to delete some of the disruptive outbursts from the data, and thereby depict their school as more peaceful and attentive than it actually is.

As apocryphal as this scenario remains, its cousins already exist in the private sector around less complex tasks. And if we are to cast the net broadly, the shadow of big data already has entered schools themselves.

Esther Kaplan, writing in Harper’s last year (“The Spy Who Fired Me”), investigates sophisticated “telematics,” that monitor worker productivity and help managers cut costs at UPS, McDonald’s, and other corporations. In the case of UPS, delivery folk and their vehicles are electronically supervised via 200 data points, such as stop and start times, back up speeds, seat belt use, length of stops, and the scanning of packages for delivery. Telematics give supervisors real time information as to whether or not a driver is on time or behind in his deliveries. In response, just as in the fictitious classroom example, drivers cut corners, driving with the seat belt permanently buckled behind them or with the rear door left open in order to save time.

Savings in labor and gas costs can be substantial. Coincident with its foray into such ‘big data,” UPS shipments have climbed by 1.4 million packages, while their number of domestic employees has dropped an astounding 22,000 formerly employed.

While arguably consumers pay less for shipping while the UPS bottom line is strengthened, the price is paid in the lives of delivery personnel; fewer employees do more. A rising number of stops are required per shift; it is not uncommon by Kaplan’s findings for workers to finish their deliveries well after their shift is supposed to be over. Injuries are chronic. Though corporations do need to cut costs, the toll on worker well-being goes beyond reasonable limits.  While not an Asian sweat house, one is reminded of the long history of corporate exploitation of labor, absent protections among workers who need to work and cannot afford to challenge work conditions.

Similar tensions face managers as well as workers at McDonald’s, all of whom are pressured to control costs. If a store manager finds his labor costs rising under the gaze of his own boss, he might log an employee as taking a fictitious break, for which there is no pay, even though the employee kept working. Another gambit is to log an employee out at the end of the normal shift, but require her to keep working, particularly if the store is experiencing a rush. To complain is to land on the manager’s bad side, and endanger one’s employment.

Kaplan cites academic studies by Rutgers’ John Aiello that corroborate these anecdotal accounts. Electronic monitoring tends to increase stress, depression, severe fatigue, and even musculoskeletal injuries among workers and to elevate productivity expectations beyond the realistic. Most pertinent to the teaching profession, if a task being monitored is difficult or complex, employees “actually show, under monitoring, impairment of performance, because a bit of their attention is diverted to the fact that someone is watching them.” Moreover, collegiality appears to suffer, which is the lifeblood of an organically functioning school staff.

Of course management should expect performance from employees. Corporations in particular thrive where costs can be curtailed.  Of course schools need to know if their students are being educated. The question remains, does the monitoring regimen improve employees’ performance and enhance the mission of the organization?

Under corporate stewardship, testing has come to define American education, beyond an original intent to measure progress. As in the testimonials from UPS and McDonald’s, metrics can become so intrusive as to radically redefine worker experience, albeit for typically less complex tasks than those that define a teacher’s work, though more intrusive electronic supervision of classrooms may not be so far-fetched, as I have imagined above.

More than a decade into a time when regular testing has become a fixture in the American classroom, academic skills in aggregate continue to be stagnant. Perhaps this kind of measurement has distorted the landscape in ways that frustrates the purpose of education and alters the observation of its impact, a dilemma more common to inquiries in physical science.

Could it be that the original benign deployment of corporate type metrics to measure academic growth has so altered classroom chemistry as to shift the trajectory of progress? Has teaching, a complex science and an art, been so reduced to mechanics as to marginalize some of its more expansive and vital overtones?


This entry was posted in School Reform, Schools and Culture and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s