Summary: The use of “big data” and derivative algorithms to predict which applicants will be successful hires in companies may prove useful in the identification of quality teacher candidates.

The fact that 46% of new teachers leave the profession within five years continues to revisit my personal screen, and is one of those conundrums that never seem to disappear because so compelling in a scratch that scab kind of way. It is a factoid that seems to define school struggles in this country as much as the poor match to the highly technical economy of our schools’ STEM graduate output, or our chronically mediocre test results vis a vis other first world nations.

What seems obvious is that there is somehow a very poor fit between candidates who choose teaching as a career, how they are hired, perhaps how they are supported, and the nature of this difficult work.

So drops in my lap the December Atlantic with Don Peck’s inquiry into some game changers in corporate hiring practices, “They’re Watching You at Work.” Though the title and some of the discussion has eerily Big Brother like connotations, in fact the new use of very large accretions of data (“big data”) to predict who will be a successful employee is one reflection of technological prowess in such contemporary creepiness as Google’s offering up of consumer goods based on one’s spending patterns and demographic profile, or the NSA gathering bites well beyond gigs in order to isolate terrorist patterns (which could be dangerous domestically in the hands of the next J. Edgar).

Elements in the private sector are exploring evolving technologies in order to survive and earn profit at a much more cutting edge pace than the public sector, generally speaking. So it is useful to take a look at corporate experiments in hiring, because not all is well and efficient on that front, either, the reputed superiority of the private sector as trumpeted on the right notwithstanding.

According to a study quoted in Peck’s article conducted by the Corporate Executive Board, “nearly a quarter of all new hires leave their company within a year,” and “hiring managers wish they’d never extended an offer to one out of every five members on their team.” In the corporate world profit margins get squeezed; in schools it is not hard to see the exodus of teachers as a drag on test scores, and on other softer measures of academic “profit” in student progress.

Game simulations have long been used to prepare leaders and workers to function in subsequent real time exercise of their function, and to anticipate scenarios that may later have to be dealt with, whether in a military or in a business context.

After decades of experience with continually more sophisticated video and other computer games, it comes as no surprise that such devices are being used to predict the quality of potential hires. Knack, a Bay Area start up, markets a suite of games designed by neuroscientists, psychologists, and data scientists to tease out a set of sophisticated skills in potential hires. The data generated by the games can reveal such characteristics as “how long you hesitate before taking every action, the sequence of actions you take, and how you solve problems” among other factors. The assemblage can be used to “analyze your creativity, your persistence, your capacity to learn quickly from mistakes, your ability to prioritize, and even your social intelligence and personality.” Ultimately, “an assessment of your potential as a leader or an innovator.” If accurately divined, would not any organization in a hiring position, whether public or private, be interested in such data?

A project underway at Shell shows promise in the use of these games to focus in on employees whose intellectual and creative byways are most apt to produce innovative ideas for the good of the company.

The wag in me can’t help wonder how such live wires might function in the top down environment of a school system, but that is another story, other than to comment that a change is one element of a system usually requires adaptations in other parts of the system. The hierarchy in schools would have to adapt into an environment able to nurture the innovation Shell actively seeks out.

While Shell is looking at sophisticated qualities that typify highly pivotal employment, this new generation of human resource tools have shown more clear market success in less complex hires, specifically for such positions as retail sales and customer service. It is here that “big data,” the accumulation of mammoth registries of information on human behavior and employment (which are so large as to neutralize statistical sample bias) can be used effectively to predict which job applicants possess the qualities that echo those of previous incumbents who flourished in similar positions.

Note that the same growth in computing capacity and data storage that makes the current NSA questions even possible has squired this generation of employment prediction and made it more reliable.

Xerox has hired yet another Bay Area firm, Evolv, in order to upgrade the quality of their hourly hires for positions in retail sales and customer service. By comparison with the measurement of intangibles like creativity, the productivity of hourly workers is relatively susceptible to the sales yardstick, and, as Peck says, there are a lot of these workers to make up a statistically reliable sample, from which algorithms that make hiring predictions can be fashioned. Evolv has developed just such a data base of workers and their characteristics, which in turn can be used to measure an applicant’s fit to the set of qualities known to be deployed by previously successful retail sales people, for example. (Not surprisingly, one key characteristic is persuasiveness, but also decisiveness and spatial orientation.)

The measurement of success is a critical ingredient, as followers of the teacher evaluation debate are only too aware. Sales figures are one thing; the influence of a teacher on a student’s learning is more elusive, even if steps toward using academic test scores within a more broad perspective seem valid to me. Test scores are relatively easy to use because quantifiable; more ineffable aspects of teacher influence and quality less so, and so either get left out of the conversation or seem artifacts of overly romantic notions on the part of teacher partisans. In the current climate we have trouble talking about what we cannot quantify.

For example, in the December 14 Seattle Times, Anacortes, Washington superintendent Mark Wenzel notes the importance of hope in low income students, probably as a part of sense of personal efficacy, and relates his district’s efforts to incubate this elusive variable in its students. Similarly, though Head Start cannot be shown to raise its alumni’s grades or test scores, its graduates do graduate from high school at a higher rate than otherwise, and enter post-secondary education at a higher rate than their non-Head Start peers. Please tell me how we enter such subtle but critical factors into teacher hire algorithms that translate into long term outcome improvement.

Closer to the interpersonal texture of the schoolhouse staff is the work of Sandy Pentland, director of the Human Dynamics Lab at MIT. He has pioneered work with a kind of socio-emotional “fit-bit” that subjects wear and which records information about the quality of interactions with others, such as degree of empathy, the frequency of conversation, both talking and listening, and so forth. According to Donald Peck, Pentland’s purpose was to gauge the interpersonal qualities of productive teams by comparison with less productive teams in an organizational context. A significant portion of a team’s success, he concluded, can be predicted simply by the number of face to face exchanges team members have. Moreover, successful leaders “circulate actively, give their time democratically to others, engage in brief but energetic conversations, and listen as much as they talk.”

Pentland and colleagues are working up apps that will help team members and team leaders evaluate their interpersonal quotient, with an eye to upgrading both their own performance and that of the team.

For schools, which in important ways can be defined as a set of purposeful interactions between people, such tools might prove very useful, between peers, in teacher-student exchanges, and as a measure of how fluidly information moves upward in the chain of command, as well as downward, a kind of anti-bureaucratic measuring stick.

So, with the caveat that the set of skills that makes a good teacher are more complex, and the outcomes more textured and therefore more elusive of measurement than those of a sales person, what progress is there on the teacher hiring front? For there to be algorithms that accurately finger a high quality hire, we would have to have isolated and be able to measure the set of skills that produce clearly identified outcomes, which themselves would have to be quantified in some kind of measurable form. Again, it is clear why test scores have gained such ascendance. At least they can fit into a rough algorithm.

75% of professionals in the game could probably agree on 75% of what makes a good teacher, and the same professionals could probably agree 75% on a desirable set of outcomes. It’s the conversion of these factors into useable instruments that puts the effort to rationalize the process in its infancy.

Doesn’t mean some aren’t trying.

According to Benjamin Herold in an Education Week blog post, “Companies Offer Big Data Tools to Predict Teacher Candidates’ Impact,” there are at least two companies, TeacherMatch and Hanover Research, that have created and marketed teacher hiring instruments that echo some of the mining of “big data” common to the work of Evolv and Knack. The outcome portion of the algorithms they have developed are limited to “student achievement,” but if accurately predictive they may well be a huge boost in the hiring and hopefully the retention of capable teachers.

TeacherMatch in particular is based on a research project conducted by the University of Chicago and the Northwest Evaluation Association as lead entities, and which sorted years of information that link teacher candidate qualities to presumably test score based positive outcomes.

The conclusion of the research on what makes a good teacher candidate? Hardly revolutionary. The attributes of the desirable applicant are, according to Herold, “qualifications, including the selectivity of the college or university a candidate attended; attitude, including indicators related to a candidate’s penchant for persevering through difficult challenges; cognitive ability, as measured on tests of content and knowledge; and teaching skills.”

The devil seems to be in the details. How does one measure a “penchant for persevering,” as well as other items in “attitude?” And how does one evaluate “teaching skills” by other than observation, preferably over time? I admit I am surprised that some measure of interpersonal intelligence is not prominent. All of these characteristics seem to pose difficult translations for the algorithm screening instruments of the sort utilized by Evolv and Knack, but TeacherMatch and its similar competitor Paragon K-12 (from Hanover Research) seem to be worth watching as they dive into the real world of school systems. With their research heritage, one assumes that their algorithms will undergo revision with more refined data.

That said, the innovative work crafted in the private sector by Knack and Evolv, and by Sandy Pentland are worthy of experiments in adaptation to the school community. Pentland’s work clearly delves into capacity for teamwork, which is crucial in a well-functioning school. Might there be a correlation between a successful school and the pattern of communication and teamwork it displays? Schools would do well to patch in Knack’s work in identifying hires with a capacity for innovation, and Evolv seems to have done the most complete work of the group in moving from hire to outcome, albeit with more circumscribed role hires.

Evolv has even moved beyond their pre-hire assessments to ongoing evaluation of aspects of the new hires’ life within the company. According to Don Peck again, this includes evaluation “about not only performance and duration of service but also who trained the employees; who has managed them; whether they were promoted to a supervisory role, and how quickly; how they performed in that role; and why they eventually left.” Simply the list of the types of data Evolv looks at would be instructive for schools whose personnel sophistication is generally on a significantly lower level.

Time isn’t really on our side, but the evolution of hiring instruments will take time – another reminder that the reform of American schools, and their transition into 21^st century organizations, involves a long term conversion of a culture.