About a decade ago, I was approached by a group of law school admissions directors. They asked whether I, as a cognitive psychologist interested in new forms of assessment, could help to redesign the law school aptitude test. I asked one question, which effectively terminated the mission of these good people. The question: “Do you want to change law school?”
Buried in this question are two assumptions that dominate assessment in this country at this time. We might call the first the “end state” assumption. It is assumed that the capacities purportedly measured by an instrument like the LSAT are critical for successful execution of an ultimate goal or “end state.” I am not an expert on law school, but I suspect that the LSAT is much better as a predictor of who will make law review or become a good professor of law than as a predictor of who will be a good litigator or good arbitrator, or even who will make the most money. Law school, however, remains the domain of law professors, so we could reasonably expect that they would favor instruments that allow them to replicate themselves.
The second assumption could be termed the “trait” assumption. Drawing on a long (but perhaps prejudicial) tradition in Anglo-American thought, such instruments assume that individuals possess (or fail to possess) certain “traits,” and that not much can be done about this state of affairs. Some individuals have the “it” required to be an effective law student and lawyer; the earlier and more unequivocally we can ascertain who has “it,” the better for all concerned.
I join Susan Sturm and Lani Guinier in questioning both of these assumptions. I suspect that there are many legal “end states” and that present instruments are heavily skewed toward just a few of the more readily measurable ones. In addition, I doubt that “lawyerness” (whatever it is or isn’t) is something that individuals possess (or fail to possess) to a certain degree. I think it far more likely that many individuals have the potential to be lawyers of quality and–key point–that much more powerful ways could be developed to ascertain that potential.
Until now, I have limited my remarks to the practice of law. I must stress, therefore, that these same assumptions about assessment dominate every sphere of educational and professional life. From admission to independent school to admission to the bar, from the police force to the teaching force, we assume too blithely that we know the relation between the “entry point” and the “end state.” Moreover, we assume too blithely the existence of traits or aptitudes that individuals either possess or lack. Indeed, it was less than a decade ago that the College Board reluctantly removed the word “aptitude” from the SAT–while keeping largely unchanged the assumptions built into its instruments.
Without explicitly noting it, Sturm and Guinier have built their case on a concept developed 75 years ago by a brilliant Russian psychologist named Lev Semonovich Vygotsky. Vygotsky speculated about two individuals–let’s call them Boris and Mikhail–who received the same score on some task–say, a set of items on an intelligence test. The average observer would assume that these individuals had the same ability, trait, aptitude, or skill. Vygotsky had the clever idea of providing the same amount of training or tutelage to both of these individuals–to use a current word of art, he provided them both with considerable “scaffolding.” If both individuals’ scores remained the same at the end of this period, or if both improved by the same amount, we could reasonably infer that they had similar ability.
Suppose, however, that Boris’s score went up just a tad while Mikhail’s score improved by 25 percent. Moreover, a few weeks later, Boris’s score remained steady, while Mikhail’s had continued to rise. We should properly infer that there is an important difference between the two lads–that Mikhail has a greater potential to learn and would presumably benefit more from an enriched educational environment. Vygotsky labeled his concept “the zone of proximal development”: application of this concept allows one to ascertain amount of improvement–particularly lasting improvement–under conditions of moderate help.
I read Sturm and Guinier as witting or unwitting Vygotskians. They are arguing that we should identify skills and capacities that are important and determine the extent to which individuals exposed to an appropriately rich environment can pick up or amplify these skills and capacities. To put it in vernacular terms: they seek to import the mindset of “on-the-job training” to the terrain of “assessing potential.” To put it in terms of the assumptions I introduced above: Sturm and Guinier would like us to discard the notion that there is a pure trait that is presumed to be a phenotype of future success in a domain. Instead, they urge us to place a candidate in an environment in which experts are exhibiting requisite skills and capacities, and then determine the extent to which, and the speed with which, the candidate can acquire these skills and capacities. Instead of a presumed distant end state, we provide actual “living” instances of the end state. And instead of a mysterious inborn “trait,” we look for individuals who can begin to exhibit the desired skills and behaviors as the result of limited but targeted immersion in the appropriate environment.
The critiques of standardized tests offered by Sturm and Guinier are familiar ones. Indeed, the journalist Walter Lippmann offered most of them in the 1920s. Various attempts have been made to overthrow this orthodoxy, from logical arguments to legal challenges to blatant ridicule–some thirty years ago, psychologists developed the “Chitlin test” of intellectual competence, which (as the name implies) presupposed cultural literacy in the African American world.
Observers sympathetic to the critiques must ask why standardized tests remain so robust, particularly in this country. Sturm and Guinier identify some of the reasons. To them, I would add two more: the lack of an alternative way of thinking about assessment and the paucity of alternative instruments or methods.
In their essay, Sturm and Guinier contribute chiefly to our thinking about new ways of assessing. I broadly endorse their stress on performance of relevant tasks rather than on measurement of hypothesized abilities or traits, and I think that the Vygotskian analysis can provide intellectual support for this position. In the absence of demonstrated alternatives, though, standardized tests (as we have grown to love or hate them) will continue to hold sway.
Let me offer two modest proposals, which might allow one to explore the applicability of their ideas–for the sake of argument, in the case of prospective law students. The first involves a summer or term-long internship. Interested candidates could agree to work in a law office as volunteers or for a modest stipend. They would be involved immediately in tasks for which they do not require special training, but which could test their capacity to “pick up” needed understandings. Those who catch on most successfully to relevant legal roles (including the social, creative, and flexible skills emphasized by Sturm and Guinier) would earn special credits toward admission to law school. Of course, we would have to ascertain whether students thus admitted could succeed in (standard or revised) law school and go on to practice law successfully thereafter.
The continuing improvement of computer software suggests a second approach. One could create simulations of skills and proficiencies important for success in the study and actual practice of law. Some law schools now include a special course called “lawyering,” which seeks to cultivate a wide gamut of skills relevant to successful legal practice. The simulations could be drawn from the lawyering curriculum. Vygotsky-style, they could provide varying degrees of scaffolding. Those students who perform well in such simulations, and who continue to improve over successive trials, would earn similar credits toward admission. Again, it would be necessary to determine whether success in such simulated environments predicts success in law school and in the law office.
Until now, we have skewed education everywhere toward “uniform” approaches–one way of teaching, one way of assessment, onestandard of success. Reflective practitioners know, however, that this is at best an oversimplification and perhaps an ultimate disservice to candidates, to the profession, to the larger society. Sturm and Guinier have put forth an appealing alternative vision. Only a powerful sense of purpose, artful technology, and support to carry out needed experiments can reveal whether this alternative vision is viable.