In the fall, we published Nathan J. Robinson's "Forensic Pseudoscience," which argued that the forensic sciences are so unscientific, and so poorly regulated, that they cannot be trusted to aid the criminal justice system. Two eminent forensic podiatrists, Wesley Vernon and Michael Nirenberg, have responded to Robinson's article with a defense of their field. Below their rebuttal, Robinson offers a final response.
• • •
Forensic science has been subject to criticism on a number of occasions. In 2009, the National Academy of Sciences produced a report criticizing the state of forensic science. Since then, much work has taken place across all forensic science disciplines to address the issues raised. Genuine change has taken place since the NAS publication—changes which continue to gather momentum.
Examples of such changes include the creation of the Organization of Scientific Area Committees by the National Institute of Standards and Technology and the Department of Justice in the United States to “support the development and promulgation of forensic science consensus documentary standards and guidelines, and to ensure that a sufficient scientific basis exists for each discipline”; the appointment of a Forensic Science Regulator in the UK; the further development and monitoring of standards of practice, conduct, and ethics for forensic scientists (e.g., the UK Forensic Science Regulator Codes of Practice and Conduct as an extension to ISO/IEC 17025); amendments to the UK Criminal Practice Directions; work within the International Association for Identification to define the role and scope of the practice of forensic disciplines; and the development of competency testing schemes for forensic practitioners.
While uncertainty exists in all scientific fields, such conditions are typically greater in the applied sciences. This is not peculiar to forensic science.
When Robinson claims that forensic science is unique among the sciences because it has the aim of convicting criminals—and thus “works” when criminals are found guilty—he misrepresents forensic scientists as intrinsically biased in favor of the criminal justice system. Publications related to forensic practice, policy, standards, and ethical codes make it clear that expert witnesses (including forensic scientists and practitioners) should take an objective and unbiased stance in helping the criminal justice system to acquit the innocent and convict the guilty. In this task the forensic scientist considers propositions both for and against the prosecution’s hypotheses in order to reach an unbiased conclusion as to which perspective the evidential analysis supports. The International Association for Identification Code of Ethics and Standards of Professional Conduct requires practitioners to be “unbiased, and objective, approaching all assignments and examinations with due diligence and an open mind.” The UK Skills for Justice National Occupational Standard for Forensic Science 902 (2013) specifically requires forensic scientists to consider “when and how to consider alternative propositions, and how these might be tested,” and furthermore to recognize “the importance of impartiality and how to present balanced opinions and conclusions.” In the United Kingdom, disregarding critical findings that would support or undermine the prosecution’s or defense’s position is seen as a conflict of interest and countermanded by the Forensic Regulator’s Codes of Practice and Conduct (2014). Although a number of these examples are specific to the United Kingdom, they reflect the obligations of forensic scientists worldwide.
Forensic science is applied science undertaken in real-world situations. As such, it is very much about reaching conclusions under conditions of uncertainty in the presence of many variables. While uncertainty exists in all scientific fields (hence, for example, the need to present degrees of confidence when statistical calculations are presented), such conditions of uncertainty are typically greater in the applied sciences. This is not peculiar to forensic science. To deal with such uncertainties, the principles of scientific interpretation apply—in other words, interpretation within context. Part of the expertise of a forensic scientist requires an ability to take into consideration all such uncertainties and weigh the evidence under consideration accordingly. Robinson does not appear to appreciate this essential aspect of forensic practice.
Robinson writes that forensic science is decentralized, and therefore that forensic scientists have little accountability, “with judges taking the place of accreditation boards.” On the contrary, forensic scientists do have accountability: those practicing the discipline are accountable not only to forensic science governing bodies, but a number of forensic disciplines such as forensic podiatry are also accountable to professional boards and associations to which all such professions must be affiliated. In effect, forensic scientists have multiple accountabilities, with multiple different and relevant standards and governances.
It is particularly unfortunate that Robinson has twice referred to the field of forensic podiatry as an example of a problematic discipline—initially as an example of a “dubious” subfield and second by linking it to comments made by Daniel Medwed about “rogue scientists” flourishing in conditions of minimal oversight. Pressing his points, Robinson continues, “The localized, disparate, and unmonitored nature of so much forensic practice makes for massive nationwide inconsistency.” We in principle agree that the absence of mandatory competency testing can create doubt about some individual forensic scientists’ competencies; however, this does not by extension call the entire discipline into question. While it is not the court’s job to “police” the expertise of forensic scientists, it is nevertheless the case that the establishment of any expert’s credibility to the court is routine. Experience suggests that where doubts exist as to expertise, this will inevitably come out during cross-examination.
When Robinson writes that the law “is a poor vehicle for the interpreting of scientific results,” he is impeded by his own argument. With rare exceptions, lawyers are not trained forensic scientists, and as such may be in a poor position to judge forensic science and practice. If his conclusion is correct, then Robinson is also in a poor position to judge the performance of forensic practitioners. One could say that Robinson is in the very position regarding lack of accountability of which he himself complains.
• • •
Nathan J. Robinson Responds
One need only look at this reply from Wesley Vernon and Michael Nirenberg to see why we ought to doubt the reliability of the forensic sciences. In my article, I gave examples, cited experts, quoted reports, and reasoned carefully. In response, Vernon and Nirenberg avoid offering any actual evidence that their results can be trusted, or that a single fact I cited was incorrect. Instead, we get proof by assertion; they simply insist that their profession has changed, that accreditation standards are sound, and that their discipline is fundamentally rigorous.
This is a perfect example of the problem I was attempting to illustrate. As I wrote, the problem with forensic science is not that it is wrong, but that it is hard to know when it is right. That is because forensic scientists like Vernon and Nirenberg do not believe they have to offer any factual substantiation for what they say. So, for example, they insist that “genuine change” has occurred since the National Academy of Sciences concluded in 2009 that many forensic sciences were essentially little more than fancily credentialed hocus-pocus. But they mention no actual changes, nor do they give an argument for why the particular changes can safely be assumed to have solved the problem, even though this is the entire question under discussion. Of course, they could well be correct, but since they rely on nothing more than their own vehement insistences, it is impossible to know. My point could not have been better proven.
The question is not, “Are you supposed to be biased?” but rather, “Are you in fact biased?”
This allergy to evidence is present in every claim Vernon and Nirenberg make. They rebut suggestions that their profession is biased by stating that their profession’s ethical codes prohibit bias. This is like an employer replying to a racial discrimination claim by pointing out that the company handbook prohibits racial discrimination. The question is not, “Are you supposed to be biased?” but rather, “Are you in fact biased?”
Their idea of what constitutes accountability is also telling. They wave away suggestions that courts are poor evaluators of science by relaying that, in their experience, challenges to expert witnesses’ credibility “will inevitably come out during cross-examination.” But the point is not whether experts’ findings will be challenged, it is whether judges and juries will then be able to figure out who is right, given their mostly negligible knowledge of the underlying scientific fields. A defense attorney can loudly insist that a supposed expert is a quack, but how are twelve teachers, baristas, and hairdressers supposed to sort through peer-reviewed journals or assess the licensing rigor of the hair and fiber examiners’ professional association?
Vernon and Nirenberg tell me that I do not “appear to appreciate” that uncertainty is a key, ineradicable part of what they do. It is a strange rejoinder, considering that my entire purpose was to show that uncertainty is a key, ineradicable part of what they do. But whereas I am concerned by the implications this has for the judicial process, they appear uninterested in all questions of how one should assess fallible evidence from potentially unreliable parties.
Their final point is an amusing bit of sophistry: if I think that law is so bad at evaluating scientific evidence, they snort, then perhaps I as a lawyer should stop evaluating scientific evidence. This cheeky taunt rests on a confusion between evaluating evidence itself and evaluating the institutional processes by which evidence is examined. I am not proposing myself qualified to take up amateur toxicology analysis. Rather, I am suggesting that I and my fellow laymen need good reasons to think that the toxicology analysis of others can be depended upon, if we are expected to use analysts’ findings to determine individuals’ fates. It is deeply alarming that Vernon and Nirenberg have a difficult time grasping this idea.
Vernon and Nirenberg’s reply should therefore serve as Exhibit A in the indictment of forensic science’s trustworthiness. Nothing more succinctly illustrates the faulty, credulous thinking that should caution us against allowing people’s lives to depend on the soundness of such men’s expertise.
Return to Robinson's "Forensic Pseudoscience"