I’ve gotten a bit of a crash course over the past few months in the state of forensic science. I am a member of a Wisconsin law reform commission charged with making recommendations to reduce the risk of wrongful convictions. The commission has already done some good work with videotaped interrogations and sequential suspect line-ups. More recently, we’ve turned to problems with forensic science, which have been found (alone or in combination with other factors) in more than 60 percent of DNA exoneration cases. Previously, I knew little about forensic science–I don’t even watch CSI–and I have been dismayed to learn there is much less “science” to it than I assumed.
For the most part, DNA evidence seems to be handled in a scientific fashion (some high-profile cases of fraud and incompetence notwithstanding), but the state of the other “individualization” specialties (e.g., fingerprint, bite mark, bullet, hair, and handwriting analysis that purports to associate crime scene evidence with a unique source) is more troubling. The problems relate both to reliability and validity. On the reliability front, there is considerable subjectivity in some fields in declaring a “match,” and crime labs generally do not employ techniques like blind testing and evidence line-ups that would minimize observer biases. (Imagine this scenario: detective gives fingerprint analyst two samples and says, “Here is a partial fingerprint from the crime scene and here is a full print from the guy we arrested. Can you tell me if they match?”) On the validity front, even assuming 100% reliability in declaring a match, data is generally not available (other than for DNA) to show the significance of a match, i.e., what are the odds that two different people could have produced the match.
For an example of what can go wrong, bullet lead analysis was routinely employed, often unquestioningly, in securing criminal convictions for about 50 years. The assumption was that each batch of bullets manufactured had a unique combination of trace elements, so that a chemical composition match between a bullet found in a body and a bullet found in a suspect’s possession meant the bullets actually came from the same batch (and by inference likely the same box). However, the National Academy of Sciences, in a 2004 study, concluded that the basic assumptions of this whole field of forensic science were unsupported, and in 2005 the FBI stopped using bullet lead comparison.
What can be done to improve forensic science? My commission seriously considered recommending the creation of a state forensic science commission that would have some sort of an oversight role over state crime labs, but ultimately decided against it. It turns out that many states (at least seven) have recently established such commissions, but (after speaking with representatives of three) we could see little record of accomplishment beyond pushing crime labs to get accredited–something not necessary in Wisconsin since all of our crime labs are already accredited. Otherwise, it appears that existing commissions in other states have not been given the resources necessary to deal with more systemic problems, and–given the fierce resistance we saw in our commission from the law enforcement/crime lab establishment to the whole notion of a forensic science commission–I saw little reason to expect a Wisconsin version would be able to drum up any greater support.
The chief alternative to a commission seems to be beefing up the capacity of the adversarial process to test the quality of forensic science in the courtroom. But this requires lawyers and judges who are much more scientifically sophisticated than most are today, for instance, in dealing with admissibility issues and cross-examining expert witnesses. At one point, all eyes turned to the legal academics in the room, as questions were raised about whether we were preparing a more scientifically literate next generation of lawyers. The answer was no. Is this a problem, and are there effective ways of dealing with it? One idea I have had is a forensic science clinic, cotaught by a law professor and a faculty member from a science department. Students in the clinic, who might include both law students and graduate students in the sciences, would get basic instruction in scientific methodology and then work with lawyers who represent indigent defendants in criminal cases that have important forensic science issues. The students could help the lawyers with the identification of relevant expert witnesses, the preparation of expert reports and testimony, and the development of cross-examination strategies to be used for the state’s experts. Do such clinics already exist?
Posted by Michael O’Hear on December 19, 2007 at 06:09 PM
Comments
I would class forensic science within the larger rubric of “procedural” issues and questions which practioners and theorists alike might devote more attention:
Over at his Legal Philosophy Blog, Brian Leiter shares the following from Frederick Schauer:
“I wish there were more attention paid by legal philosophers to questions of legal procedure, in the largest sense of that idea. This attention would include the kinds of issues of legal epistemology that have been the subject of recent work by philosophers such as Susan Haack, Alvin Goldman, and Larry Laudan, but would also include addressing questions about the idea of a burden of proof, about the nature of the adversarialy approach to legal decision-making, about the relationship between procedure and larger questions of fairness, and about much else….Law is, in part, characterized by the procedures it employs, and it is law’s procedures that, in large part, distinguish it from other policy-making and decision-making domains. Consequently, it would be a welcome development if the law’s procedures, and indeed the idea of procedure itself, were subjected to the same kind of sustained philosophical attention that we have seen for various areas of substantive law.”
Relatedly, there was the following announcement from Larry Solum’s Legal Theory Blog:
Conference Announcement: Episteme, Law & Evidence, at Dartmouth CALL FOR PAPERS EPISTEME: A Journal of Social Epistemology http://www.episteme.us.com
2008 CONFERENCE: LAW AND EVIDENCE
Episteme will hold its fifth annual conference at Dartmouth College on June 20-21, 2008. The focus of this year’s meeting is law and evidence.
Confirmed participants include: Susan Haack (University of Miami), Larry Laudan (National Autonomous University of Mexico), Jennifer Mnookin (UCLA), Dale Nance (Case Western Reserve), Michael Saks (Arizona State University), Frederick Schauer (Harvard University), Edward Stein (Cardozo School of Law), and Walter Sinnott-Armstrong (Dartmouth College).
SUBMISSION GUIDELINES: Papers should be no more than 5,000 words, excluding notes and references, and should be prepared for blind review. Electronic submissions should be sent to Walter Sinnott-Armstrong at [email protected] by January 15, 2008. Approximately six papers will be selected from the submissions for presentation at the conference. A smaller subset of these papers will be published in an issue of Episteme, with Frederick Schauer and Walter Sinnott-Armstrong as Guest Editors.
Conference organizers are: Frederick Schauer and Walter Sinnott-Armstrong
Posted by: Patrick S. O’Donnell | Dec 21, 2007 11:49:19 AM
I’ve got a list of suggestions, compiled from various sources, in my forthcoming Iowa Law Review article. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=963461
Posted by: david bernstein | Dec 20, 2007 7:13:44 PM
Is one of the problems a lack of research into the general reliability and error-rates associated with various methods?
For instance, would it be helpful for law professors to start empirical research projects testing forensic scientists’ hit- and miss-rates for fingerprint matches in double-blind studies?
Posted by: dc user | Dec 20, 2007 3:48:48 PM