CONTENTS I Introduction II What Jurors Need: The Basic Thesis (in Its Strong Form) A Some Caveats III Enlightenment 'Experts': Contrasting Conventional Legal Practices A Inadequate Insights: Legal Criteria and Their Limitations 1 Conceding Methodological Limitations 2 Speculation about the Validity and Reliability of Opinions Derived from Untested Techniques 3 Speculation about Error and Uncertainty 4 Most Legal Criteria and Heuristics 5 Moderating the Expressions and Terminology Available to the Forensic Analyst 6 The Apparent Independence or Impartiality of the Forensic Analyst 7 Allowing the Jury to Decide Based on What Transpires at Trial 8 Deference to the Forensic Analyst B The Limits of Trial Safeguards in Response to Forensic Science and Medicine Evidence C The Apparent Strength of the Case and the Unrecognised Problem of 'Masking' IV Jury Competence: Jury Research and Legal Attitudes A Research on Expert Evidence and the Common Law Jury B Judicial Experience and the Experience of the Law V Irrationality, Agnosticism and Post-enlightenment Judging I Introduction
Historically, in response to proffers of forensic science and medicine evidence, concerns about the competence of juries have been assuaged by expressions of confidence in jury abilities, the celebration of trial safeguards and most recently through initiatives intended to improve the presentation of expert evidence, notably training, the use of primers, refined terminologies, new display technologies, encouraging questions, and revised directions and warnings. (1) Whereas trial and appellate judges have maintained strong public confidence in the effectiveness of the trial and the competence of juries based largely on their (individual or institutional) experience, experimental psychologists have based their confidence on empirical studies of jury performance, particularly jury verdicts, exit surveys and responses to complex evidence. Where empirical studies identify limitations--often via experimental recreations of adjudicative processes--jury researchers (and others) tend to suggest that these might be substantially mitigated or overcome through more careful presentation of evidence and relatively minor modifications to conventional trial procedures. (2)
In response, this article explains why pervasive legal confidence, as well as some of the dominant readings of relevant empirical research, seem to be misplaced. It suggests that the reluctance to require techniques (or processes) relied upon by forensic analysts to be formally assessed means that a considerable volume of forensic science and medicine evidence is not conducive to, and cannot be presented in ways that are susceptible to, rational evaluation. (3) Liberal approaches to admissibility (specifically current interpretations of uniform Evidence Acts ss 55, 56, 79, 135 and 137) (4) result in the routine admission of expert opinions without attending to 'specialised knowledge' or the kinds of information and interpretive tools necessary to unpack, understand and evaluate them. (5) This article explains why the failure to formally test techniques means that in most cases limitations and risks cannot be gauged or credibly explored via traditional trial mechanisms such as cross-examination (and testimonial concessions), the use of rebuttal witnesses, or careful judicial instruction. (6) None of these legal safeguards can replace formal scientific evaluation. They do not provide the kinds of information that would enable a decision-maker to assess a technique or ability and rationally assign a probative value to a derivative opinion. This article explains the importance of validating techniques in regular use. Simultaneously, it reinforces the need for more rigorous admissibility gatekeeping by prosecutors and judges. For, unless they are presented in a manner that is susceptible to rational evaluation, the admission of expert opinions threatens the overarching objectives of factual rectitude and fairness.
II WHAT JURORS NEED: THE BASIC THESIS (IN ITS STRONG FORM)
There are certain (normative) assumptions, which do not seem to be controversial, governing obligations and the way evidence is adduced and presented in accusatorial trials. (7) It is, for example, the responsibility of the state, usually through a prosecutor, to persuade the tribunal of fact (or fact-finder) of the accused's guilt. (8) Requiring the state to prove the case against the accused beyond reasonable doubt encapsulates the burden and standard of proof flowing from the presumption of innocence and the premium placed on not convicting the innocent. Apart from a handful of exceptions, such as where mental incapacity is in issue, the accused has few formal obligations and is not expected to prove his or her innocence. (9) Because of the way the burden of proof is allocated the accused does not need to testify or even adduce evidence.
In terms of the evaluation of evidence, there are few rules or prescriptions and even fewer means of imposing them upon the trier of fact. Australian juries decide in camera and do not provide reasons. Their understanding and evaluation of the evidence, along with their reasoning, is largely inscrutable. (10) Nevertheless, the orthodox and longstanding legal commitment underpinning accusatorial prosecutions is to the fact-finder being able to understand all of the evidence presented during the trial. There is no obligation upon the fact-finder to accept it, but the fact-finder should consider all of the admissible evidence. (11) In accordance with this commitment, fact-finders must be placed in a position that enables them to understand and evaluate admissible evidence. (12)
This article is concerned with expert opinion evidence adduced by the state and the ability of the jury (and simultaneously lawyers and judges) to understand and evaluate it. (13) At trial, the fact-finder should be placed in a position to understand and evaluate any opinion presented by a forensic analyst. (14) They should not have to defer to the authority or standing of the expert or the plausibility of the claim. Inexorably, this article draws attention to the performance of lawyers and judges, as well as the effectiveness of criminal proceedings, procedures, rules and safeguards in relation to forensic science and medicine evidence. In relation to the focus of this article, the overwhelming majority of forensic science and medicine evidence is prepared and adduced by the state, and the state maintains a special interest in the conduct of criminal proceedings. (15)
My thesis builds on these orthodox commitments but introduces a largely unexplored set of issues with radical implications for how we understand the operation of our criminal justice institutions. The basic thesis runs as follows: specific types of information are required to evaluate (ie 'weigh') most types of forensic science and medicine evidence. In the absence of this information many types of forensic science and medicine evidence are not susceptible to rational evaluation. To put this in more legally sensitive terms: the attempt to assign a probative value to incriminating opinion evidence is, in the absence of specific kinds of information, unavoidably speculative or impressionistic. (16)
Difficulties arise because criminal trials do not usually supply the requisite information or an environment conducive to the rational evaluation of expert opinion evidence. (17) That is, the kinds of information ordinarily required to make sense of opinions--including insights into the conceptual or methodological reasons for specific kinds of testing, the meaning of test results, as well as the way opinions are expressed--are not routinely provided and are rarely presented in a balanced way. In consequence, the fact-finder is expected (and effectively required) to evaluate incriminating opinions and assign a probative value without the kinds of information and assistance that would enable them to do so rationally.
The thesis, and concerns about conspicuous informational voids, emerged from my work on comparison and pattern recognition techniques, sometimes described as the identification sciences. (18) These are the staples of the forensic sciences, and include: DNA profiling, drug and chemical assays, comparisons of latent fingerprints, ballistics and tool marks, documents and handwriting, gait, shoe, foot and tyre prints, glass, paint, hair, fibre and bite marks, as well as use of image or voice recordings to assist with identification, some forms of digital evidence, and so forth. Following from the thesis it is my contention that in order to understand and evaluate the probative value of expert opinions in these (and many other) domains, quite specific information is required. (19) What follows is a hierarchical list of factors (or criteria) providing the kinds of information that facilitate rational fact-finding. (20) The first three are the most important and, to varying degrees, the remaining criteria flow from them. (21)
First, the most fundamental...