Forensic examiners are asked to decide whether a mark matches a finger, whether a DNA mixture includes a suspect, or how a person died, and in many disciplines the instrument that makes that decision is a trained human being. Two decades of research have asked whether such decisions can be moved by information that has no bearing on the evidence itself: a detective's theory, a confession, a colleague's earlier conclusion, the reference sample lying beside the trace. This narrative review sets out what that research has established, what it has not, and what laboratories have done in response. The experimental record shows that contextual information can change expert judgements in fingerprint comparison, DNA mixture interpretation, forensic anthropology and other fields, mainly when the evidence is ambiguous. It says much less about how often this happens in casework or how often it produces a wrong result. The safeguards now recommended, among them context management, linear sequential unmasking and blind verification, rest on sound reasoning and on reports of successful implementation, but few have been evaluated for their effect on accuracy. The review closes by identifying the gaps that most limit current practice.
In March 2004, after the train bombings in Madrid, examiners at the Federal Bureau of Investigation identified a latent fingerprint found on a bag of detonators as that of an attorney in Oregon, Brandon Mayfield. The identification was verified within the laboratory and was wrong. The Spanish National Police later attributed the print to another man. The review that followed found no misconduct. It described examiners who, once a candidate had been put before them, reasoned backwards from his print to the mark and found in a smudged impression the features they expected to see (Office of the Inspector General, 2006).
The case gave a face to a concern that had been raised a few years earlier by Risinger et al. (2002), who pointed out that forensic science, almost alone among fields that rely on human observation, took no systematic precautions against expectation and suggestion. Psychologists had documented for decades the tendency to seek and interpret evidence in ways that fit what one already believes (Nickerson, 1998). There was little reason to suppose that a fingerprint examiner or a DNA analyst was exempt, and the report of the National Research Council (2009) on the state of forensic science in the United States said so plainly and called for research on observer bias and human error.
A good deal of that research now exists, along with a set of recommended countermeasures and a continuing argument about how far either can be trusted. This article reviews the three in turn: the evidence that cognitive bias affects forensic decisions, the safeguards proposed and adopted in response, and the questions that remain open.
2. Scope and Approach
This is a narrative review of the published literature. It draws on experimental studies of forensic examiners, on earlier reviews, on official inquiries and guidance documents, and on accounts by laboratories of the procedures they have introduced. It does not report a systematic search; readers who want a structured account of the primary studies are referred to the systematic review by Cooper and Meterko (2019). The emphasis is on disciplines in which the examiner's judgement is the method, although studies from forensic pathology and forensic psychology are included because they test whether the problem is confined to the comparison bench.
3. What Is Meant by Cognitive Bias
The term is used loosely, and some of the heat in the debate comes from that. In this literature it does not mean prejudice, dishonesty or carelessness. It refers to the ordinary workings of perception and judgement, by which prior beliefs and expectations shape what a person notices and how ambiguous information is resolved. Kassin et al. (2013) coined the phrase forensic confirmation bias for the class of effects in which an examiner's beliefs, motives or situational context influence the collection, perception and interpretation of evidence in a criminal case. Because the process operates without awareness, an examiner cannot detect it by introspection, and sincerity is no protection.
Dror (2020) has sorted the sources of such influence into eight groups arranged in three tiers. The first tier belongs to the case in hand: the trace itself, the reference material and the surrounding contextual information. The second belongs to the examiner's environment and history: base rate expectations, organisational factors, and education and training. The third is common to everyone: personal factors and the architecture of human cognition. The scheme is useful mainly because it shows that removing a detective's covering note deals with only one source among several. A reference print that is examined alongside the mark can direct attention to features that would otherwise not have been marked, and a laboratory that almost never sees an exclusion from a given kind of request builds an expectation of its own.
Two further distinctions help. The first is between information that is relevant to the examiner's task and information that is merely relevant to the case. A confession bears on guilt, but it tells a fingerprint examiner nothing about ridge detail, and if it moves the opinion the court receives the same fact twice under two labels. Bias may also travel: it can cascade from one stage of an investigation to the next, and it can snowball as examiners in different disciplines learn of one another's findings (Dror et al., 2017). The second distinction is between bias, which pushes decisions in a direction, and plain inconsistency. Examiners retested after an interval of months do not always repeat their own decisions on difficult comparisons even when nothing about the context has changed (Ulery et al., 2012), and a study that fails to separate the two will overstate one of them.
4. The Experimental Evidence
The primary studies share a simple logic. The same physical evidence is presented under different contextual conditions, or to the same examiners on two occasions, and the decisions are compared. Table 1 summarises a selection of the better known studies, chosen to show the range of disciplines and of manipulations.
Selected published studies of contextual influence on forensic judgements
Discipline
What was varied
Reported direction of effect
Source
Latent fingerprints
Case story implying that a pair earlier called a match was not one
Most examiners departed from their own earlier identification
Dror et al., 2006
Latent fingerprints
Presence or absence of the reference print during analysis
Features marked in the latent mark differed between conditions
Dror et al., 2011
Fingerprint verification
Suggestion about the conclusion already reached
Examiners were influenced; experts moved towards inconclusive
Langenburg et al., 2009
DNA mixtures
Case information withheld from analysts given a casework mixture
Most conclusions differed from that of the original laboratory
Dror and Hampikian, 2011
Forensic anthropology
Prior information about the remains
Assessments of sex, ancestry and age tended to follow the context
Nakhaeizadeh et al., 2014
Forensic pathology
Non-medical details in a written case of a child death
Manner-of-death opinions differed; the findings were contested
Dror et al., 2021
Forensic psychology
Side the expert believed had retained them
Risk instrument scores leaned towards the retaining side
Murrie et al., 2013
4.1. Fingerprints and other pattern evidence
Fingerprint comparison has been studied most, partly because of the Mayfield case and partly because examiners were willing to take part. In the best known experiment, Dror et al. (2006) arranged for experienced examiners to receive, in the course of their normal work, a pair of prints that each had identified as a match some years before, now accompanied by the suggestion that it was the erroneous pair from the Madrid investigation. Most of them no longer reported an identification. The group was very small, and the manipulation was about as strong as one could devise, but the study showed that a decision widely regarded as objective could be reversed by a story.
Later work was more measured in both design and result. Langenburg et al. (2009) examined the verification stage of the analysis, comparison, evaluation and verification procedure and found that suggestion did influence examiners, but that experienced examiners tended to retreat to an inconclusive opinion and did not follow the suggestion into error. Dror et al. (2011) showed something subtler: the features an examiner marks in a latent mark depend on whether the reference print is in view at the time. That finding matters for procedure, because it locates an effect at the stage of analysis, before any conclusion is formed, and it comes from the reference material and not from a detective.
4.2. DNA mixtures
DNA profiling is often treated as the standard against which other disciplines are found wanting, and for single-source samples of good quality that reputation is deserved. Mixtures of several contributors at low template are another matter, since the analyst must decide which peaks are alleles and which are artefacts. Thompson (2009) described how knowledge of the suspect's profile can guide those choices, and likened the result to painting the target around the arrow. Dror and Hampikian (2011) gave the electropherogram from an adjudicated case to analysts in another laboratory without the case information. Most did not reach the conclusion that the original analysts, who had known the context, had reported. The study was small and compared laboratories as well as conditions, but it removed any comfort that instrumental output is immune.
4.3. Beyond the comparison bench
Effects have been reported well outside pattern matching. Nakhaeizadeh et al. (2014) found that assessments of sex, ancestry and age at death from skeletal remains shifted with the background information supplied. In forensic psychology, Murrie et al. (2013) showed that experts who believed they had been retained by the prosecution scored the same offender files as higher risk than those who believed they worked for the defence, an allegiance effect produced by nothing more than the name of the client. In forensic pathology, Dror et al. (2021) reported that opinions on manner of death in a written case varied with details that were not medical. That paper drew sharp criticism in the journal's correspondence pages, much of it on the ground that a pathologist determining manner of death is expected to weigh circumstances and that the line between relevant and irrelevant context had been drawn in the wrong place. The dispute is instructive, because it shows that the concept of task relevance, which is fairly clear for a fingerprint, becomes contestable in disciplines whose conclusions are partly investigative.
4.4. How far the evidence reaches
The studies support a limited claim firmly and a broader one hardly at all. The limited claim is that contextual information is capable of changing expert forensic judgements, across several disciplines, and that the effect is concentrated in difficult or ambiguous material. Cooper and Meterko (2019), reviewing the primary literature systematically, found the studies to be few, uneven in quality and heavily weighted towards latent prints, though consistent in direction. The broader claim would be that bias is a frequent cause of error in casework. Nothing in the experimental record establishes that. Many studies used small groups, students in place of examiners, or materials selected for difficulty, and almost none measured the effect against ground truth in routine cases. A shift of opinion is not in itself an error, since context may move an examiner towards the correct answer as easily as away from it. The objection to it is that the examiner's opinion then ceases to be independent evidence.
5. Safeguards
The responses that have been proposed follow from the diagnosis. If information is the hazard, the remedy is to control what the examiner knows and when. Figure 1 sets the main safeguards against the stages of an examination at which each applies.
Stages of a forensic comparison and the safeguard against contextual influence that applies at each
5.1. Managing the flow of information
The earliest formal proposal came from DNA analysis. Under sequential unmasking, the analyst interprets and documents the evidentiary profile before being given the reference profiles (Krane et al., 2008). Dror et al. (2015) generalised the idea as linear sequential unmasking for comparative disciplines at large. The examiner first analyses the trace alone, records the features found and the degree of confidence in them, and only then sees the reference material; changes made afterwards are permitted within limits and must be recorded. The reasoning should run from the evidence to the suspect and never in the other direction. A later extension applies the same ordering principle to decisions that involve no comparison, and asks that items of information be sequenced by their objectivity, their relevance and their power to bias (Dror and Kukucka, 2021).
Unmasking deals with the order of work. Context management deals with what reaches the examiner at all. In the model described by Found and Ganas (2013) for a handwriting unit, a member of staff who takes no part in the comparison receives the submission, strips out what the examiner does not need and passes on the remainder. The authors reported that the arrangement was workable in routine casework. A document from the National Commission on Forensic Science (2015) supplied the governing test: information is task-relevant if it is needed to draw conclusions about the propositions in question from the physical evidence, by an accepted method, and examiners should rely on nothing else.
5.2. Blind verification, training and guidance
Verification by a second examiner is a long-standing quality control, but a verifier who knows the first conclusion is checking it and not repeating it. The Mayfield inquiry made the point with some force (Office of the Inspector General, 2006). Blind verification, in which the second examiner does not know what the first decided, restores independence at the cost of time, and many laboratories reserve it for selected cases. Which cases are selected matters. If only identifications are verified, the verifier can infer the first result from the fact of being asked.
Training is the safeguard most often adopted and the one with the weakest rationale when used alone. People readily see bias in others and not in themselves (Pronin et al., 2002), and a survey of forensic examiners in many countries found the same pattern: wide acceptance that bias is a problem for forensic science, less acceptance that it affects one's own discipline, and less still that it affects oneself, together with a common belief that it can be overcome by an effort of will (Kukucka et al., 2017). Dror (2020) lists that belief among the fallacies that keep procedural reform at bay, alongside the ideas that bias is a matter of ethics, of a few bad practitioners, or of inexperience, and that technology removes it. Awareness is necessary to win acceptance for procedures, but it does not do their work. Regulators have taken the same view: the guidance issued by the Forensic Science Regulator (2015) in England and Wales asks practitioners to assess the risk of bias for each type of examination and to build mitigation into the method.
6. Remaining Gaps
The first gap is evaluative. Sequential unmasking, context management and blind verification are justified by argument and by analogy with blinding in other sciences, and there are published accounts showing that they can be run in working laboratories. There is very little evidence on whether they change the accuracy of conclusions, what they cost in time and turnaround, or which of them gives the greatest return. A laboratory manager deciding where to spend limited resources has principles to go on and little else.
The second concerns the boundary of task relevance. For a comparison of ridge detail the boundary is easy to state. For a pathologist, a fire investigator or a scene examiner, context is often part of the legitimate basis for an opinion, and the pathology dispute discussed above shows that practitioners and researchers do not yet agree where the line falls. Progress here probably requires discipline by discipline specification of which information is needed for which decision, worked out with the practitioners concerned and not imposed on them.
Several other matters are thinly covered. Most experiments manipulate case information, so the second-tier sources in Dror's scheme, such as base rate expectations and organisational pressure, remain largely unmeasured. Automated tools bring biases of their own, since a ranked candidate list from a database search or the output of probabilistic genotyping software tells the examiner what to expect before the examination begins. Little is known about how fact-finders use forensic opinions that were formed with or without safeguards, or about the wider cascade in which a forensic result shapes later witness statements and investigative choices (Kassin et al., 2013). Finally, bias and inconsistency have seldom been measured in the same design, though they call for different remedies.
7. Conclusion
The research of the past twenty years has settled one question and opened several. Forensic examiners, like other experts, can be influenced by information that ought not to bear on their judgement, and the influence is greatest where the evidence is least clear. That much is established well enough to justify procedural precautions, which are in any case cheap compared with the cost of a wrongful identification. How often such influence leads to error in casework is not known, and the safeguards now recommended have been reasoned out more thoroughly than they have been tested. The useful work ahead lies less in further demonstrations that bias exists than in field evaluation of the countermeasures and in careful definition, for each discipline, of what an examiner needs to know.
Conflict of interest
To be declared by the author(s) before publication.
Funding
To be declared by the author(s) before publication.
ReferencesCooper, G.S. and Meterko, V. (2019). Cognitive bias research in forensic science: A systematic review. Forensic Science International, 297, 35-46https://doi.org/10.1016/j.forsciint.2019.01.016Dror, I.E. (2020). Cognitive and human factors in expert decision making: Six fallacies and the eight sources of bias. Analytical Chemistry, 92(12), 7998-8004https://doi.org/10.1021/acs.analchem.0c00704Dror, I.E. and Hampikian, G. (2011). Subjectivity and bias in forensic DNA mixture interpretation. Science & Justice, 51(4), 204-208https://doi.org/10.1016/j.scijus.2011.08.004Dror, I.E. and Kukucka, J. (2021). Linear Sequential Unmasking-Expanded (LSU-E): A general approach for improving decision making as well as minimizing noise and bias. Forensic Science International: Synergy, 3, 100161https://doi.org/10.1016/j.fsisyn.2021.100161Dror, I.E., Champod, C., Langenburg, G., Charlton, D., Hunt, H. and Rosenthal, R. (2011). Cognitive issues in fingerprint analysis: Inter- and intra-expert consistency and the effect of a 'target' comparison. Forensic Science International, 208(1-3), 10-17https://doi.org/10.1016/j.forsciint.2010.10.013Dror, I.E., Charlton, D. and Péron, A.E. (2006). Contextual information renders experts vulnerable to making erroneous identifications. Forensic Science International, 156(1), 74-78https://doi.org/10.1016/j.forsciint.2005.10.017Dror, I.E., Melinek, J., Arden, J.L., Kukucka, J., Hawkins, S., Carter, J. and Atherton, D.S. (2021). Cognitive bias in forensic pathology decisions. Journal of Forensic Sciences, 66(5), 1751-1757https://doi.org/10.1111/1556-4029.14697Dror, I.E., Morgan, R.M., Rando, C. and Nakhaeizadeh, S. (2017). The bias snowball and the bias cascade effects: Two distinct biases that may impact forensic decision making. Journal of Forensic Sciences, 62(3), 832-833https://doi.org/10.1111/1556-4029.13496Dror, I.E., Thompson, W.C., Meissner, C.A., Kornfield, I., Krane, D., Saks, M. and Risinger, M. (2015). Context management toolbox: A linear sequential unmasking (LSU) approach for minimizing cognitive bias in forensic decision making. Journal of Forensic Sciences, 60(4), 1111-1112https://doi.org/10.1111/1556-4029.12805Forensic Science Regulator (2015). Cognitive Bias Effects Relevant to Forensic Science Examinations (FSR-G-217). Forensic Science Regulator, BirminghamFound, B. and Ganas, J. (2013). The management of domain irrelevant context information in forensic handwriting examination casework. Science & Justice, 53(2), 154-158https://doi.org/10.1016/j.scijus.2012.10.004Kassin, S.M., Dror, I.E. and Kukucka, J. (2013). The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2(1), 42-52https://doi.org/10.1016/j.jarmac.2013.01.001Krane, D.E., Ford, S., Gilder, J.R., Inman, K., Jamieson, A., Koppl, R., Kornfield, I.L., Risinger, D.M., Rudin, N., Taylor, M.S. and Thompson, W.C. (2008). Sequential unmasking: A means of minimizing observer effects in forensic DNA interpretation. Journal of Forensic Sciences, 53(4), 1006-1007https://doi.org/10.1111/j.1556-4029.2008.00787.xKukucka, J., Kassin, S.M., Zapf, P.A. and Dror, I.E. (2017). Cognitive bias and blindness: A global survey of forensic science examiners. Journal of Applied Research in Memory and Cognition, 6(4), 452-459https://doi.org/10.1016/j.jarmac.2017.09.001Langenburg, G., Champod, C. and Wertheim, P. (2009). Testing for potential contextual bias effects during the verification stage of the ACE-V methodology when conducting fingerprint comparisons. Journal of Forensic Sciences, 54(3), 571-582https://doi.org/10.1111/j.1556-4029.2009.01025.xMurrie, D.C., Boccaccini, M.T., Guarnera, L.A. and Rufino, K.A. (2013). Are forensic experts biased by the side that retained them?. Psychological Science, 24(10), 1889-1897https://doi.org/10.1177/0956797613481812Nakhaeizadeh, S., Dror, I.E. and Morgan, R.M. (2014). Cognitive bias in forensic anthropology: Visual assessment of skeletal remains is susceptible to confirmation bias. Science & Justice, 54(3), 208-214https://doi.org/10.1016/j.scijus.2013.11.003National Commission on Forensic Science (2015). Ensuring That Forensic Analysis Is Based Upon Task-Relevant Information. US Department of Justice, Washington, DCNational Research Council (2009). Strengthening Forensic Science in the United States: A Path Forward. National Academies Press, Washington, DChttps://doi.org/10.17226/12589Nickerson, R.S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220https://doi.org/10.1037/1089-2680.2.2.175Office of the Inspector General (2006). A Review of the FBI's Handling of the Brandon Mayfield Case. US Department of Justice, Washington, DCPronin, E., Lin, D.Y. and Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28(3), 369-381https://doi.org/10.1177/0146167202286008Risinger, D.M., Saks, M.J., Thompson, W.C. and Rosenthal, R. (2002). The Daubert/Kumho implications of observer effects in forensic science: Hidden problems of expectation and suggestion. California Law Review, 90(1), 1-56Thompson, W.C. (2009). Painting the target around the matching profile: The Texas sharpshooter fallacy in forensic DNA interpretation. Law, Probability and Risk, 8(3), 257-276https://doi.org/10.1093/lpr/mgp013Ulery, B.T., Hicklin, R.A., Buscaglia, J. and Roberts, M.A. (2012). Repeatability and reproducibility of decisions by latent fingerprint examiners. PLoS ONE, 7(3), e32800https://doi.org/10.1371/journal.pone.0032800