Subtleties in Bayesian decision-theoretic analysis for forensic findings: Notes on recent discussion of the role of validation study data in rational decision making.

This technical note extends a recent discussion in this journal of the role of validation study data in rational decision making. One argument that has been made in this context, using elements of Bayesian decision theory, is that further aggregation of validation study data into error rates involves a loss of information that compromises rational inference and decision making and should therefore be discouraged. This technical note seeks to explain that this argument can be developed at different levels of detail, depending on the definition of the propositions of interest, the forensic findings to be evaluated (and hence the form of the likelihood ratio), and the characterization of the relative desirability of decision consequences. The analyses proposed here reveal the cascade of abstractions and assumptions into which discussions about the use of validation study results in forensic science have fallen. This reinforces the conclusion that further aggregation of validation study data into error rates is problematic. It also suggests that even if a definition of error rate(s) could be agreed upon and defensively quantified in a given application, we should rethink and possibly adjust our expectations about what exactly error rates can practically contribute to rational modes of reasoning and decision making in legal contexts.