Developing More Uniform Comparison Language in Forensic Science Reportings

An ongoing challenge within the forensic science community is the development of consistent report and testimony language that conveys results in a meaningful manner. Ideally, that language allows the reader to understand what was examined, how it was examined, and the results. A piece that is often absent from the report is what the results mean in the context of possible conclusions that could have been reached. An effort to add context has been particularly of interest in the characterization and comparison of trace evidentiary materials (such as fibers, glass, hair, paint, and tape), where the greatest criticism in the past decade has been directed toward the language used to simultaneously convey results to the technical, legal, and lay communities. The meaning of the descriptions becomes more understandable to the greater spectroscopic community when referring to spectral searches of an unknown (or test) specimen to a known material or a reference spectral library. Various working groups are trying to define the technical conclusions that may be drawn by comparing samples and what legal ramifications may be arrived at by making these comparisons.

To address this goal, forensic laboratories and consensus bodies are working toward descriptive language to better align report writing and testimony language across the field. As an example, the National Institute of Standards and Technology (NIST) currently oversees the Organization of Scientific Area Committees (OSAC), a consortium of Scientific Area Committees (SACs) that develops consensus-based best practices, test methods, and guidance documents. Draft documents are then sent to Standard Development Organizations (SDOs) such as ASTM International for ratification and dissemination. The Materials (Trace) Subcommittee of the Chemistry SAC (1) has developed an interpretation guidance document for use in reporting comparative examinations involving trace evidentiary materials, the “Standard Practice for Interpretation and Report Writing in Forensic Comparison of Trace Materials” (2), which provides a set of terms and a verbal scale that can be used to convey the significance of a comparative association. The interpretation scale is organized as a range of conclusions that could be used to describe a reported result. The conclusion that best fits the determined result is dependent on the material, its condition, the number of recommended orthogonal comparative techniques that were used to evaluate the materials, and the relative number of other materials that could also fulfill the conditions described within a particular type of conclusion. Following the general outline of the verbal scale, examples of report wording for a given scenario are provided to assist a trace evidence examiner with determining what this approach would look like in practice for the materials they evaluate and compare.

When comparing two or more samples, the most definitive conclusions are that the samples could not have originated from the same source (elimination) or that they share features so rare as to have once been part of a larger intact item that fractured at the free ends of the intercompared items (physical fit). Between these two extremes lies a range of possible conclusions that describe the inability to distinguish samples based on their respective physical, optical, or chemical properties. In this class association, the evaluated indistinguishable features are also shared with any other sample that contains these same properties and therefore are not exclusive to the items being compared. Associations of this type can vary in significance and the number of like samples that would share a particular property or set of properties can decrease as the number of evaluated characteristics increases.

In part, therefore, the content of a verbal scale is meant to provide a framework within which to provide context to a class association. This context is particularly helpful for mass-produced manufactured materials that are readily available in the consumer marketplace (such as automotive paint). For example, a report on the inability to distinguish the physical and chemical characteristics of blue metallic automotive paint samples might fall within the context of a class association as follows:

Association with Discriminating Characteristics - An association in which items could not be differentiated based on the examinations conducted. Therefore, the possibility that the items came from the same source cannot be eliminated. Other items have been manufactured or could occur in nature that would also be indistinguishable from the submitted items and could be encountered in the relevant population. This provides moderately strong to strong support for the proposition that the items originated from the same source as opposed to the proposition they originated from
different sources (2).

This interpretation of the reported conclusions states that, even though comparative testing was able to conclude that the paint was a four-layered automotive coating with a blue-colored basecoat and effect (metallic) pigmentation, it is still a mass-produced entity and available on every make and model of like-colored vehicle produced at the same assembly plant. The significance addressed with this language is twofold: Other vehicles produced at other plants (even of the same make and model) would not share the same set of physical and chemical properties; and, other vehicles produced at a given plant with the same set of physical and chemical properties would be indistinguishable. This approach to interpretation of the results attempts to convey both the significance and the limitations of a typical class association.

Sometimes, the evidence contains features that are not as common within a given material class. Alternatively, some alteration may have occurred that makes it obvious that the material now contains features that are not mass-produced. For reporting these types of class associations, wording is used to convey that the number of indistinguishable samples that would also share these same characteristics would be less than expected for a mass-produced material. The verbal scale language to address this type of conclusion would read as:

Association with Highly Discriminating Characteristics - An association in which items could not be differentiated based on the examinations conducted. Therefore, the possibility that the items came from the same source cannot be eliminated. Additionally, the items share unusual characteristics that would rarely or never be expected to occur in the relevant population. This provides very strong to extremely strong support for the proposition that the items originated from the same source as opposed to the proposition they originated from different sources (2).

An example of this type of association would be an automotive paint system that contains an architectural paint layer as well. Regardless of the number of physical and chemical properties that could not be used to differentiate the general class of these samples, this category would indicate that there are some properties that set these comparative samples apart into a smaller subset of like samples (for example, a vehicle coated with an architectural finish by a consumer). It is not a conclusion of individuality because the indistinguishable properties remain class characteristics. It is also possible that another vehicle could be painted in the same manner; however, there is less expectation that such an unusual feature would be mass produced or widely repeated on unrelated samples.

Conversely, features can be evaluated between materials that have fewer characteristics available for comparison. This situation can be the result of a more generic material or as the result of insufficiently sized or contaminated samples. For reporting these results, the comparative language must be phrased to convey that a greater number of samples would also share indistinguishable measured properties. The verbal scale language would read as:

Association with Limitations - An association in which items could not be differentiated based on the examinations conducted. Therefore, the possibility that the items came from the same source cannot be eliminated. This provides slight to moderate support for the proposition that the items originated from the same source as opposed to the proposition they originated from different sources. As compared to the categories above, this type of association has decreased evidential value. For example, the items are more commonly encountered in the relevant population, a complete analysis was not performed due to limited characteristics or a limited analytical scheme, or minor variations were observed in the data. Minor variations could be due to factors such as known contamination of the sample(s) or having a sample of insufficient size to adequately assess heterogeneity of the entity from which it was derived. (2)

This category would be appropriate in the event that an incomplete paint layer system was received, such that characteristics normally present for comparison and known to provide increased levels of discrimination would not be available for testing and evaluation. Therefore, the number of samples that share the tested properties is larger than it would be if all the known properties were able to be evaluated. The inability to measure all of the expected properties does not render the class association as meaningless, but it does expand the number of samples that could also be undifferentiated under these conditions.

This verbal scale approach to providing consistency in describing forensic class associations has been years in the making. Current plans are to offer this type of scale model to the larger forensic science trace evidence community for use in interlaboratory exercises. Analysts will be given comparative scenarios and asked to assign a conclusion using the verbal scale model. Once the results of these test scenarios have been evaluated the results will be published. It is anticipated that valuable feedback will be gained on the efficacy of this model as well as the content of the guide. Comments on the draft guide may be sent to dmwright@fbi.gov.

Disclaimer

This is publication number 20-63 of the FBI Laboratory Division. Names of commercial manufacturers are provided for identification purposes only, and inclusion does not imply endorsement of the manufacturer, or its products or services by the FBI. The views expressed are those of the authors and do not necessarily reflect the official policy or position of the FBI or the U.S. Government.

References

1. The Materials (Trace) Subcommittee of the Chemistry Scientific Area Committees of NIST. Home page: https://www.nist.gov/topics/organization-scientific-area-committees-forensic-science/materials-trace-subcommittee. Accessed May 20, 2020.
2. The Organization of Scientific Area Committees for Forensic Science (OSAC), Materials (Trace) Subcommittee, Standard Practice for Interpretation and Report Writing in Forensic Comparison of Trace Materials, Draft (NIST, Gaithersburg, MD, February 28, 2020).

Diana Wright is a forensic examiner and Technical Leader of the Paints and Polymers group within the Chemistry Unit of the FBI Laboratory in Quantico, Virginia. She is also the current Chair of the Materials (Trace) Subcommittee of the Organization of Scientific Area Committees (OSAC) of NIST. Direct correspondence to dmwrightosac@gmail.com