Reference Ballistic Toolmark Database for Research and Development of Identification Systems and Confidence Limits

The project objective is an open-access research database of bullet and casing reference data, consisting of traditional reflectance microscopy images and three-dimensional surface topography. The database will foster the development and validation of advanced algorithms, mathematical similarity criteria, and quantitative confidence limits for objective ballistics identification. In 2009, a report by the National Academies called into question, amongst other issues, the objectivity of visual toolmark identification by firearms examiners. The National Academies recommended development of objective toolmark identification criteria and error rate estimates. Industry, academia, and government laboratories are pursuing two promising approaches towards this goal: 1) development of mathematical criteria and advanced algorithms for the objective and automated identification and scoring of potential matches, and 2) supplementing traditional reflectance microscopy images with three-dimensional surface topography measurement data. Development and validation of both these approaches to objective toolmark identification are hindered by a lack of access to toolmark data sets that 1) represent the large variety of ballistic toolmarks encountered by toolmark examiners, and 2) represent challenging identification scenarios, such as those posed by consecutively manufactured firearms components. It is not economically feasible for individual companies or institutions to generate their own data sets. This makes it difficult for these entities to objectively evaluate their solutions. During the symposium held at NIST entitled “Measurement Science and Standards in Forensic Firearms Analysis 2012,” one of the priority requests from the attendees was the construction of a website and database where bullet, cartridge case and toolmark surfaces could be shared between researchers, whereby new systems, methods, and algorithms could be tested, refined, and compared.

The proposed database will provide the representative variety of toolmark data required, ranging from crime lab test fires to test fires conducted using consecutively manufactured barrels, firing pins, slides and other firearm surfaces. The database will contain both reflectance microscopy images and three-dimensional surface topography data. The database and associated web site will consist of two parts: 1) an open “read-only” site consisting of indexed data acquired at the National Institute of Standards and Technology (NIST) using state-of-the-art instruments and measurement procedures, and 2) a secure “sandbox” site that consists of shared, indexed, user-supplied, acquisitions in a common format. Some of the data collected will be stored in a closed database for possible future application to the validation and comparison of correlation software. The proposed database will enable researchers to test and validate new approaches to objective, mathematics-based, toolmark identification while easing the transition to three-dimensional surface topography data. The database will provide a foundation for a scientific knowledge base on the degree of similarity that can be found between marks made by different firearms and the variability in marks made by an individual firearm. The current, ‘fairly limited,’ knowledge base is a fundamental barrier to the development and validation of objective mathematical similarity criteria, and associated confidence limits, applicable to a broad range of firearms and ammunition brands.

Although not addressed in this project, the database will provide opportunities for: 1) quality control of crime lab measurements and instrumentation development through supplementary toolmark replicas corresponding to reference images in the database, and 2) Government-organized competitions between instrument or software vendors with the goal of advancing the technology as quickly as possible. This is similar to the approaches taken for the development of systems for fingerprint identification and for facial recognition. The project builds on expertise at NIST in the measurement and analysis of ballistic toolmark data and the development of data standards and databases for various biometric modalities (fingerprint, face, iris, voice, and DNA).