Ballistic Imaging National Research Council Peer Review. "At the current time it is not recommended that a mandate for implementation of this technology be made. Further testing, analysis and evaluation is required."

Ballistic Imaging

National Research Council

Telephone News Conference

March 5, 2008

Opening Statement

by 

John Rolph

Professor of Statistics, Marshall School of Business, University of Southern California

and

Chair, Committee to Assess the Feasibility, Accuracy, and Technical Capability

of a National Ballistics Database

Good morning, and thanks for joining us for the release of our report Ballistic Imaging.

In recent years, the desire to aid the investigation of gun crimes and lower the nation's level of gun violence has led legislators and advocacy groups to urge new initiatives to help link ballistic evidence found at crime scenes to the weapons from which they were fired.

One solution that has been proposed is the creation of a reference database that would contain ballistic images from all new and imported guns sold in the United States. The National Institute of Justice of the U.S. Department of Justice asked the National Research Council to assess how accurate and useful such a database is likely to be.  Our committee's new report does so, and also discusses a complementary approach called "microstamping," a newer technique to help link cartridge cases and bullets to guns.

Ballistic images depict the marks left on bullets and cartridge cases when they are fired from a gun. Called toolmarks, these marks have been assumed to be unique to each gun and have long been used to investigate crimes. For example, an investigator might compare a bullet found at a crime scene to one test-fired from a suspect's gun, to try to determine whether the toolmarks match.

Databases of ballistic images help investigators conduct these comparisons on a far larger scale, allowing them to search across the images of toolmarks from many thousands of firearms.  An existing national database run by the Bureau of Alcohol, Tobacco, Firearms, and Explosives, for example, contains ballistic images from weapons used in crimes and from guns recovered by police.  It is important to note that database searches only suggest possible matches; a final determination of a match is made by a firearms examiner, through direct physical comparison of the exhibits in question.

A national reference database would include images from all new and imported guns sold in the United States, not just weapons linked to crimes. When a new gun is sold, images of cartridge cases fired from it would be entered into the database. Investigators around the country could enter images of crime-scene ballistic evidence and search the reference database for images of toolmarks that match -- which would presumably help them identify possible weapons and the places they were sold. Such reference databases have already been established in Maryland and New York. 

The committee concludes that a national reference database of ballistic images should not be established. 

Images from more than a million guns would be entered into such a database every year, and many of these would have similar toolmarks. Because current technology for collecting and comparing toolmarks is not sufficiently precise in distinguishing extremely fine marks among so many images, searches would return too many possible matches to be practically useful. In addition, the type or brand of ammunition used in the initial firing of a gun would not necessarily be the same as the ammunition later used in a crime.   This difference could be a significant source of error in generating possible matches. 

The committee was not charged with issuing a verdict on whether toolmarks are unique -- that is, whether each gun leaves marks that could not be made by any other firearm. Nor was the committee charged to recommend whether toolmark evidence should be admissible in court.  However, it became clear to the committee early on that the assumption that toolmarks are unique to each gun has not yet been fully scientifically demonstrated. Much more research would be needed to determine whether toolmarks are truly unique, or even to estimate the probability that they are unique.

Consequently, the committee advises against one statement frequently made by firearms examiners -- that toolmarks link bullets or casings to a particular gun "to the exclusion of all other firearms."  Given that uniqueness has not been demonstrated, such statements do not have a firm statistical basis. These claims, which imply an error rate of zero, also don't reflect the element of subjectivity involved in declaring a match, a determination that's always made by a firearms examiner.

The conclusion that uniqueness has not been fully established doesn't mean that toolmarks or ballistic imaging technologies are not useful.  It is clear to the committee that the toolmarks generated by firing guns are not completely random and volatile, and that the current imaging technology is definitely helpful in generating leads for law enforcement investigation.  Thus, our report recommends many ways to improve ATF's existing database of crime-related ballistic evidence, called the National Integrated Ballistic Information Network or NIBIN, which permits investigators at about 230 participating law enforcement agencies to search evidence against images from their own jurisdiction as well as from other departments.

For example, ATF should consider establishing protocols for entering images of multiple exhibits from the same gun, ideally involving various types of ammunition; currently, only a single exhibit is entered in some jurisdictions. The report also recommends several ways to improve the database's technical platform -- for example, by making it simpler to conduct searches across multiple regions of the country.  In addition, we recommend more research on a possible future shift from using two-dimensional photographic techniques to using three-dimensional surface measurement techniques.

We also urge further research on microstamping, a promising alternative to creating a national ballistic database that has been legislated in California and is pending in Congress. This technique places a unique identifier -- such as an alphanumeric code -- on gun parts or ammunition, and it would have the formidable advantage of imposing uniqueness on ballistic evidence.  However, studies have not yet determined how durable microstamped marks are under various firing conditions, how susceptible they are to tampering, or wha

t their cost would be for manufacturers and consumers. We strongly encourage research on these and other issues related to microstamping, as this method may indeed be a viable future approach to firearms investigation.  

This concludes my opening statement. We are now happy to take your questions.

http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=03052007

Date: March 5, 2008

Contacts:    Sara Frueh, Media Relations Officer

Luwam Yeibio, Media Relations Assistant

Office of News and Public Information

202-334-2138; e-mail <news@nas.edu> 

FOR IMMEDIATE RELEASE

Report Advises Against New National Database of Ballistic Images

 WASHINGTON — A national database containing images of ballistic markings from all new and imported guns sold in the U.S. should not be created at this time, says a new report from the National Research Council.  Such a database has been proposed to help investigators link ballistics evidence -- cartridge cases or bullets found at crime scenes -- to a firearm and the location where it was originally sold.  But given the practical limitations of current technology for generating and comparing images of ballistic markings, searches of such an extensive database would likely produce too many candidate "matches" to be helpful, the report says. 

The report notes that the fundamental assumption underlying forensic firearms identification – that every gun leaves microscopic marks on bullets and cartridge cases that are unique to that weapon and remain the same over repeated firings – has not yet been fully demonstrated scientifically.  More research would be needed to prove that firearms identification rests on firmer scientific footing, said the committee that wrote the report. 

Nevertheless, current ballistic imaging technology can be useful in generating leads for law enforcement investigation, said the committee.  Its report recommends ways to improve the usefulness of an existing ballistic image database – limited to ballistics evidence associated with crimes – that is administered by the U.S. Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) and used by more than 200 state and local law enforcement agencies.  It also recommends further research on "microstamping," a technique that imprints unique marks on guns or ammunition.  This promising method could be an alternate way to attain the same basic goal as the proposed database.

National Database Would Be of Limited Usefulness

"Toolmarks" are created on cartridge cases and bullets when a gun is fired -- for example, when a bullet scrapes against grooves on the inside of the gun barrel, or when high gas pressure forces the walls of a cartridge case against the gun's firing chamber.  These toolmarks have long been used to help solve crimes -- for example, a firearms examiner might compare a crime-scene bullet to one test-fired from a suspect's gun to determine whether the marks match.  Since the 1980s, computerized imaging has allowed law enforcement agencies to input toolmark images in databases of crime-related ballistic evidence and search for images of bullets or cases with similar marks. 

The National Institute of Justice of the U.S. Department of Justice asked the National Research Council to assess the feasibility of a national database that would contain images of toolmarks from all new and imported guns; about 4.5 million new guns are sold in the U.S. each year, including about 2 million handguns.  With such a system, when a gun is sold, images of cartridge cases from a firing of that gun would be entered into the database, possibly with information on its original purchaser.  Investigators around the country who collect ballistic evidence at crime scenes could search the database for possible matches.  Maryland and New York already operate such databases for guns sold or manufactured in those states.

A number of problems would hinder the usefulness and accuracy of a national database, the report says.  Ballistic images from millions of guns could be entered each year, and many of the images would depict toolmarks that are very similar in their gross characteristics.  Research suggests that current technology for collecting and comparing images may not reliably distinguish very fine differences in large volumes of similar images, the report says.  Searches would likely turn up too many possible "matches" to be useful.  Also, the type of ammunition actually used in a crime could differ from the type used when the gun was originally test-fired – a difference that could lead to significant error in suggesting possible matches.  

The report does recommend 15 improvements to the ATF's National Integrated Ballistic Information Network (NIBIN), an existing database that contains ballistic images from crime scenes and suspects' weapons.  Seven recommendations focus on improving the operation of the NIBIN program; for example, the program should consider protocols for entering multiple images from the same gun – ideally involving multiple ammunition types – rather than relying on a single "best" case.  The report also recommends eight ways to improve the database's technical platform -- for instance, by simplifying routines for conducting searches across multiple regions of the country.  The committee examined the possibility of using three-dimensional surface measurement techniques rather than two-dimensional photographic images, but suggests the need for further research and testing before such a change is made.  

Claims of Certainty About 'Matches' Without Firm Grounding

The report does not assess the admissibility of firearm toolmark evidence in legal proceedings, since making such a determination was not part of the committee's charge.  However, it cautions that the statement commonly made by firearms examiners that "matches" of ballistic evidence identify a particular source gun "to the exclusion of all other firearms" should be avoided.  There is currently no statistical justification for such a statement, and it is inconsistent with the element of subjectivity inherent in any firearms examiner's assessment of a match.

If firearms identification is to rest on firmer scientific ground, more research would need to assess the fundamental assumption that toolmarks are unique and remain recognizable over time, despite repeated firings.  Such research should include a program of experiments covering a full range of factors that may degrade a gun's toolmarks, as well as factors that might cause different guns to generate similar toolmarks.  Intensive work is also needed on the underlying physics, engineering, and metallurgy of firearms, in order to better understand the mechanisms that form toolmarks as a weapon is fired.

Microstamping Should Be Studied

The report also recommends more research on a promising alternative approach to providing links between crime-scene evidence and the original weapon.  "Microstamping" etches or engraves unique markings -- such as an alphanumeric code -- on gun parts, which in turn generate unique marks on spent cartridge cases; microstamped marks could also be applied to individual pieces of ammunition.  These marks could be rapidly examined at crime scenes using equipment as simple as a magnifying glass.  However, more in-depth studies are needed on the durability of microstamped marks under various firing conditions and their susceptibility to tampering, as well as on their cost impact for manufacturers and consumers.  California recently passed a law to require microstamping on internal parts of new semiautomatic pistols sold in the state by 2010.

The study was sponsored by the U.S. Department of Justice's National Institute of Justice.  The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies.  They are private, nonprofit institutions that provide science, technology, and health policy advice under a congressional charter.  The Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering.  A committee roster follows.

Copies of Ballistic Imaging are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu.  Reporters may obtain a copy from the Office of News and Public Information (contacts listed above). 

NATIONAL RESEARCH COUNCIL

Division of Behavioral and Social Sciences and Education

Committee on Law and Justice and Committee on National Statistics

and Division on Engineering and Physical Sciences

National Materials Advisory Board

Committee on Assessing the Feasibility, Accuracy, and Technical Capability of a National Ballistics Database

 
John E. Rolph (chair)
Professor of Statistics
Marshall School of Business

University of Southern California
Los Angeles

Eugene S. Meieran 1 (vice chair)
Senior Fellow and Director of Manufacturing
Strategic Support
Intel Corp.
Chandler, Ariz.

Alfred Blumstein 1
J. Erik Jonsson Professor of Urban Systems and Operations Research
H. John Heinz III School of Public Policy and Management
Carnegie Mellon University Pittsburgh

Alicia Carriquiry
Professor
Department of Statistics
Iowa State University
Ames

Scott Chumbley
Professor
Department of Materials Science and Engineering
Iowa State University
Ames

Philip J. Cook 2
ITT/Terry Sanford Distinguished Professor of Economics and Sociology
Terry Sanford Institute of Public Policy
Duke University
Durham, N.C.

Marc De Graef
Professor
Department of Materials Science and Engineering 
Carnegie Mellon University
Pittsburgh

David L. Donoho 3
Professor of Statistics, and
Anne T. and Robert M. Bass Professor of Humanities and Science 
Stanford University
Stanford, Calif.

William F. Eddy
John C. Warner Professor of Statistics
Carnegie Mellon University
Pittsburgh

George T. Gray III
Fellow
Materials Science Division
Los Alamos National Laboratory
Los Alamos, N.M.

Eric Grimson
Bernard Gordon Professor of Medical Engineering and Head
Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Cambridge

Daniel P. Huttenlocher
John P. and Rilla Neafsay Professor of Computing, Information Science, and Business
Department of Computer Science, and
Stephen H. Weiss Fellow
Cornell University
Ithaca, N.Y.

Michael M. Meyer
Google Inc.
Seattle

Vijay Nair
Donald A. Darling Professor and Chair
Department of Statistics, and
Professor of Industrial and Operations Engineering
University of Michigan
Ann Arbor

Angelo M. Ninivaggi Jr.
Vice President, General Counsel, and Secretary
Plexus Corp.
Neenah, Wis.

David W. Pisenti
Law Enforcement Consultant

Fredericksburg, Va.

Daryl Pregibon
Research Scientist
Google Inc.
New York City

Herman M. Reininga
Senior Vice President of Operations
Rockwell Collins (retired)
Cedar Rapids, Iowa

James K. Stewart
Senior Fellow 
The CNA Corp.
Alexandria, Va.

Michael R. Stonebraker 1
Professor of Computer Science
Massachusetts Institute of Technology
Cambridge

Julia R. Weertman 1
Walter P. Murphy Professor Emerita of Engineering
Northwestern University
Evanston, Ill.

RESEARCH COUNCIL STAFF

Carol Petrie

Study Director

Director, Committee on Law and Justice

Daniel Cork

Senior Program Officer

Committee on National Statistics

Gary Fischman

Director, National Materials Advisory Board

1  Member, National Academy of Engineering

2  Member, Institute of Medicine

3  Member, National Academy of Sciences

Not Picture Perfect