Miami University mineralogist Dr. Mark Krekeler was well aware of hyperspectral imaging — the technique regularly used to pinpoint mineral deposits — when it occurred to him that the same technology could be used to find humans, both dead and alive. Hyperspectral imaging, developed by NASA’s Jet Propulsion Laboratory in the late 1970s, has become an important tool in several scientific fields, including agriculture, environmental monitoring, food processing, and cell analysis in biology.
Krekeler, an associate professor in geology and environmental earth science at the Oxford, Ohio-based university realized that the technology could use drones with sensors to quickly cover large areas and find human bodies. Supported by the right library of spectral data, such imaging could identify a hiker lost in the wilderness, or the body of a crime victim dumped in the woods.
The key to interpreting the spectral data from a sensor depends on an extensive library that can match information from the sensor with catalogued objects in the library. But there was a gap in existing spectral library data that prevented the detection of “human materials such as human skin and hair, blood, bone and clothing,” Krekeler wrote in a 2015 research proposal to the National Institute of Justice (NIJ).
He proposed creating “a robust hyperspectral library and software tool that contains high quality reference spectra and extensive metadata for human materials and related geologic, environmental, and confuser materials.” He wanted to be able to find not only individual bodies, but crime scenes and, perhaps, mass graves. He received an NIJ grant of more than $500,000 and began collecting the “human materials” data.
“The idea is that as we use hyperspectral drones more and more, law enforcement and other U.S. government agencies will be able to search large areas looking, for example, for missing hikers,” Krekeler said. “We really hoped it would save lives.” And, he noted, “you can also search for human remains.”
In the example of a lost hiker, Krekeler said, authorities would tell the person analyzing the hyperspectral imagery that the hiker is a Black woman wearing a blue cotton dress. If the appropriate spectral data is in the library, he said, “You can make that spectra (skin tone and the blue cotton dress) and then look for it, provided that you have the imagery.”
Creating the “human materials” data was the primary goal of the 2015 NIJ grant, and Krekeler has added hundreds of spectra to his library. His database now includes the spectra of more than 300 clothing samples, in both wet and dry conditions (moisture changes the spectra), skin tone data from 110 people of various ages and ethnicities, and blood, both fresh and aged. In order to obtain accurate data on blood spectra, Krekeler used human blood purchased online and pipetted it onto clothing, rock, sediment, and wood. About 8,000 individual spectra from blood on different materials, both fresh and aged, were collected, he said, as well as “blood confuser materials” such as ketchup, red wine, lipstick, and barbecue sauce. Spectra were also taken from food items such as table salt; confectioners’, brown, and granulated sugar; and all-purpose flour — items that could be misconstrued as drugs.
To make Krekeler’s spectral library as comprehensive as possible, the United States Geological Survey hyperspectral library was integrated into the database, as was street sediment from Gary, Indiana, and a set of samples from the New Mexico-Texas border. Krekeler said those samples were chosen because that area is of interest for immigration and human and drug trafficking issues.
Extensive data were also collected on fuel spills for gasoline, diesel, and jet fuel, in both warm and cold weather conditions, on the assumption that hyperspectral imaging could be valuable in surveying crash sites, including mass casualty sites from airplane crashes.
Krekeler worked with technology experts at the Colorado-based L3Harris Geospatial to incorporate his forensic database into the company’s existing ENVI (Environment for Visualizing Images) software tool.
“ENVI is a full-featured remote sensing platform that is used in image and geospatial analysis,” said Andrew Fore, of L3Harris Geospatial. “Mark’s project came out of our solutions delivery team — that is our team of scientific experts in programming, environmental science, and application development. This was unique to us but everyone was happy because, although there was some customization, the existing hyperspectral tools within ENVI were able to handle the changes.”
Fore detailed an example of the possibilities using the case of a missing hiker in dense woods. If authorities were flying drones and using Krekeler’s tool as part of a hyperspectral sensor search, the technology could distinguish between a bear or a mountain lion and a person in the woods.
Krekeler noted that although human hair looks similar to other animal hair to a sensor, “our clothing does not, particularly synthetic clothing.” Additionally, he said, “Our skin is different, so there would be a high probability of distinguishing between a deer and a person. You want to distinguish people from other things, and it is going to help to have some knowledge [in the database] of what you’re looking for.” That is where the spectral scans of clothes, skin tones, and blood are valuable, he said.
In his final report to NIJ, Krekeler noted that a key feature of his spectral library is the ability to synthetically mix the data to approximate most conditions expected in search and rescue, criminal investigations, and other scenarios. “For example,” he wrote, “sediment can be mixed with clothing to look for a lost hiker.” And blood “can be mixed with clothing and sediment to search for a body, and rock and sediment can be mixed with blood to track a wounded or escaped person.”
Krekeler said he was finalizing his work so his hyperspectral tool can be made available to law enforcement. “It’s a great tool for forensics,” he said. “We want to set up a link where people in law enforcement can go and download the software and use this at no cost.” He expected the program to be available within a couple of months. Dr. Krekeler can be contacted at [email protected].
Developing a Protocol
The next step in implementing drones with hyperspectral imaging sensors for law enforcement and forensic applications is to develop protocols for use. A 2019 NIJ grant to Texas State University plans to do just that. Dr. Daniel Wescott will be using evidence-based research data to develop best practices for “search and recovery teams with easy-to-read, flexible recommendations for detecting and documenting outdoor crime scenes.” As these technologies become more affordable and more widely available, evidence-based guidelines will be invaluable for routine investigative and forensic use, in addition to search and recovery. The project aims “to provide law enforcement, anthropology, and civilian search teams with flexible scientifically based best practices and protocols for non-invasive search and documentation of clandestine graves and surface human remains using unmanned aerial systems equipped with multispectral remote sensors for outdoor crime scenes.”
About This Article
The research described in this article was funded by NIJ grant 2015-DN-BX-K011 awarded to Miami University. This article is based on reports submitted to the National Institute of Justice.