Expert Systems Help Labs Process DNA Samples

Criminal justice professionals who work in the field of DNA analysis know that a backlog of convicted offender samples exists in our nation's laboratories. It takes a long time to analyze a DNA sample of a convicted person. Two forensic analysts must visually review the sample and apply a set of standard operating procedures that can have many sets of rules. The procedures can be difficult to apply consistently. After the review is complete, the data must be entered into the national database.

Completing all the steps quickly is a formidable challenge.

New software programs called "expert systems" are helping increase the speed of the review process. Expert systems capture all possible circumstances that experts encounter when they do their jobs and dictate what the appropriate responses should be. For forensic analysts, expert systems not only allow them to get consistent, accurate results more quickly, they also help them review and upload many DNA profiles into the national database faster.

But can these expert systems help reduce the backlog of convicted offender samples that exists in our nation's laboratories?

According to the results of an evaluation by the National Institute of Justice (NIJ), they can. The NIJ Expert Systems Testbed (NEST) project, hosted by the Marshall University Forensic Science Center, evaluated the ability of three commercially available expert systems, designed specifically for forensic DNA laboratories, to rapidly and accurately review convicted offender single-source DNA samples for eventual upload into the national DNA database. Project researchers — including this author — found that the three programs will help reduce the backlog and ensure timely submission of data into the national database so ultimately more crimes can be solved.[1],[2]

What Are Expert Systems?

Expert systems are a subset of artificial intelligence. They store knowledge on how to respond to a particular result and, when a challenge is presented, use the stored knowledge in the program to respond with an explanation. The system, however, cannot completely replace a human expert because it contains only rules to solve most commonly encountered problems. The system must be able to recognize cases that are outside its rules and scope of knowledge or those in which there is a possible alternate judgment and alert a human expert reviewer.

Expert systems are developed with the close collaboration of the software development team and forensic science experts. Developers create the system by first conducting extensive interviews with experts in the field where the software will be used, writing algorithms to apply to the data and testing the programs with data appropriate for the software. The knowledge from these experts is then taken and applied consistently to the data review process by using the algorithms, automating the analysis and freeing analysts to focus on more complex challenges.

Use of this technology continues to increase in several industries as they become more aware of its benefits — for instance, expert systems are assisting physicians in making medical diagnoses, supporting NASA's space program and managing inventories for large factories. They serve as scheduling and planning tools when the everyday consumer makes an online airline reservation. A search engine finds all possible airline companies, determines the routes based on particular requests and attempts to find the requested time frame. Once the consumer chooses a particular flight plan, the system can determine the class of travel, price and seat assignments.

For forensic DNA analysis, expert systems could easily be one of the most important advances in analyzing convicted offender samples. By reviewing routine data rapidly and accurately, the system will allow analysts to focus on those samples with problems or "flags." Also, because DNA analysis requires considerable training and experience, software that interprets information with little or no work by the forensic analyst is a noteworthy advancement for the DNA community.

Rapid, Accurate Analysis

Our evaluation focused on the technical review of software programs that:

• Are publicly available for purchase.
• Are configurable, off-the-shelf software.
• Are housed in a laboratory.
• Can be used by people who are unfamiliar with computer code.
• Meet technical specifications outlined in the forensic standards.[3]

We evaluated three software packages that met these criteria:[4]

• GeneMapper^®ID v. 3.2 (Applied Biosystems, Foster City, Calif.).
• FSS-i³™ (Promega Corporation, Madison, Wis.), used with GeneMapper^®ID v. 3.2 for peak detection and sizing.
• TrueAllele^® System 2 (Cybergenetics, Pittsburgh).

We evaluated every step involved from buying the programs to running them. Our evaluation examined the time it takes to import data, run the application and complete the analysis. We also critically checked the flags, rules and features available with each program. We evaluated their ability to run without assistance by the analyst and to flag the samples needing human expert review. We further looked at the customer service and training provided by the vendors' technical staff.

Our team found that the three expert systems evaluated for single-source samples will help reduce the DNA convicted offender backlog.[5] We determined that each is able to rapidly and accurately conduct routine reviews (reviews of DNA samples without flags) and can significantly reduce the time spent in the human review of DNA profiles. When implementing one of these expert systems, only one analyst is needed to accept the good quality samples (with no expert system in place, two analysts are needed during a review), thereby reducing labor hours by half.

We determined, however, that the systems cannot fully replace a human expert or DNA analyst. The systems can be used as tools to evaluate single-source DNA samples and to alert the DNA analyst when results do not meet defined rules or there is a possible alternate judgment. But the final decision-making process in cases when results raise flags must remain in the hands of the trained DNA analyst.

As part of our evaluation, we also presented the features and limits of each system to help forensic analysts determine which program will most benefit their laboratories and to aid managers when deciding whether to purchase or subscribe to expensive software. Here is a brief summary of each expert system that we evaluated.

GeneMapper^®ID Software v. 3.2

We found the purchase of GeneMapper ID straightforward. The laboratory in which we evaluated the product purchases only software from the vendor; hardware is purchased separately. The consumer determines the number of licenses required by the laboratory and can self-install the software from a CD. The vendor provides a one-day on-site training on setup and basic software tools with purchases as well as technical support for telephone and electronic questions and free periodic webinars. More extensive training can be purchased. Our evaluation found many features in GeneMapper ID to be intuitive and the sizing and genotyping analysis to be simple and effective.

FSS-i³™ Expert Systems Software

With FSS-i³, hardware is purchased separately; the laboratory purchases only software from the vendor. The vendor offers on-site or company-site training at a separate cost. The five-day sessions include installation, optimization and training. The vendor also offers technical support for telephone and electronic questions. Overall, we found the operation of the software straightforward.

TrueAllele^® System 2

With TrueAllele, hardware is included with the software. The laboratory can renew its support and updates annually. The vendor provides two company-site training sessions: a two-day executive training session for laboratory managers and administrators and a four-day training session for scientists using the software. The vendor also provides technical support for telephone and electronic questions. During our evaluation, we found that we had a steeper learning curve for TrueAllele, but once we understood the setup, it was fairly straightforward and the analysis of the size standard and samples was simple and effective.

It is not the objective of the NEST project to identify the single best expert system for a laboratory. We recognize that crime laboratories are different, and each laboratory should consider its specific needs when choosing an expert system. Instead, the goal of the NEST project is to continue to give managers and analysts the tools they need to determine which expert system would best fit their laboratory and ultimately help reduce the backlog of convicted offender samples across the country so more crimes can be solved.

Considerations When Selecting an Expert System

When deciding whether to purchase an expert system, managers and analysts should consider their throughput needs, budget, human resources and information technology (IT) support as well as the features of the system and the continuing support of the vendor. Questions to ask include:

• How many instruments will be used to process single-source samples?
• On average, how many samples do you plan on processing in a week?
• How fast do you need to process your data?
• What is your review backlog?
• How many analysts do you have on your staff for processing single-source samples?
• What is the level of expertise for data review in your staff?
• Do you want the vendor to provide continued data optimization, or do you want to have all optimization control?
• Do you have information technology support in your agency?
• Do you have information technology support specifically for the DNA laboratory?
• Is the provision of an expert system tool at multiple workstations important? Would a single workstation be easier to manage?
• What is your budget for purchasing an expert system?
• Do you have the budget for the maintenance and support contracts?
• What are the needs for your quality assurance program?
• Would you like to have features in your expert system to help identify quality assurance problems?
• Which training program fits the needs and philosophy of your agency?
• Do you prefer training at your laboratory or training at the vendor site?
• Is a Web-based training program sufficient for your needs?

About This Article

This article appeared in NIJ Journal Issue 260, July 2008.