January 2009

Ever since the Supreme Court’s decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), expert opinion testimony has been subject to increased scrutiny relative to its ability to withstand "testing pursuant to the scientific method." Computer modeling and simulation can be a tool for withstanding a Daubert challenge, by providing a method for testing expert opinion testimony. This article will address the foundations of Daubert, with a primary focus on Daubert’s "testing factor." This article will also address how computer modeling and simulation can be used as a testing method to substantiate, both scientifically and legally, the admissibility of expert opinion testimony.

Prior to the adoption of Daubert, federal courts and most state courts evaluated the admissibility of expert testimony based upon the "general acceptance test" as set forth in Frye v. United States, 293 F.1013 (D.C. Cir. 1923). Per Frye, courts admitted "expert testimony deduced from a well recognized scientific principal or discovery, [so long as] the thing from which the deduction is made [is] ... sufficiently established to have gained general acceptance in the particular field in which it belongs." Id. at 1014. However, in 1993, per the landmark Daubert decision, the Supreme Court held that the "Frye test" had been superceded by the adoption of the Federal Rules of Evidence. See Daubert, 509 U.S. at 587. Accordingly, federal judges were called upon to serve as "gatekeepers" with regard to the admission of expert opinion testimony

The trial judge must determine ... whether the expert is proposing to testify to (1) scientific knowledge that (2) will assist the trier of fact to understand or determine a fact in issue. This entails a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue. We are confident that federal judges possess the capacity to undertake this review.

Id. at 509 U.S. at 592-93.

Most lawyers, commentators and courts considered the Daubert decision to be a significant departure from the established rules governing the admissibility of expert opinion testimony at trial. Many articles, treatises, and Court decisions subsequently addressed the meaning and significance of Daubert. By 2000, the Federal Rules of Evidence were amended to reflect the Supreme Court’s decision in Daubert and its progeny, in an attempt to codify the standard by which expert opinion testimony would be admitted at trial.

If scientific, technical or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training or education, may testify thereto in the form of an opinion or otherwise if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principals and method reliably to the facts of the case. Fed. R. Evid. 702 (2000). The ultimate impact of Daubert and Rule 702 has been summarized as follows:

Under Rule 702 and Daubert, district courts must act as "gatekeepers" which admit expert testimony only if it is both reliable and relevant. District courts are charged with this gatekeeping function "to ensure that speculative, unreliable expert testimony does not reach the jury" under the mantle of reliability that accompanies the appellation "expert testimony." To fulfill their obligation under Daubert, district courts must engage in a rigorous inquiry to determine whether: "(1) the expert is qualified to testify competently regarding the matters he intends to address; (2) the methodology by which the expert reaches his conclusions is sufficiently reliable as determined by the sort of inquiry mandated in Daubert; and (3) the testimony assists the trier of fact, through the application of scientific, technical or specialized expertise, to understand the evidence or to determine a fact in issue."

Rink v. Cheminova, Inc., 400 F. 3rd 1286, 1291-92 (11th Cir. 2005) (citations omitted).

The Daubert decision highlighted several "factors" to be utilized by trial judges to determine whether expert opinion testimony should be admitted at trial. Since 1993, the application of these "factors" has been the focus of an enormous amount of reported and unreported court decisions. Most courts have emphasized that Daubert’s "factors" are not a definitive "check list," while some courts have since added factors to their case-by-case evaluations of expert opinion testimony. Nevertheless, the first "factor" addressed in Daubert was "testing" pursuant to the scientific method

Ordinarily, a key question to be answered in determining whether a theory or technique is scientific knowledge that will assist the trier of fact will be whether it can be (and has been) tested. "Scientific methodology today is based on generated hypotheses and testing them to see if they can be falsified; indeed, this methodology is what distinguishes science from other fields of human inquiry."

Daubert, 579 U.S. at 593 (citations omitted). Notably, the "testing factor" - - albeit described as being no more important than any other factor - - has been the focus of a disproportionate amount of court battles concerning the admissibility of expert opinion testimony; and thusly, it is also the focus of this article. See also Phillips v. Raymond Corp., 364 F.Supp. 2d 730, n.2 (N.D. Ill. 2005) ("The first factor, the existence of testing employing a verified scientific method is ‘a very significant Daubert factor - - probably the single most important factor.’" Chapman v. Maytag Corp., F.3d 682, 688 (7th Cir 2002).).Those involved in the litigation process are well aware of the ever growing challenges that are posed with respect to the admissibility of expert opinion testimony. Daubert based challenges often focus their attack on the inability of the expert to demonstrate that his or her opinions were tested pursuant to the scientific method. See e.g. Thierfelder v. Virco, Inc., No. 06-4084-CV-C-NKL, 2007 WL 1859437 (W.D.Mo. June 26, 2007). (Court considered motion to exclude admission of expert testimony due to expert’s failure to conduct any tests, or computer modeling, to substantiate opinions to be offered at trial); Avance v. Kerr - McGee Chemical, LLC, No. 5:04 CV209, 2006 WL 3912471 (E.D. Tex. Dec. 6, 2006) (Court considered motion to deny admission of expert testimony due to expert’s failure to conduct tests in accordance with a scientific method). In sum, today’s courts are inundated with motions to exclude testimony due to the failure of experts to perform testing in accordance with the scientific method, as well as the experts’ inability to provide testimony based on such testing.

As a result, some courts have commented that they would look forward to opportunities to consider expert testimony that could be supported by tests founded on computer modeling and simulation. In Ortiz v. Yale Materials Handling Corp., No. CIV 03-3657FLW, 2005 WL 2044923 (D.N.J. Aug. 24, 2005), the trial judge granted the Defendant’s Motion to Strike the Plaintiff’s Expert Testimony, due to the expert’s failure to comply with the dictates of reliability set forth in Rule 702 and Daubert. In Ortiz, the court pointed out that the expert’s opinion testimony was unreliable, because he failed to perform relevant testing in accordance with the scientific method. Throughout his trial Order, the judge noted in Ortiz that the expert testimony at issue could have been more reliable and, thus, admissible through the use of computer simulation.

Here, Sevart has never performed any dynamic testing with a moving forklift, either with a dummy or human, nor has he done any computer simulations to test his proposed rear door alternative design and stay in-the-truck theory with the Yale forklift model at issue in this case. In fact, Sevart himself acknowledged that he did not performed any tests with a stand-up forklift, with or without a rear door, in a lateral tip-over, which is the type of accident at issue in this case. Sevart also admitted that he did not conduct any analysis or tip-over tests using the particular Yale model forklift that Plaintiff was operating at the time of the accident. In fact, Sevart never saw or operated the Yale model forklift involved in Plaintiff’s accident.

Id. at *6 (citations omitted) (emphasis added).

In light of this data, the Court repeatedly asked Sevart what statistical analysis he performed to account for the fact that there were substantially more operators who jumped from the forklift than stayed in the operator compartment. Sevart merely responded that the data did not reflect a "sophisticated analysis," and that he did not conduct any technical, statistical, or mathematical analysis with respect to such data.

Sevart merely responded that there was "no specific mathematical model created. It’s simply the numbers themselves were compared."

Id. at *7 (citations omitted).

The Court finds incredulous Sevart’s position that there was no way to test and obtain reliable answers in the area of forklift safety and lateral tip overs without using human subjects. While the Federal Rules of Evidence do not have specific provisions governing the admission of computer generated simulations, reconstruction and animation as substantive evidence, such computer generated evidence has long been accepted as an appropriate means to communicate complex issues to a lay audience, so long as the expert’s testimony indicates that the processes and calculations underlying the reconstruction or simulation are reliable. To restrict accident reconstructions to those involving only human tests subjects not only places such individuals in physical danger but also is a further indication of a unreliability of Sevart’s methodologies and opinions.

Id. at *9 (citations omitted) (emphasis added). Clearly, Ortiz serves as one example of a case where a Court was seeking evidence of testing via computer simulation, in order to buttress and ultimately support the admission of expert opinion testimony. See e.g. Agee v. Perdue Pharma., LP, No. 05-6002, 2007 WL 2045482 (10th Cir. July 18, 2007) (computer modeling substantiated the rejection of an expert’s own testimony).

II. Computer Modeling and Simulation

In the age of computer generated special effects, where the impossible becomes believable, it is critical to distinguish between "computer animation" and "computer simulation." Though these terms are used interchangeably in most commonplace conversations - - and sometimes even in the legal context - - there are very important distinctions between these two concepts. These distinctions also affect their credibility and accuracy and, thus, their admissibility in the courts.

Computer simulation is a method of "scientific modeling," where mathematical algorithms and solution techniques are utilized to solve problems in accordance with scientifically accepted and recognized equations, subject to certain conditions, boundary limits and constraints. Accordingly, scientific modeling through the use of computer simulation programs provides volumes of data that can then be made available for subsequent scrutiny. Traditionally, the results of scientific modeling are depicted in large tables and graphs which exhibit numerical relationships. It is often difficult to fully comprehend these tables and graphs, making the expert’s opinion testimony relating to his or her "mathematical tests" nearly impossible for lay witnesses to comprehend. Computer simulation allows the forensic expert to use computers to demonstrate the meaning of the mathematical studies.

It is important to note that computer simulation is not "computer animation." Computer animation is not necessarily defined by scientific principals.

Animation is merely used to illustrate an expert’s testimony while simulations contain scientific or physical principals requiring validation. Animations do not draw conclusions; they attempt to recreate a scene or process. Thus, they are treated like demonstrative aides. Computer simulations are created by entering data into computer models which analyze the data and reach a conclusion. Because the simulations themselves draw conclusions, they may have independent evidentiary value. As these later computer simulations appear on the scene, the Courts will be faced with making the Daubert analysis.

Harris v. State, 13 P. 3rd 489, (Okla. Crim. Ct. App. 2000) (citations omitted).

Most importantly, computer-generated models and simulations are not merely illustrative or demonstrative tools. They can also form the ultimate basis or foundation for an expert’s opinions, since they apply scientific principles, data and methods to reformulate the way in which events actually occurred. Again, it is the mathematical modeling, along with scientific principles, that leads to the validity of the result from a computer simulation. See 2 McCormick On Evidence, 218 (6th Ed.) (2007).

Computer models can also be used to test hypotheses of experts as to what did or could happen concerning events related in litigation. The inputs contain variables as well as known information; the computer runs these inputs through formulae based on scientific principals. Based on the results of the model, depending on the variable inputs, the expert may be able to form a conclusion about his hypothesis and then will base his opinion upon these results. Typically, there is no graphic image created by this type of "data modeling" as the terms is used here. The foundation for such models parallels the foundation described above for simulations.

Id.; See generally State v. Serge, No. 01-CR-260, 2001 WL 34058294 (Pa. Commw. Ct. Sept. 14, 2001).

Computer simulation is a valid foundation for expert opinion testimony. In McCurdy v. Ford Motor Co., No. 1:04-CV-155 (WLS), 2006 WL 2793167 (M.D. Ga. Sept. 26, 2006), the trial court considered the Defendant’s Motion to Exclude Expert Testimony based on the notion that the expert did not test his opinion or theories. The Court in McCurdy rejected the Defendant’s motion and found that the expert witness had actually tested his theories in accordance with a scientific method, although the expert did not personally perform all of the testing involved. The court specifically recognized that the expert’s opinion withstood a Daubert challenge due to the usage, in part, of computer simulation, albeit the simulation was performed by another expert.

Dr. Flowers performed original test (sic) and research regarding the tensile strength of the different sized links. He found that the 10 mm link offers a significant improvement in resisting bending handling load stress. Further, Dr. Flowers offered generally accepted and recognized literature in the field to back up his analysis. The remainder of Dr. Flowers’ opinions are based upon his review of the literature, much of which was provided by Ford, computer simulations, and the result of the NHTSA investigation. At this point, there is no reason to doubt that his method or source material is unreliable or untestable.

Id. at *4.

Likewise, in Montgomery v. Mitsubishi Motors Corp., No. 04-3234, 2006 WL 1892719 (E.D. Pa. June 10, 2006), a trial court rejected a Motion to Exclude Expert Testimony due to the failure of the expert to properly test his hypothesis. In Montgomery, the Court recognized that certain tests were performed with the use of computer simulations. These tests were deemed part of the analysis that supported the admission of the expert’s testimony. "In his capacity working through these companies, Mr. Tandy has conducted various tests, including ‘on track handling tests’ and ‘ADAMS computer simulations.’" Id. at *1. Accordingly, the court in Montgomery concluded that the expert would be allowed to testify, because his opinions were deemed more reliable due to the use of computer simulation tests. Today, courts are willing to accept computer simulation tests as the foundation for the admission of expert opinion testimony at trial, per the dictates of Daubert. See e.g. Appelera Corp. v. Micromass UK, Ltd., 204 F.Supp. 2d 724 (D. Del. 2002); and Livingston v. Isuzu Motors, Ltd., 910 F.Supp 1473 (D. Mont. 1995); Cf. Lyons v. J.A. Augur, Inc., 821 So.2d 536 (La. Ct. App. 2002) (computer simulation tests were held admissible, but not based on the Daubert).

III. "Real-Life" Examples

Forensic engineers should use science, i.e., mathematics and engineering, to assist in the explanation of the evidence and the determination of facts through the litigation process. Forensic engineers are trained problem solvers, who specialize in determining the cause and/or origin of technically difficult subjects such as accident reconstruction, product failure, and work-related injury. Sometimes "common sense" and "logic" tempt forensic engineers into believing that no systematic procedure is needed to form a valid opinion, and that general knowledge and experience can provide a sufficient foundation for a conclusion. However, since Daubert, expert opinion testimony, in order to be admissible, must also pass muster per "the scientific method."

The scientific method is a process by which the forensic engineer seeks to construct an accurate and consistent explanation for a loss, while minimizing the influences of bias and prejudice. The scientific method consists of four steps: (1) Data collection through observation and description; (2) Formulation of a hypothesis; (3) Testing of a hypothesis in order to predict quantitatively the results of new observations, in accord with scientific principles; and (4) Performance of experimental tests in order to verify, refine or reject a hypothesis, as well as determine the sensitivity of the hypothesis’ assumptions and tolerances.

Computer modeling and simulation is a means by which such matters can be represented on a systematic basis. When utilized as a tool for forensic engineering, computer modeling is mostly a group of mathematical relationships or formulae governed by the laws of physics and the engineering sciences. These models can then be mathematically solved, yielding results which can also be compared to the actual circumstances, i.e., data. By using scientific modeling techniques, computer simulations can effectively "test" the validity, or invalidity, of an expert’s opinions. The following computer modeling and simulation examples are simplifications of actual events, but were used to test expert opinion testimony.

Roof Collapse

A building is constructed in a location requiring roof design and storm drainage capable of safely handling a rainfall rate of 4 ½ inches per hour. During a rain of 2 inches per hour for 2 ½ hours, a roof portion collapses in the area surrounding a main, bowl-type roof drain, resulting in substantial losses. Forensic investigations include analysis of the building and roof structure, storm drainage system, design plans, actual construction, and cause of failure. Structural engineers indicate the cause of the roof collapse is a progressive failure due to repeated high loading of additional live load (water weight during rain water draining of the roof).

During the investigation, it is observed that the EPDM roof membrane within the mounting ring of the main drain had a hole cut within the membrane that is smaller than proper installation instructions indicate. As a result, the flow area of the membrane opening is approximately half of the proper installation per the roof drain’s instructions. Expert opinions indicate the smaller hole size of the membrane within the main drain is a proximate cause of the roof failure. Modeling and simulation methods provide a means to quantitatively test the effect of the reduced membrane hole size, as well as the performance of the drain system on the live roof load during rain events.

A model of the roof drain system and the EPDM membrane within the mounting ring (partially shown in Figure 1) is used for the membrane deflection analysis. The behavior of the EPDM within the drain ring is mathematically represented. The analysis results are shown by the color coded displacement and deformed shape of the membrane due to water loading during drainage. The non-deformed position and size of the membrane hole is shown by the asterisk label, indicating the original membrane hole edge. The analytical membrane model shown is then used to predict the membrane response to water bag loading during an actual physical test. In this way, the membrane model is tested and validated, too. (See Figure 1).

The roof drain and EPDM model are "connected" to the drain system piping model in order to represent the performance of the entire drain system. The rain profile is defined by the intensity (rate of fall) and duration of the rain falling on the roof’s surface. The rain profile information is obtained by weather radar data for the subject building’s location. The accumulation of water on the roof in the area by the drain (the collapsed area) is influenced by the rain rate, the roof slope geometry, and the rate of water removal by the drain system performance. The drain system performance, in turn, is dependent on the depth of water and corresponding pressure of the water flowing into the drain opening. The time varying system is also modeled by "time-domain" simulation, where the governing equations for the entire roof drainage system are repeatedly solved over small increments of time. In this way, the overall behavior of the roof drainage system can be quantified over time. The effect of the size of the actual (smaller) EPDM membrane hole compared to the "proper" (larger) hole is consequently quantified in order to test the experts’ opinions.

The functional capacity difference of the smaller and larger membrane holes is shown by subjecting the roof to a theoretical rain profile of 2 inches/hr over 4 hours. As can be seen by a plot, the water depth in the drain area reaches a maximum depth of 1.5 inches when the drain system contains the larger membrane hole. In comparison, the actual smaller hole restricts the drain flow rate; and the depth of water continues to accumulate, resulting in a 90% increase in live load on the roof. By providing actual rain profile data for the building location, the time domain simulation is employed to quantify the additional load on the roof under simulations of "real life" circumstances. Through the use of computer modeling and simulation methods, the opinions of the experts can be tested, and the sensitivity to assumptions (tolerances) established. (See Figure 2).

Asphalt Kettle Explosion

While heating asphalt in a dual burner kettle, an explosion occurs within the kettle, forcing the lid open, and expelling asphalt onto the kettle operator (kettleman). The kettleman is severely injured. At the time of the explosion, the kettle is being operated manually, using only one burner due to a broken handle, while also relying on the integral temperature sensor and indicator on the kettle. Forensic investigation determine the conditions of operation of the kettle prior to the incident and the observable data following the explosion. The experts’ opinions widely varied as to the mechanism and cause of the explosion; the contributory influence of the kettleman’s operation and use; and the inherent dangers associated with the kettle design and performance. Based on the evidence, one expert opines that the cause of the explosion is auto-ignition of volatile gases within the kettle due to over heating of the asphalt. The expert additionally states that the product design is defective and unreasonably dangerous due to an inaccurate representation of the actual asphalt temperature by the kettle’s thermal well and temperature indicator, thereby causing the asphalt to be overheated.

Using the design and manufacturing drawings of the kettle manufacturer, a computer model of the main portion of the kettle is "constructed." The kettle and melted, liquid asphalt are modeled using finite elements. This is shown in Figure 3.

Through the use of a computer model and thermal analysis, the components of the kettle, along with the asphalt temperature throughout the tank, are determined under a variety of operating conditions. In addition, the temperature at the thermal well, and therefore the indicated sensor temperature, is determined. The effect of the kettleman’s use and the resulting operating conditions are then tested within the safety of this "mathematical laboratory," in order to evaluate the expert hypotheses.

The model analysis and simulated conditions of operation yield the temperatures within the kettle. In Figure 4, the structure of the kettle components have been removed, and the asphalt has been sectioned to show the temperatures at a specific depth within the liquid asphalt. When the model is analyzed under substantially similar operating conditions as prior to the "real life" explosion, it is shown that the asphalt temperature exceeds the level required to produce volatile gases capable of combustion. The model shows the highlighted flue portion (above the top level of the liquid asphalt within the tank and below the metal top panel around the burner well) reaches the auto-ignition temperature for the asphalt vapor/air mixture. A significant result of the analysis and testing of the hypotheses indicates that the asphalt temperature indicated by the thermal well is significantly less than the maximum temperature within the asphalt when the pump is not circulating the asphalt within the tank. (See Figure 4).

The computer modeling and analysis, under varied simulated conditions, provided both a means to theoretically test opinions and guide physical tests of the kettle under substantially similar conditions, without producing an explosion. As shown in Figure 5, the actual kettle involved in the incident is tested under prescribed conditions. The left view is a thermal image of the kettle, which can read the temperature at every pixel within the image. The right view shows the temperature control knob set at 450ºF, the temperature as indicated by the thermal well as 475ºF, and the measured temperature within the liquid asphalt as 588ºF (above the Flash Point of the asphalt). The test was run under highly controlled conditions with the hydrocarbon and air mixture above the asphalt within the tank monitored continuously to avoid recreating the explosion. (See Figure 5).

The simulated conditions and model predictions were further verified by actual testing. As shown in Figure 6, thermal images of the flues are compared to the simulated operating conditions by the superimposed model results of the kettle. Both the physical testing of the actual asphalt kettle in operation and the computer’s model analysis yielded consistent results. Based on the validated model, the effect of alternative conditions of operation and kettle design are quantitatively assessed to ultimately test the expert’s opinion testimony. (See Figure 6).

Automobile Accident Reconstruction

A white Ford Focus turned left at an intersection and was struck by a dark blue Hyundai Triburon, fatally injuring both the driver and front seat occupants of the Ford. The posted speed limit for the roadways in all directions is 45 mph. Forensic investigation and accident reconstruction methods are used to determine the vehicles’ speeds prior to the collision. Through the systematic observation of data and combined application of damaged-based and trajectory-based reconstruction techniques, a vehicle collision is analyzed. Much of the initial data used in reconstruction is the collision evidence. Collision evidence includes tire marks, impact marking, roadway markings, damage to the vehicles, debris locations, positions of rest, damage to property and the environment, among others. As shown in Figure 7, scene photographs are used to independently map (separate from the homicide investigation) the scene and identify collision evidence. The evidence analysis is employed to determine the point of impact and post-collision paths of each vehicle as depicted. (See Figure 7.)

Computer modeling and simulations systematically apply engineering sciences, principles of physics, vehicle properties, and test data. The conservation or momentum serves as the theoretical basis for reconstruction of impact speeds in vehicle collisions. (It should be noted that the magnitude of external forces produced by the tires, as well as gouging and scraping of vehicle components on the ground during the collision, are normally small when compared to the magnitude of the forces of the collision; however, they should not be ignored depending on the evidence.) Energy methods are used to analyze the dissipation of kinetic energy. Vehicle velocities, changes in velocity (D V), rates of rotation, and other meters of the dynamics of the vehicles are calculated. Through accident reconstruction simulation, the governing equations are repeatedly solved for small time increments, in order to provide the data and visual representation of the analysis results. (Several images of the accident simulation are shown in Figure 8.)

Simulation modeling and analysis is a powerful tool in accident reconstruction to test and verify the opinions of experts, as well as consider the affects of alternative conditions or operator responses. In this example, the Hyundai was traveling 80 mph prior to the initiation of the skid. At the time the driver of the Ford initiated the left turn, the Hyundai was 340 feet away from the intersection. This computer simulation completely supported the accuracy of the witness testimony. The driver of the Hyundai was later arrested for two counts of Vehicular Homicide and Culpable Negligence.

IV. Conclusion

"Testing pursuant to the scientific method" is a critical part of any Daubert analysis. After all, it was the first "factor" referenced by the Supreme Court in Daubert. Although no single "factor" has been deemed more important than another, those involved in the litigation process know that it is often the inability to reliably "test" an expert’s opinion pursuant to the "scientific method" that ultimately leads to Daubert challenges. Through the proper usage of scientific modeling, computer simulations can be created to serve as a "modern mathematical laboratory," which can determine the validity, or invalidity, of expert opinion testimony. By properly using computer modeling and simulation, forensic experts and litigators can build the necessary scientific and legal foundation for the admission (or exclusion) of expert opinion testimony at trial, per the dictates of Daubert and its progeny.

About the Authors

Scott S. Katz, Esq. is a Partner with the law firm of Butler Pappas Weihmuller Katz Craig LLP

Elliot L. Stern, Ph.D., P.E. is an Engineer with Florida Forensic Engineering, Inc.