Samuel v. Ford Motor Co.

96 F. Supp. 2d 491, 54 Fed. R. Serv. 725, 2000 U.S. Dist. LEXIS 9491, 2000 WL 555206

• Court: District Court, D. Maryland
• Decided: April 28, 2000
• Docket: CIV. A. WMN-96-2155
• Status: Published

Opinion

MEMORANDUM AND ORDER

GRIMM, United States Magistrate Judge.

Pending is Defendant Ford Motor Company’s motion in limine (“MIL”) no. 6, which has been fully briefed. (Paper nos. 210, 245, and 259.) In this motion, the Defendant seeks to prevent one of the Plaintiffs’ expert witnesses, Michael A. Kaplan, Ph.D.,' from expressing any opinions at trial based upon the Mechanical Systems Analysis Inc. (“MSAI”) Accident Avoidance Maneuver (“AM”) Test. Dr. Kaplan has used this test to assist him in determining whether automobiles have an unreasonably dangerous propensity to roll over. 'Dr. Kaplan has testified in numerous cases that certain automobiles are defective based, in part, on the results of the MSAI test. Defendant Ford Motor Company raises Daubert/Kumho 1 challenges to the test, and asks the Court to rule that Dr. Kaplan may not offer any opinions in this case based on it. When the motion was filed, I instructed the Defendant to complete a Daubert/Kumho worksheet, (attachment A to this Memorandum), to facilitate review of this motion. Ford complied, and filed & completed worksheet, along with three notebooks of exhibits and seven videotapes. (Paper No. 277.) A hearing was conducted on April 14, 2000, during which testimony was taken from Dr. Kaplan and Ford’s expert, Mr. Lee Carr, and Plaintiffs also submitted a binder of documents. (Paper No. 288.) Following the hearing, I requested that the parties provide me with additional materials. (Paper Nos. 282-86, 289-95, and 298-99.) Having reviewed all these materials, I have reached the conclusion that the motion must be granted, for the reasons explained below.

A BACKGROUND

The essence of the Plaintiffs’ claims is that the 1993 Ford Aerostar van is defective, and unreasonably dangerous, because it has a propensity to roll over. In this ease, the Aerostar in which the Plaintiffs were riding rolled over after it was struck by Defendant Hamilton’s Ford Econoline van, resulting in the death of Mr. Michael Samuel’s wife, Alicia, and their infant child, as well as serious injuries to the Bergers, who also were occupants.

The gravamen of Ford’s motion is that the MSAI accident avoidance test used by Dr. Kaplan to evaluate the roll over propensity of the 1993 Aerostar is unreliable because it lacks an objective protocol for implementation, and does not produce results that can be duplicated when it is performed successively under the same or substantially identical conditions. In addition, Ford asserts that it is similar to another test, the Consumers Union (“CU”) Short Course, that has been criticized in peer-review literature, and which has not been accepted within the automotive industry. Further, Ford argues that the MSAI AM test subjects the tested vehicle to conditions which greatly exceed what reasonably can be expected when a vehicle is driven under real world emergency conditions.

B. LAW

As a result of the Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469, and Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 119 S.Ct. 1167, 143 L.Ed.2d 238, decisions, federal trial courts are required, pursuant to Rule 104(a) of the Federal Rules of Evidence, to screen scientific or technical evidence before it may be considered by the jury, to insure that it is both reliable, and relevant. Otherwise it is not “helpful”, as required by Rule 702 of the Federal Rules of Evidence. When reviewing scientific or technical evidence, the trial court should consider a number of non-exclusive factors including: whether the challenged expert’s opinions are based on methodology that can be tested; whether there is a known error rate, or approved technique, associated with the method of testing; whether the method of testing has been subjected to peer review; and whether it has gained general acceptance within the relevant scientific or technical community. See Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786; see also United States v. Dorsey, 45 F.3d 809, 813 (4th Cir.1995) (noting that the Supreme Court suggested four factors that courts may consider when determining admissibility of scientific evidence from an expert). As noted, these factors are illustrative, not exclusive, and other factors, such as whether the method of testing at issue has a legitimate use outside of litigation, also have been adopted by courts. See Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1316 (9th Cir.1995) (“One very significant fact to be considered is whether the experts are proposing to testify about matters growing naturally and directly out of research they have conducted independent of the litigation or whether they have developed their opinions expressly for purposes of testifying.”); see e.g., Maryland Casualty Co. v. Therm-O-Disc, Inc., 137 F.3d 780, 784-85 (4th Cir.1998) (holding that the position of the 4th Circuit is that the factors suggested in Daubert were not meant to be an exclusive or rigid test). In making its preliminary determination under Rule 104(a) of the Federal Rules of Evidence, the court must apply a preponderance of the evidence standard. See Daubert, 509 U.S. at 592 n. 10, 113 S.Ct. 2786. However, the rules of evidence, except for privilege, are not strictly applied during the inquiry. Fed.R.Evid. 104(a), 1101(D)(1).

In cases such as the present one, where the methodology used by an expert to reach his or her opinions is challenged, it is helpful to recall the Supreme Court’s discussion in Daubert regarding the difference between “reliability” and “validity” as these concepts affect scientific reasoning. A test is “rehable” if it is capable of being duplicated, and produces the same or substantially same results when successively performed under the same conditions. See Daubert, 509 U.S. at 595, 113 S.Ct. 2786. A test is “valid” if it has a logical nexus with the issue to be determined in a case. See id. at 591, 113 S.Ct. 2786. In this case, Ford asserts that the accident avoidance testing used by Dr. Kaplan is neither reliable, nor valid.

C. DISCUSSION

1. The CU Short Course and MSAI Accident Avoidance Test

Although the MSAI AM test has not been the subject of independent review, it shares many features with the CU Short Course, which has been the subject of critical review. In the CU Short Course, the subject vehicle is fitted with “outriggers” to prevent it from tipping over, then driven down the right lane of a two lane test track, through a 70 foot lane of pylons which separate the two lanes of the track. As the vehicle leaves the pylons, it must then maneuver from the right" lane to the left lane and avoid hitting an obstacle (a single pylon placed about 120 feet from the start of the course), then maneuver back into the right lane, and finish the test run by driving through an exit lane of pylons, again located in' the right lane. The test is run at various speeds, and the object is to see if the vehicle can swerve from the right lane into the left, avoid hitting the obstacle, then maneuver back into the right lane, without tipping up onto the outriggers. Typically, when the test is run, instruments placed on the test vehicle record its speed, as well as the angle and speed of the left-right turns of the steering wheel. The test usually is videotaped, and the vehicle is driven by a professional driver. A diagram of the CU short course is found at attachment B to this memorandum.

The MSAI avoidance maneuver test, developed in the early 1990’s by Dr. Kaplan and a professional driver by the name of Robert Hooker, is quite similar to the CU Short Course, "in that a test vehicle is driven by a professional driver down the right lane of a two lane test track, then must swerve from the right lane into the left lane to avoid a barrier in the right lane made of pylons, then maneuver back into the right lane. In Dr. Kaplan’s MSAI test, the entry into, and exit from, the right lane are not “channeled” by pyions. Test vehicles are fitted with outriggers, and instrumented as in the CU Short Course. A diagram of the MSAI test is found at attachment B to this memorandum. During his testimony, Dr. Kaplan acknowledged the similarity of his MSAI test to the CU Short Course.

As Dr. Kaplan testified during the hearing, a distinctive feature of his test is that the professional driver intentionally steers the vehicle in such a way as to produce sufficient lateral acceleration forces on the tires to push the vehicle to its handling limits, to see if it will tip over. If it does, Dr. Kaplan concludes that the vehicle is defective because it has an unreasonably dangerous propensity to roll over. If it does not tip up during the test, it is not defective.

Mr. Carr, Ford’s expert, testified that neither the CU Short course nor the MSAI accident avoidance test are rehable or valid because they may be manipulated by the professional test driver to intentionally create steering loads on the test vehicle’s tires in order to push it beyond the limits of its handling capabilities. This exposes the vehicle to forces which will not occur in “real-world” accident avoidance maneuvers. Dr. Kaplan responded that to perform accident avoidance testing on a vehicle without purposely pushing it to see if it rolls over when subjected to forces beyond its limits fails to test for adequate roll over resistance at all. In support of this, Plain tiffs introduced as exhibits design documents from Ford that state that its light trucks are designed to be safe, even in response to “limits handling situations.” (Pis.’ Supp’l Materials for Daubert Hearing Exs. 3 and 4, Paper No. 288.)

Thus, there is a fundamental difference in approach between Mr. Carr, and Dr. Kaplan. Carr, a former employee of Ford Motor Company, and thereafter a consultant to the automotive industry, testified that the proper purpose of roll over resistance testing is to expose a vehicle to forces and conditions that are more extreme than a consumer driver reasonably can be expected to encounter in real life, and if the vehicle does not roll over, it is reasonably safe. The goal of such testing, therefore, is not to expose the vehicle to increasingly severe forces to determine when it will roll over^ but instead to create conditions which simulate what consumer drivers will have to react to during accident avoidance maneuvering, and if the vehicle does not roll over, it is not defective. According to Carr, roll over “immunity” is impossible to achieve, but reasonable roll over resistance can be.

Dr. Kaplan, a frequent expert witness on behalf of plaintiffs in roll over litigation, believes that roll over testing such as described by Mr. Carr is inadequate because it does not have the goal of pushing the test vehicle to the point where its driving limits are overcome. According to Dr. Kaplan, only when the vehicle is pushed to this point, and either rolls over, or does not, can it be determined whether it is defective. To Dr. Kaplan, the fact that such testing may require steering inputs that greatly exceed those that can be expected from non-professional drivers is irrelevant. The goal is to expose the vehicle to forces sufficient to exceed its handling limits. While this may be an appropriate objective 'if the purpose is to determine what it will take to make a particular vehicle roll over, it does not mean that the test will be a valid one for determining whether a vehicle is defective, or unreasonably dangerous, for its intended purpose— consumer transportation.

Both Mr. Carr and Dr. Kaplan are experienced and articulate, but they clearly are advocates for their positions, and their advocacy has been polished and perfected through another rigorous test procedure— repeated testimony in contested cases, where Mr. Carr has taken the side of the auto manufacturer, and Dr. Kaplan that of the plaintiffs. While-their testimony was helpful, their views are not without bias, and to determine which of their positions regarding the MSAI test has greater merit requires the Court to address the peer review information submitted by the parties as exhibits during and after the hearing. In this regard, because of the similarities between the CU Short Course and the MSAI test used by Dr. Kaplan, analysis of the methodology of the CU course is relevant to evaluating the MSAI test.

2. National Highway Transportation Safety Administration And Related Materials

The National Highway Traffic Safety Administration (“NHTSA”) is an agency of the Department of Transportation. Its purpose is to “reduce traffic accidents and death and injuries resulting from traffic accidents.” 49 U.S.C. § 30101. On at least two occasions, in 1988 and 1997, NHTSA considered the CU Short Course, and the defects in its methodology which Ford asserts also renders Dr. Kaplan’s MSAI test inadmissible. Ford submitted copies of NHTSA’s reports as exhibits-during the hearing. (Def.’s Supp’l Materials for Daubert Hearing Exs. 9 and 10, Paper No. 277.) .

On September 8, 1988, NHTSA published in the Federal Register the reasons for its denial of a petition submitted by the Center for Auto Safety requesting that NHTSA conduct a defect investigation into the Suzuki Samurai because of its alleged roll over propensity. (Def.’s Supp’l Materials for Daubert Hearing Ex. 9, Paper No. 277.) As part of its investigation, NHTSA considered testing done on the Samurai by NBC News, as well as Consumers Union. The Agency criticized this testing as follows:

The existing test procedures for assessing the roll over propensity of vehicles are unsatisfactory because they do not provide for repeatable, reproducible results, and there are no accepted performance criteria. The testing appears to rely on the skill and influence of the driver and the presumption that the vehicle suspension, tire, and road surface characteristics will remain constant throughout the testing. At present there is no standard, accepted test or series of test procedures and performance requirements which accurately predict a vehicle’s roll over propensity....

The Consumer Union and the NBC News testing results are cause for some concern, but the test procedures do not have a scientific basis and cannot be linked to real-world crash avoidance needs, or actual crash data. Using the same procedures, probably any light utility vehicle could be made to roll over under the right conditions and driver input.

(Id., Ex. 9 at 34,967.)

In June 1997, NHTSA’s Office of Defects Investigation published its report denying petitions which had been filed by Consumers Union requesting defect investigations concerning the roll over propensity of two vehicles, the Isuzu Trooper, and the Acura SLX. (Def.’s Supp’l Materials for Daubert Hearing Ex. 10, Paper No. 277.) The petitions filed by Consumers Union were based, in part, on testing done on the vehicles using the CU Short Course. The NHTSA report noted that the CU Short Course:

Requires the vehicle to be steered quickly, and since these quick steering maneuvers generate angular accelerations ... that are proportional to the mass moments of inertia, changes in a vehicle’s mass moments of inertia could have a significant effect on the vehicle’s performance on the course.

(Id., Ex. 10 at 5.) NHTSA also analyzed the relationship between the CU Short Course and real world crash avoidance behavior. It noted that a driver coming upon an unexpected obstacle “will naturally try to avoid striking that obstacle. Such a driver generally will try to stop and/or turn.” (Id., Ex. 10 at 12.) NHTSA observed that the professional drivers going through the CU Short Course turn the steering wheels of the vehicles “hundreds of degrees at a very fast rate, exceeding 1,000 degrees/sec. and often exceeding 1,300 degrees/sec.” (Id.) In contrast, however, NHTSA referenced several studies done on driver behavior during crash maneuvers. Those studies showed that average maximum steering rates for all successful drivers were “about 520 degrees/sec.”, and that when these drivers steered at rates in excess of 850 degrees/sec. they lost control of the vehicle. (Id., Ex. 10 at 14-15.) NHTSA concluded:

These three studies suggest that the maximum steering input a [non-professional] driver is willing or able to make in response to an unexpected event are within the range of 145 to 230 degrees .... [Djuring the tests conducted by CU drivers on the CU short course in which the subject vehicles tipped up, the initial (left) steering inputs were usually within this range (with one exception), but the second (right) steer was much greater. These studies also indicate that the rate a typical driver turns the wheel is not greater than 1000 degrees/sec. 2 ... In tests on the CU short Course conducted by NHTSA at speeds above 36 mph, the steering rate was generally in the range of 900 to 1,350 degrees/sec.

(Id., Ex. 10 at 15.) 3 Reviewing the test data from the CU Short Course runs, NHTSA noted that when a CU driver performed the test with total steering inputs of less than 400 degrees (the sum of the left and right turn steering angles)— which is within the range a non-professional driver could be expected to steer when confronted by an unexpected obstacle — the vehicle did not tip. However, when the total steering inputs exceeded 500 degrees, the vehicle tipped. (Id., Ex. 10 at 17.)

NHTSA correlated the data from the CU tests, its own testing, and that performed by Isuzu and others, and concluded:

These data indicate that while testing at the same nominal speed, dramatic differences in the outcome can be generated by the test driver. A slight increase in the left steer with similar steering rates, followed by a substantial increase in the right steer at a much higher rate pro-dueed tip-up, while the vehicle could go through the course successfully with less severe steering inputs.

(Id., Ex. 10 at 22.) Further, NHTSA concluded, a test driver purposely can cause a roll over: “The methodology used to intentionally tip the vehicle (characterized by the test driver as ‘whipping’ the steering wheel) was to increase the steer angle and steer rate, especially for the reverse steer. The initial steer was also slightly delayed.” (Id., Ex. 10 at 22.)

As a result of its investigation, NHTSA issued findings which included:

ODI’s 4 review of tests performed by CU drivers on the CU short course demonstrates that the test can be and has been conducted with a variety of strategies, including timing of turns, severity of steering inputs, and rate of steering inputs, which can lead to different results under similar initial conditions.

And:

Data developed during ODI’s testing and its review of the CU tests, demonstrates that the CU short course test, as conducted by CU, does not provide a sufficient scientific basis on which to determine the existence of a safety-related defect.

(Id.)

As a result of the deficiencies in the CU testing, and its review of the other test data, NHTSA declined to initiate a defect investigation into the Isuzu Trooper and Acura SLX vehicles. (Id., Ex. 10 at 33.)

In addition to the NHTSA materials, Ford submitted as an exhibit a 1991 report by Great Britain’s Department of Transport, Transport and Road Research Laboratory, titled “Handling tests on four wheel drive multi-purpose vehicles” which was similarly critical of the CU Short Course test:

“An important criticism of the modified Consumers Union test is that it is very driver dependent. There is considerable opportunity for the driver to steer through the manoeuver using different techniques leading to different results.”

(Def.’s Supp’l Materials for Daubert Hearing Ex. 11 at 13, Paper No. 277.)

Ford also submitted as an exhibit the deposition testimony of Mr. Robert Hooker, the co-developer of the MSAI accident avoidance test, who acknowledged that the MSAI test is even more driver dependent than the CU Short Course, because in the CU test the specific course is laid out (by the entry and exit lanes), while in the MSAI test there are no such lanes. (Def.’s Reply Mot. No. 6 Ex. 1 at 33-34, Paper No. 259.)

3. Reliability of MSAI Test Performed on the Aerostar

The MSAI test performed by Dr. Kap-lan on the Ford Aerostar subjected it to the same level of extreme steering inputs that NHTSA criticized in connection with the CU Short Course.

Following the hearing on April 14, the Court requested that Plaintiffs’ counsel provide it with the data from the MSAI tests performed by Dr. Kaplan in this case. During the pretrial conference on April 17, 2000, Plaintiffs’ counsel provided the Court with a chart, (attachment C to this Memorandum), which contained the requested data. 5 According to the explanation of the chart provided by Plaintiffs’ counsel during the pretrial conference, the eight test runs labeled “EAM” 6 were the MSAI tests performed on the Aerostar. Counsel explained that the first entry for each run under the heading “steer angle (degrees)” represents the angle, in degrees, of the initial, or left, steer of the accident avoidance maneuver. The block immediately below the first one represents the angle of the second (right) turn, also in degrees. The chart also contains a heading titled “Avg. Steer Rate (deg./sec)”, which represents the average speed of the turns (degrees per second). The steering data relevant to this motion may be summarized in the following chart:

TOTAL

STEER STEER AVERAGE STEER

RUN # SPEED INPUTS ANGLE RATE_RESULT

ab 158 25 mph L 865 deg. 764 deg. L 582 deg/sec no tip

_R 399 deg._R 598 deg/sec_up_

ab 159 30 mph L 337 deg. 707 deg. L 522 deg/see no tip

_R 370 deg._ R 777 deg/see_up _

ab 160 35 mph L 357 deg. 694 deg. L 590 deg/see no tip

_R 337 deg._R 578 deg/see_up_

ab 161 40 mph L 370 deg. 746 deg. L 726 deg/see tip up

R 376 deg. R 812 deg/see on R

__;_steer_

ab 162 45 mph L 297 deg. 657 deg. L 495 deg/see tip up

R 360 deg. R 720 deg/see on R

__steer_

ab 169 45 mph L 328 deg. 668 deg. L 675 deg/gec tip up

R 340 deg. R 704 deg/see ' on R

_steer_

ab 170 40 mph L 369 deg. 738 deg. L 663- deg/see tip up

R 369 deg. R 766 deg/see on R

_^_steer_

ab 171 35 mph L 406 deg. 753 deg. L 679 deg/see no tip

R 347 deg. R 715 deg/see_up

As can be seen, all of the individual steering inputs exceeded the “normal range” of 145-230 degrees, as identified by NHTSA. (Def.’s Supp’l Materials for Dau-bert Hearing Ex. 10 at 15, Paper No. 277.) Further, in each of the four instances where the Aerostar tipped up during the second (right) steer, the average steering rate exceeded 700 degrees/second, well above what NHTSA has determined is the average maximum steering rate for all successful drivers, as shown by several studies of driver behavior reported in the 1997 report regarding the Trooper and Acura SLX. (Id., Ex. 10 at 14.) This data, therefore, shows that the MSAI tests performed by Dr. Kaplan on the Ford Aerostar were characterized by steering inputs and rates similar to those criticized by NHTSA with respect to the CU Short Course. It is this aspect of the MSAI test which renders it unreliable, invalid, and, as will be discussed in greater detail below, inadmissible.

4. Plaintiffs’ Arguments In Support of General Acceptance

In their supplemental memorandum regarding general acceptance, (paper no. 282), Plaintiffs attempt to address the defects in the MSAI testing methodology which are subject to the same criticisms discussed by NHTSA in.its 1997 study of the' Isuzu Trooper. Plaintiffs argue that there are many different types of accident avoidance maneuver (“AM”) tests used in the automotive industry, 7 and that Ford itself has stated that its design objectives for its light trucks include making them safe during the most extreme accident avoidance situations. (Pis. Supp’l Materials for Daubert Hearing Exs. 3 and 4, Paper No. 288.) Further, Plaintiffs cite to deposition testimony of former Ford engineers, taken in other roll over litigation, 8 as well as Ford’s own accident avoidance test, the P6-101 test 9 as evidence that AM testing has gained general acceptance within the automotive industry. This argument misses the mark. The scope of Ford’s Motion No. 6 is quite narrow. It seeks only to prevent Dr. Kaplan from testifying about the results of his MSAI testing performed on the Aerostar van. It does not seek to preclude him from providing otherwise admissible testimony about other AM testing he did on the Aerostar, such as the “J-Turn” test. Moreover, the fact that the automobile industry itself may use various forms of AM testing, for a variety of reasons, does not mean that the specific format of Dr. Kap-lan’s MSAI test — the essential feature of which is to produce lateral acceleration forces on the test vehicle’s tires in order to reach its limits of handling — has achieved general acceptance, or that its methodology is scientifically reliable and valid. Further, the fact that the Plaintiffs have produced selective portions of deposition testimony from former Ford engineers in unrelated litigation, the content and context of which is not possible to discern from the excerpts provided, does not establish that Dr. Kaplan’s MSAI test has achieved general acceptance. Finally, while general acceptance of a test method certainly is a relevant factor under the Daubert/Kumho analysis, it is not alone dispositive. Polygraph examinations widely are accepted and used in employment, law enforcement and security contexts, yet this fact does not make them admissible as evidence in trials. To the contrary, despite their widespread acceptance and use, they are not admissible because they have not been determined to be reliable. The fact that an entire industry may use a test of insufficient reliability does not make it admissible into evidence.

Plaintiffs also argue in their supplemental memorandum that the steering angles and rates used by Dr. Kaplan in his MSAI test are consistent with those used in other AM testing. (Pls.’ Supp’l Memo. Regarding General Acceptance at 7-10, Paper No. 282.) However, they concede, as they must, that the steering angles used by Dr. Kaplan in his MSAI tests of the Aerostar van were “somewhat greater” than those reported in connection with other AM testing. 10 (Pis. Supp’l Materials for Daubert Hearing Ex. 14 at fig. 6, Paper No. 288.) In fact, as addressed in the preceding footnote, see supra n. 10, Dr. Kaplan’s steer angles in some instances were more than double the angles reported in the study the Plaintiffs cite and, as already discussed, within the exact range that NHTSA concluded was excessive. Additionally, Plaintiffs argue that the steering angles used by Dr. Kaplan in his MSAI testing on the Aerostar ar,e comparable to those used in other AM testing.. (Pis.’ Supp’l Memo. Regarding General Acceptance at 8, Paper No. 282.) However, the tests cited by Plaintiffs, the “J-Turn” and “Fishhook” tests, although not criticized by NHTSA with regard to methodology, were not the subject of Ford’s Motion No. 6. The comparison is unpersuasive. The “J-Turn” test does not involve the left-right turn sequence that the MSAI test does, which can subject the vehicle' to the “whipping” phenomenon noted by NHTSA. The “Fishhook” test was not performed by Dr. Kaplan on the Aerostar, and is not at issue in this case.

Plaintiffs also argue that the steering rates (degrees/second) recorded by Dr. Kaplan in his Aerostar tests are consistent with those reported in other AM testing. (Pis.’ Supp’l Memo. Regarding- General Acceptance at 8-9,. Paper No. 282.) However, the studies cited by Plaintiffs do not undermine the correctness of the conclusions reached by NHTSA, and frequently reveal data consistent, with what NHTSA reported.

Far more telling, however, are the data reported in a 1999 study also cited by the Plaintiffs. This study, described in some detail in footnote 3, see .supra n. 3, .measured .steering angles and rates for nonprofessional drivers .forced to employ accident avoidance maneuvers when confronted with unexpected, realistic crash-imminent situations. (Letter from Pis. to Court of 4/24/00 Ex. 2, Paper No. 298.) The study measured average avoidance steering angles at 53 degrees (dry pavement test), and 74 degrees (wet pavement test). (Id., Ex. 2 at 165.) The highest steering angles ranged from 271 degrees (dry pavement) to 289 degrees (wet pavement). (Id.) The average maximum steering rates for the emergency avoidance maneuvers was 262 degrees/second (dry pavement), and 294 degrees/second (wet pavement). (Id.) While the highest rates recorded were 1159 degrees/second (dry pavement) and 1335 degrees/second (wet pavement), ninety-five percent of the steering rates observed were less than 643 degrees/second. (Id.) These data are closely in line with the steering angles and rates reported for accident avoidance maneuvers by NHTSA in 1997. The fact that they were observed in “real world” drivers undertaking emergency avoidance maneuvers under perceived real emergency conditions is highly probative. The steering angles are substantially below those reported in Dr. Kaplan’s MSAI test, and the average maximum steering rates as well as ninety-five percent of the steering rates observed were substantially lower than the rates reported in Dr. Kaplan’s test when a professional driver experienced tip ups.

' Finally, Plaintiffs argue that any deficiencies in Dr. Kaplan’s testing may be brought out during cross examination, and that if Ford’s motion is granted, this Court also must preclude Ford from offering testimony about its P6-101 testing. These final arguments are quickly disposed of. First) the mere fact that cross examination may expose deficiencies in a test begs the issue of whether it is reliable and relevant in the first place, which is the condition precedent for its admissibility. If it were otherwise, then any test, no matter how unreliable, could be admitted, trusting in the opposing party’s ability to expose its flaws during cross. This was not the holding in Daubert or Kumho. Finally, regarding Ford’s P6-101 test, I have no proffer from Ford as to how it intends to use this test at trial, if at all. Absent such information, I cannot at this time rule on any challenge to its admissibility. This issue must, therefore, be reserved until trial. For all the above reasons, the Plaintiffs supplemental memorandum regarding general acceptance, (paper no. 282), does not mitigate the criticism of the methodology of the MSAI AM test as performed by Dr. Kaplan on the Aerostar.

D. ANALYSIS OF DAUBERT FACTORS

The first DaubeH factor is whether the challenged opinion can be tested. While it is true that the MSAI AM test, like the CU Short Course, has a general protocol for testing which describes how the test course is to be laid out, and the tests run, it is lacking in one critical area. There is no way to standardize and control for the steering angles and rates which, as noted by NHTSA, can dramatically influence the outcome of the test. Indeed, Dr. Kaplan has been quoted in a published opinion, Ford Motor Co., v. Ammerman, 705 N.E.2d 539, 557 (Ind.App.1999), as having “affirmed that his [MSAI] test had no protocol for steering inputs.” This deficiency also means that the MSAI test lacks reliability, in that it cannot be expected to produce the same or substantially similar results when run under the same conditions. In the words of NHTSA, the test is not reliable because it does not “provide for repeatable, reproducible results, and there are no accepted performance criteria.” (Def.’s Supp’l Materials for Daubert Hearing Ex. 9, Paper No. 277.)

Similarly, the MSAI test has not been shown to be valid because the steering angles and speeds used to produce a roll over have been shown to exceed the angles and rates of steering which reasonably can be expected from non-professional drivers performing emergency avoidance maneuvers. (Def.’s Supp’l Materials for Daubert Hearing Ex. 10 at 14-15, Paper No. 277; Letter from Pis. To Court of 4/24/00, Ex. 2, Paper No. 298.) Thus, if the MSAI test produces roll overs only when steering angles and rates are used which bear no relation to what is reasonably likely to occur in real world accident avoidance maneuvers, the test results are not “valid” because they do not measure what is at issue in this case. As the Daubert court stated, the “fit” between the test and the issues in this case is not a good one. Daubert, 509 U.S. at 591, 113 S.Ct. 2786 (noting that the consideration of “fitness” is “whether expert testimony proffered in the case is sufficiently tied to the facts of the case that it-will aid the jury in resolving a factual dispute”).

While Dr. Kaplan testified that exhibits 18-20 and 22 from Plaintiffs’ supplemental materials for the Daubert hearing, (paper no. 288), show that the results of his MSAI tests are “repeatable”, it is not possible from reviewing these charts to reach this conclusion, since key information regarding the steer angles and rates. are not contained in the charts.

The second Daubert factor is whether the methodology has been subjected to peer review. As already discussed above, the 1988 and 1997 NHTSA reports criticizing the methodology of the CU Short Course are objective — and therefore persuasive — evidence that the MSAI test similarly is flawed.

The third Daubert factor is whether there is a known error rate associated with the challenged methodology. In this regard, Mr. Carr testified that the error rate associated with the MSAI test can be as high as 100 percent, given the fact that the results of the test can so dramatically be influenced by the manner in which the test-driver steers the vehicle. This testimony was supported by the deposition testimony of Mr. Hooker, a co-developer of the MSAI test, who acknowledged the driver dependency of the test, and that it could be run at two different times on substantially identical vehicles, yet produce different results. (Def.’s Supp’l Materials for Daubert Hearing Ex. 2, Paper' No. 277, Sept. 19, 1996 deposition testimony of Robert Hooker at 373-377.) Although Dr. Kaplan testified that he disagreed that his test is driver dependent, this opinion is not credible in light of all the evidence presented during and after the hearing.

The final Daubert factor is whether the methodology has been generally accepted within the relevant scientific or technical community, and it has been extensively considered in the discussion above. It is sufficient to say in this regard, that the criticisms of NHTSA regarding the CU Short Course also show that the MSAI test, as conducted by Dr. Kaplan, has not achieved general acceptance within the automotive testing and safety community. Dr. Kaplan himself admitted during his testimony that with the exception of the U.S. Air Force, which, he asserts, hired Mr. Hooker to evaluate one of its vehicles, no government agency or automotive manufacturer uses the MSAI test to evaluate a vehicle for roll over resistance.

Finally-, courts also have considered whether a challenged test or methodology has a use independent of litigation as a factor to evaluate its admissibility. See Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1316 (9th Cir.1995) (“One very significant fact to be considered is whether the experts are proposing to testify about matters growing naturally and directly out of research they have conducted independent of the litigation or whether they have developed their opinions expressly for purposes of testifying.”); see also 4 WEINSTEIN’S FEDERAL EVIDENCE § 702.05[2] (2d ed,1997)(noting that whether or not there is a “litigation taint” in the research is a factor courts consider when making a Daubert determination). Mr. Carr testified that the MSAI test was created by Dr. Kaplan and Mr. Hooker in connection with their engagements as plaintiffs’ experts in roll over litigation, and that its use has been restricted to this purpose thereafter. In addition, Mr. Hooker has testified that the MSAI test was “born, in litigation”. (Letter from Def. to Court of 4/19/00, Paper No. 285.) While Dr., Kaplan does not agree with this characterization, there has been sufficient evidence that the MSAI test has been used principally in connection with litigation such that its usefulness in any other context is questionable.

Dr. Kaplan did testify that he has been permitted in approximately 7 other cases to offer opinion testimony based on the MSAI AM testing. The details of all of these cases have not been provided, but I found two reported decisions in which Dr. Kaplan’s opinions based on the MSAI test have been allowed. In the first, Livingston v. Isuzu Motors, 910 F.Supp. 1473 (D.Mont.1995), the court did not do a Daubert analysis, since the defendant challenged the evidence under rules 401, 402, and 403 of the Federal Rules of Evidence as not being sufficiently similar to the accident which was at issue in that case. Livingston, 910 F.Supp. at 1492-93. Accordingly, Livingston offers little help in analyzing a Daubert challenge to Dr. Kaplan’s test.

In the second, Ford Motor Company v. Ammerman, 705 N.E.2d 539 (Ind.App.1999), the state court did analyze the MSAI test under each of the Daubert factors, and concluded that it was not reversible error for the trial court to have admitted it. However, a careful review of the opinion shows that the court did not have before it the same evidence that has been submitted in connection with this case, especially the NHTSA materials. Thus, the opinion is devoid of any analysis of the steering inputs and rates used by Dr. Kap-lan during his tests. Thus, while the decision of the Indiana Court of Appeals is thorough and thoughtful, it clearly was decided on facts other than those present in this case.

Additionally, counsel for Ford brought to the Court’s attention the bench ruling from the trial court in Moses v. Ford Motor Co., Case No. C96-01651, (Cal. Sup. Court of Contra Costa County Oct. 20, 1998). (Letter from William Conroy to Court of 4/18/00 Ex. A, Paper No. 284.) The transcript of the Court’s ruling admitting Dr. Kaplan’s MSAI testing reflects that the issue of admissibility was governed by state law, not Rule 702 of the Federal Rules of Evidence or Daubert and Kumho. Accordingly, the ruling is of little help in this ease.

Finally, having viewed the videotapes of actual test runs using the MSAI test, submitted as exhibits 1, 3, 5, and 6 by Ford, I am convinced that the dramatic tip-ups portrayed by those video’s would have a highly prejudicial effect on the jury, which would exceed any probative value they would have. The image of the test vehicles tipping dramatically on the steer back into the first lane, prevented from rolling over only by the presence of the outriggers, would leave an indelible image in the minds of the jury which no limiting instruction could offset. Thus, under Rule 403 of the Federal Rules of Evidence, in addition to the Daubert analysis, I would not admit this evidence.

E. CONCLUSION

For the reasons stated, I have concluded that, as a result of the steering angles and rates used by Dr. Kaplan in the MSAI AM tests performed on a Ford Aerostar van in connection with this case, the results of his testing are not reliable, nor are they relevant. Therefore, they may not be admitted into evidence, nor may Dr. Kaplan base any opinion testimony at trial on this testing.

Attachment A

Daubert/Kumho Worksheet

1. Name of Expert Challenged.

2. Brief summary of opinion(s) challenged (if more than one, designate separately ), including reference to the source of the opinion (ie., Rule 26(a)(2)(B) disclosure, deposition transcript references, interrogatory answers). Attach highlighted copy of source materials as exhibit.

3. Briefly describe methodology/reasoning used by expert to reach each opinion which is challenged. Include reference to source of challenged methodology/reasoning, and attach a highlighted copy as an exhibit.

4. Briefly explain the basis for the challenge to the reasoning/methodology used by the expert (for example, methodology unreliable; methodology reliable, but not valid for application to this case; failure to use standardized or accepted methodology (for example, with a standardized test); etc.) Attach a highlighted copy of affidavit or other source material supporting challenge to methodology/reasoning as an exhibit.

5. Is the challenged methodology/reasoning subject to a known or potential error rate? If so, briefly describe it, and attach a highlighted copy of any relevant source material as an exhibit.

6. Summarize relevant peer review materials relating to methodology/reasoning challenged, and attach a highlighted copy of any relevant source material as an exhibit.

7. If the challenge to the opinion is based upon a contention that the methodology/reasoning has not been generally accepted within the relevant scientific or technical community, briefly explain the basis for this contention. Attach highlighted copy of any relevant supporting materials as an exhibit.

[[Image here]]

[[Image here]]

[[Image here]]

1 . Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993); Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999).

2 . The NHTSA report noted that "[r]eal drivers in everyday situations will not react as quickly as professional drivers on the CU short course. A real driver will lose time and distance between the point (s)he first observe [sic] the obstacle and the point at which action is begun.” (Id., Ex. 10 at 15 n. 9.)

3 . NHTSA acknowledged that the three studies regarding the maximum steering inputs a driver is willing or able to make in response to an emergency were based on tests done in the 1970’s. It observed that differences in the handling of mid-70’s vehicles and modern sports utility vehicles, with their higher centers of gravity, and different handling characteristics, could have an effect on the maximum steering angles and rates a driver can perform. (Def.’s Supp’l Materials for Dau-bert Hearing Ex. 10 at 15 n. 8, Paper No. 277.) Nevertheless, NHTSA did not alter the conclusion it reached with respect to its criticisms of the CU Short Course steering methodology. In their supplemental memorandum regarding general acceptance (paper no. 282), Plaintiffs included at exhibit 5 a March, 2000 study which undertook to measure driver steering abilities on two 1990’s model sedans. The study "found that steering rates generated in a recent vintage vehicle were generally higher than those previously reported.” (Pis. Supp’l Memo. Regarding General Acceptance Ex. 5 at 4, Paper No. 282.) Those rates spanned from lows in the low to mid 500 degrees/second range to highs in the mid 900-1100 degrees/second range. However, these findings do not detract from the usefulness of NHTSA’s 1997 findings regarding the defects in the methodology of the CU Short Course test because NHTSA took into account the fact that steer angles and rates could be different for more modern vehicles when it published its conclusions in 1997. Further, the test course used in the March, 2000 study was not an accident avoidance course, but consisted primarily of right turns. {Id.., Ex. 5 at 3.) Therefore, it did not measure the effect of the left-right steering maneuvers required in an AM test, nor did the study purport to analyze the effect that this combination of turns, performed at the fast rates and steering angles it recorded, would have on the performance of the vehicles. Moreover, another study cited by Plaintiffs, titled "Driver Crash Avoidance Behavior with ABS in an Intersection Incursion Scenario on Dry versus Wet Pavement”, (Letter from Pis. to Court of 4/24/00 Ex. 2, Paper No. 298), published in 1999, further supports NHTSA’s conclusions. Although not designed to test vehicles for roll over propensity, the study did méasure steering angles and rates of non-professional drivers confronted with realistic crash avoidance situations. The study reported that average maximum avoidance steering inputs were between 53 and 74 degrees, and the highest steering angles were between 271 and 289 degrees. More' tellingly, the average maximum steering rate was between 262 degrees/second and 294 degrees/second. While there were some high steering rates between 1159 and 1335 degrees/second, ninety-five percent of the steering rates observed were less than 643 degrees/second. {Id., Ex. 2 at 164.) These data, taken from non-professional drivers confronted with authentic "crash imminent” situations prompting the need for accident avoidance maneuvering, produced steering angles and rates within the same range described in NHTSA’s 1997 study, and far below those used by Dr. Kaplan in his MSAI tests on the Aerostar.

4 . Office of Defects Investigation.

5 . During the conference. Ford's counsel expressed doubt about whether the data provided by Plaintiffs' counsel was entirely accurate, based on his recollection of Dr. Kaplan's description of the test results during his deposition. In a letter to the Court dated April 21, 2000, (paper no. 291), Plaintiffs' counsel took issue with the correctness of Ford's assertions. For the purposes of resolving this motion, I need not decide who is more correct, instead I will assume the version of the facts most favorable to the Plaintiffs, and use the data provided by Plaintiffs’ counsel.

6 . Emergency Avoidance Maneuvers, test runs number ab 158-162, ab 169-171.

7 . For example, "double lane change” courses, like the CU Short Course and the MSAI AM test, the "J-Turn” single steer test, and the "Fishhook/Reverse Steer” test. (Pls. Supp'l Memo. Regarding General Acceptance at 1-6, Paper No. 282.)

8 . Exhibit 1 of the Plaintiffs’ supplemental memorandum regarding general acceptance, (paper no. 282), consists of six pages of deposition excerpts from the July 1, 1999 deposition of Daniel F. Tandy, a former Ford engineer, in connection with litigation in Federal Court in California brought by Isuzu against Consumers Union. In the few pages of deposition testimony provided, it is not possible to determine the scope of Tandy’s responsibilities at Ford, or the full context of his testimony. Thus, it is difficult to assess the impact his testimony may have in this case, except to note that it likely would not constitute an admission under Rule 801(d)(2)(D) of the Federal Rules of Evidence because, when given, Tandy apparently no longer was an employee of Ford. In the excerpts, Tandy did acknowledge that while he was employed at Ford, it used the "P6-101” AM test in connection with testing the design of its vehicles. This testimony is confirmed by Ford's 30(b)(6) desig-nee in this case, Mr. Snodgrass. Tandy also acknowledged that the P6-101 AM test, a copy of which was attached as exhibit 2 to the Plaintiffs' Opposition to Defendant's Motion No. 6, (paper no. 245), is vaguely defined, and does not prescribe specific protocols for speed and steering maneuvers. Tandy also testified that from 1986 to 1995 there was no one AM test used by auto manufacturers.

9 . A copy of this test was attached as Exhibit 2 to Plaintiffs' Opposition to Defendant's Motion No. 6. (Paper No. 245.) In their argument, however, Plaintiffs overstate the purpose of this test. Plaintiffs assert that Dr. Kaplan defines a "true” AM test as running a vehicle on the test course at speeds and steering angles and rates sufficiently great to induce the limit of lateral acceleration forces on the vehicle's tires. If the vehicle "slides” in such circumstances, it is not defective, but if it tips up onto its outriggers, it is defective. (Pis.' Supp'l Memo. Regarding General Acceptance at 5-6, Paper No. 282). They conclude that since Ford itself uses an AM test as part of its design process, it must agree with their definition of such a test. However, the exact language of the P6-101 test used by Ford states: "Perform lane changes ranging in severity from light to near the limit of the vehicle capability to assess steering response versus lateral acceleration and roll.” (Pis.' Opp’n Memo. Ex. 2 at AERO 21030, Paper No. 245) (emphasis added.) Thus, Ford’s P6-101 AM procedure does not contemplate subjecting the vehicle to testing at its limits of lateral acceleration, which is the key component of Dr. Kaplan's MSAI test. The fact that Ford uses a version of AM testing which is different in this significant aspect from Dr. Kaplan's MSAI test is hardly evidence that Dr. Kaplan’s methodology has gained general acceptance.

10 . The study to which Plaintiffs refer is titled “Performance of Driver-Vehicle System in Emergency Avoidance.” The chart to which Plaintiffs refer shows that the steering angles studied range from just over 200 degrees for the left turn, and 200 degrees for the right, which is consistent with the report’s finding that "[t]he maximum steering wheel angles in the first steer input for emergency avoidance are concentrated in the range from approxi mately 200 to 230 degrees.... This generally corresponds with the maximum steering wheel angle to which the steering wheel can be turned with the hands kept on the same parts of it.” (Pis. Supp’l Materials for Dau-bert Hearing Ex. 14 at 13, Paper No. 288.) However, the steer angles reported in Dr. Kaplan’s MSAI testing of the Aerostar ranged from left steers in the range of 297-406 degrees, and right turns in the range of 337-399 degrees. To say that these steer angles are "somewhat greater” than those reported in the study cited by Plaintiffs is an ambitious understatement.