MET. ST. LOUIS EQUAL HOUSING OPPORTUNITY COUNCIL v. Gordon A. Gundaker Real Estate Co., Inc.

130 F. Supp. 2d 1074, 56 Fed. R. Serv. 3d 556, 2001 U.S. Dist. LEXIS 1684, 2001 WL 135430

• Court: District Court, E.D. Missouri
• Decided: February 16, 2001
• Docket: 498CV837SNL
• Status: Published

Opinion

130 F.Supp.2d 1074 (2001)

METROPOLITAN ST. LOUIS EQUAL HOUSING OPPORTUNITY COUNCIL, et al., Plaintiffs,
v.
GORDON A. GUNDAKER REAL ESTATE CO., INC., Defendant.

No. 498CV837SNL.

United States District Court, E.D. Missouri, Eastern Division.

February 16, 2001.

Kenneth M. Chackes, M. Susan Carlson, Van Amburg and Chackes, St. Louis, MO, Michael A. Ferry, Gateway Legal Services, Inc., St. Louis, MO, for Metropolitan St. Louis Equal Housing Opportunity Council, a Missouri non-profit corporation, plaintiff.

John Gianoulakis, Partner, Kohn and Shands, St. Louis, MO, Shevon L. Harris, Lewis and Rice, St. Louis, MO, for Florissant, City of, a municipal corporation, plaintiff.

Alan N. Zvibleman, Rothman and Sokol, St. Louis, MO, Paul F. Devine, John R. Hamill, III, Vatterott and Shaffar, Maryland Heights, MO, Daniel K. Barklage, Barklage and Barklage, St. Charles, MO, for Gordon A. Gundaker Real Estate Co., Inc., a Missouri Corporation, defendant.

Joan A. Magagna, Isabelle M. Thabault, Eric I. Halperin, U.S. Department of Justice, Washington, DC, for United States of America, movant.

MEMORANDUM OPINION

LIMBAUGH, District Judge.

The parties have filed cross-motions for summary judgment.^[1] In connection with these motions, the parties have filed cross motions challenging the opposing party's experts. These motions are as follows: defendant's motion to exclude plaintiffs' experts William Peterman, Susan Scovill, Clifford Schrupp, and Shanna Smith (# 53)^[2]; defendant's motion to strike plaintiffs' response in opposition to defendant's Daubert motion (# 60); plaintiffs' motion in limine to exclude defendant's proposed expert testimony of Donald MacPherson (# 95); plaintiffs' motion in limine concerning expert testimony as to plaintiffs' testing methodology (# 91)^[3]; plaintiffs' motion to reconsider the Court's order of April 5, 2000^[4] excluding the affidavits and/or testimony of Juliet Saltman (# 115); and the motion of the United States to intervene with an amicus curiae brief (# 101).

After discussion with counsel, and over objection of plaintiffs, the Court determined that it would be necessary to conduct a so-called "Daubert hearing" pursuant to the Supreme Court decision in Daubert, supra. It appeared that the hearing would be required to consider whether any one or all of the plaintiffs' experts (Peterman, Scovill, Schrupp, and Smith) and defendant's experts (David Graeven, Mike Tiktinsky, and Donald MacPherson^[5]) could qualify as an expert at the trial, and give their expert opinions.

A Daubert hearing was held on April 3 and 6, 2000. Plaintiffs' experts Peterman, Schrupp, and Smith testified; as did defendant's experts Graeven, Tiktinsky, and MacPherson. The Court allowed plaintiffs' to file the deposition testimony of Scovill in lieu of live testimony. The Court received into evidence plaintiffs' exhibits 1, 2, 4, 5, 6A, 8B, 8C, and 12, without objections. The Court received into evidence defendant's exhibits BW, CA, CB, CF, CM and CR with no objections. The Court further received into evidence subject to objections the defendant's exhibits A through Z (the EHOC^[6] test report forms). After careful consideration, the Court now overrules all objections to defendant's exhibits A-Z. Furthermore, the Court will deny the plaintiffs' motion for reconsideration of the Court's decision to exclude the testimony of John Farley and Juliet Saltman; and grant the motion of the United States to intervene and file an amicus brief in this matter. Finally, the Court will consider all exhibits filed with the parties' respective dispositive motions as part of the Daubert hearing record.

The parties have filed extensive pre-hearing and post-hearing briefs. The following findings of fact and conclusions of law are based on the parties' briefs, various memoranda of law filed in connection with the subject motions, exhibits and depositions filed in connection with the hearing, witness' testimony; and exhibits filed in connection with the parties' respective summary judgment motions.

The Court has set out in some detail certain facts relevant to the pending motions for summary judgment. Insofar as they are relevant to the Daubert challenges, they will be adopted. These facts, as well as facts derived from the hearing and briefs filed in connection with the hearing provide some necessary background in the consideration of the Daubert challenges.

In 1996, plaintiff City of Florissant was concerned about possible discriminatory practices of real estate agencies with regard to home sales in Florissant. City officials contacted the EHOC^[7] to investigate whether real estate agencies were engaging in racially discriminatory housing practices in violation of the Fair Housing Act. In particular, city officials were concerned that home seekers were being directed to certain areas or neighborhoods in Florissant because of their race by real estate agents.^[8] This practice is commonly referred to as "steering".

The EHOC decided to conduct an investigation or "audit" of the housing practices in Florissant. It targeted seven (6) real estate agencies in the North County area^[9] including two (2) Gundaker real estate offices. Based upon the 1996 test results, the EHOC concluded that five (5) of the seven real estate agencies were engaging in patterns of racial steering in violation of the Fair Housing Act (FHA).^[10] The only agency believed not to be engaging in racial steering was the sole African-American owned agency, the Betts Realty Co.

In April 1997, the City contracted with the EHOC to do a follow-up audit of the five (5) real estate agencies previously identified by the EHOC in its 1996 study as practicing racial steering in Florissant. Based upon the 1996 test results, the EHOC concluded that the two (2) Gundaker real estate offices were engaging in racial steering in violation of the FHA.^[11]

Armed with the 1996 and 1997 test results, the EHOC and the City of Florissant filed this lawsuit against defendant Gundaker Real Estate Co. for violations of the FHA.

1996 EHOC Tests

James Buchanan was the Test Coordinator for the 1996 tests. Before coming to the EHOC in 1995, Buchanan worked in law enforcement. His training, in connection with the 1996 tests, consisted of a two-day seminar on fair housing conducted by a Memphis lawyer specializing in fair housing issues; a two-day training seminar on fair housing laws and testing conducted by the Justice Department, and a two-week fair housing course at the John Marshall School of Law in Chicago, Illinois.

The tests in general consisted of sending out black and white testers or "checkers" to the targeted real estate agencies pretending to be potential home buyers. The testers were to obtain information regarding the availability of for-sale residential property in the City of Florissant or the Florissant area. The testers approached different agents at these targeted agencies either as individuals or as "couples". The black and white testers were to be matched as closely as possible in terms of familial status, appearance, income, and size and price of the home sought, in an attempt to isolate race as the only variable between the testers. The testers were given a formal "scenario" or profile to present to the agents, but were advised that they could "ad-lib" certain information if the agent asked questions outside the "script". After completing the test, the testers were to complete a test form which asked them a number of questions relating to their contact with the agent such as name and description of agent, what the agent said about availability of the types of homes requested, whether the tester(s) were shown any of the homes and general information about the homes shown, financial information sought by the agent, what particular information about the tester(s) background was sought by the agent, etc. Each of the tester "couple" was to fill out a form. After filling out a tester form, each tester was to be "debriefed" by the Test Coordinator who would then complete a debriefing form.

Buchanan designed the 1996 test forms using some materials supplied by the Justice Department's Fair Housing Manual and his "own experience as an investigator". His forms included not only a variety of information concerning the contact between the tester(s) and the agent, but also required a listing by the tester of the homes shown and a map on which to locate these homes. The test forms were submitted to Bronwen Zwirner, Executive Director of EHOC, for review and revision.

Buchanan also was in charge of training the testers. The testers attended a 5-6 hours training session which consisted of classroom presentations, classroom role-playing, and discussions as to fair housing laws. The testers also participated in practice field tests to "try out" different profiles.

The 1996 tests were conducted from January 1996 through March 1996. With regard to the Gundaker agencies, a white individual or couple, equipped with a profile and a specific description of the type of housing sought (e.g. a two-bedroom house in the $65,000.00 to $75,000.00 range with a set number of bathrooms, etc.) would contact one Gundaker agent seeking information on for-sale property meeting their criteria, and then report their findings on the test forms. Then, a black individual or couple with the same background and same housing needs would contact another Gundaker agent, and report their findings on the test forms. The testers would be debriefed regarding the information put on the forms. Buchanan compared the test results by doing a line-by-line comparison of the test form and attached materials (e.g. descriptive printouts of property identified in the test forms given to the tester). He then mapped this information by placing dots, corresponding to the addresses of homes shown listed by the tester(s), on a residential street map of Florissant.

Two tests were conducted on the Betts Realty Company, and each time only Mr. Betts (the owner) was contacted. The Gundaker agencies were contacted several times, and on each occasion a different agent was contacted.

Based upon his analysis, Buchanan and Zwirner concluded that five (5) of the six (6) agencies, including the two Gundaker agencies had engaged in steering practices. Only the Betts Realty Company was absolved of engaging in racial steering.

1997 EHOC Tests

Bernadine Parker was the Test Coordinator for the 1997 tests. She began her employment with the EHOC in July of 1996. In the summer of 1997 she became the Test Coordinator. Her training to be the Test Coordinator consisted of an approximately one (1) week attendance at a fair housing training session at the John Marshall School of Law, and a one and one-half (1½) weeks training session with HUD. She made no changes to the 1996 test forms or the telephone narrative form (that was to be completed when initial contact was made by telephone). However, changes were made to the forms by someone else. There were two significant differences, other than changes to the questions on the test form, in the 1997 tests. Firstly, the testers were specifically instructed to make initial contact by telephone, and secondly, the test forms no longer contained a map upon which the testers were to show the addresses of the houses shown to them.

Training of the testers varied from the training conducted in 1996. There were no practice field tests. Instead, the testers (as couples) were paired as one experienced tester and one inexperienced tester, and then sent out on an actual test. The protocol for testing, however, was similar to the protocol used for the 1996 tests.

Parker's debriefing method differed significantly from Buchanan's debriefing method. She did not complete any particular type of debriefing form. She simply talked to the tester(s) in person; sometimes she did it over the telephone. Sometimes she had the test form with her when she debriefed, sometime she did not. Unlike Buchanan, debriefing did not always occur within an approximate 48-hour period after the test and completion of the test form. However, her test analysis methodology was similar to Buchanan's methodology.

The 1997 tests were conducted on the five (5) previously identified agencies from the 1996 tests; including the two (2) Gundaker agencies. Based upon Parker's analysis of the test results, the EHOC concluded that the two Gundaker agencies were engaging in racial steering.

ANALYSIS

The initial issue to be addressed by the Court is the plaintiffs' contention that Daubert and its progeny are not applicable in this case because the evidence offered by their witnesses is not the type of scientific evidence contemplated by Daubert. The plaintiffs argue that their experts are really doing nothing more than explaining the importance of tester evidence in racial steering cases, and that tester evidence has always been found to be reliable. They further contend that the defendant is doing nothing more than attacking the credibility of the tester evidence, and that credibility is a jury issue. Finally, plaintiffs contend that even post-Daubert cases allowed the admission of tester evidence.

After careful consideration of the matter, it is clear to the Court that the credibility of the testers is not the issue before this Court. It is the testing protocol under which the testers operated and upon which the plaintiffs' experts derived their opinions which is under scrutiny. Defendant's argument is two-fold; firstly, that the plaintiffs' experts should be excluded from rendering any expert testimony at trial based upon the data generated from the testers and summarized by EHOC staff because such data was the result of an unreliable and scientifically invalid testing methodology; and secondly; the methodology employed by plaintiffs' experts to analyze the tester evidence and form the bases for their opinions fail to meet a proper standard of reliability.

Federal Rule of Evidence, 702, provides that:

"If scientific, technical or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training or education, may testify thereto in the form of all opinion or otherwise."

Rule 702 allows for the admission of expert testimony from a wide range of fields of expertise; however, it provides a safeguard to prevent confusing a jury with so-called "junk science" by building in a reliability factor. Unfortunately, Rule 702 fails to set forth any guidelines in assessing this "reliability" factor.

In Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993) the Court focused on the admissibility of scientific expert testimony and discussed certain specific factors which would be helpful in determining the reliability of a particular scientific theory or technique. The Court determined in Daubert that the trial judge has a "gatekeeping" obligation to determine whether expert testimony is admissible under Rule 702.

In Kumho Tire Company, Ltd., et al. v. Carmichael, et al., 526 U.S. 137, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999), the Court concluded that Daubert's holding setting forth the trial judge's general gatekeeping obligation, applies not only to testimony based on scientific knowledge but also to testimony based on technical and other specialized knowledge. Thus, the gatekeeping duty applies to all expert testimony. Kumho gave the district courts broad latitude in determining what kind of technical, non-scientific evidence was admissible as expert evidence.

The Kumho court stated, "We also conclude that a trial court may consider one or more of the more specific factors that Daubert mentioned when doing so will help determine that testimony's reliability." Kumho, 526 U.S at 141, 119 S.Ct. 1167. The United States Court of Appeals for the Eighth Circuit has followed the Kumho holding. Blue Dane Simmental Corporation v. American Simmental Association, et al., 178 F.3d 1035, 1039 (8th Cir.1999), Jaurequi v. Carter Mfg. Co., Inc., 173 F.3d 1076, 1082, 1081 (8th Cir. 1999).

Given Rule 702's broad field of "expertise" and Kumho's broadening of the application of Daubert to all fields of expert testimony regardless of the field of expertise (i.e. no longer limited to fields historically considered to be "scientific"), the Court determines that the gate-keeping function of this Court as first set out in Daubert is applicable to the testimony that plaintiffs and the defendant seek to elicit through their respective experts. The Court concedes that there are few, if any, reported cases challenging behavioral and social science evidence under Daubert^[12]. However, wherein such evidence is used as a primary source of support for legal theories of liability, such in this case, this Court feels such evidence must be scrutinized under the standards set forth in Daubert and its progeny as to the methods, analysis, and principles these experts used in reaching their respective opinions.

"We conclude that Daubert's general principles apply to the expert matters described in Rule 702. The Rule, in respect to all such matters, `establishes a standard of evidentiary reliability.' 509 U.S. at 590, 113 S.Ct. 2786, 125 L.Ed.2d 469. It `requires a valid ... connection to the pertinent inquiry as a precondition to admissibility.' Id., at 592, 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469. And where such testimony's factual basis, data, principles, methods, or their application, are called into question, see Part III, infra., the trial judge must determine whether the testimony has `a reliable basis in the knowledge and experience of [the relevant] discipline.' 509 U.S. at 592, 113 S.Ct. 2786, 125 L.Ed.2d 469."

Kumho Tire, 526 U.S. at 149, 119 S.Ct. 1167.

When addressing the admissibility of Rule 702 evidence, especially in the face of a Daubert challenge, the district court must look to both the relevancy and reliability of the proffered expert testimony. Blue Dane Simmental Corp., at 1040; see also, Kumho, 526 U.S. at 152, 119 S.Ct. 1167; Daubert, at 509 U.S. at 589, 113 S.Ct. 2786; Jaurequi, at 1082. To assist courts in this herculean effort and promote its gate-keeping function, Daubert sets forth four factors to guide the district courts in resolving admissibility questions:

(1) whether the expert's methodology has been tested;

(2) whether the technique has been subjected to peer review and publication;

(3) whether the technique has a known or knowable rate of error; and

(4) whether the technique has been generally accepted in the proper scientific community.

Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786; Turner v. Iowa Fire Equipment Co., 229 F.3d 1202, 1207-08 (8th Cir.2000). However, these four factors are not exclusive and may not be particularly pertinent to a court's reliability assessment. They may be used exclusively, or as a starting point for the court's inquiry and analysis. The trial court is granted considerable flexibility in adapting its analysis to fit the particular facts of a case. Jaurequi, at 1082 citing Kumho, 526 U.S. at 152, 119 S.Ct. 1167 ("Rather we conclude that the trial judge must have considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable."). "The polestar, however, must always be `scientific validity — and thus the evidentiary reliance and reliability — of the principles that underlie a proposed submission.'" Jaurequi, at 1082 quoting Daubert, 509 U.S. at 593, 113 S.Ct. 2786.

Using these principles for guidance, the Court determines that given the nature of the case and the proposed purpose for which the experts' testimony is being offered, the testing protocol must be reviewed for its "scientific validity" and reliability before addressing each individual expert's testimony.^[13]

After careful review of the testers' depositions, the agents' depositions, the actual test forms completed by the testers, the debriefing forms, and the deposition excepts of Zwirner, Buchanan, and Parker, it is this Court's determination that the testing protocol of both the 1996 and 1997 tests was seriously flawed and lacked any reasonable indicia of reliability. Once again, in making this determination, the Court stresses that it did not adjudge the credibility of the testers or the agents, but looked at the methodology underlining the creation and execution of the tests.

Neither Buchanan nor Parker had an extensive background in fair housing matters, in particular fair housing testing. Neither were especially familiar with the HUD audit studies^[14] nor with the case of Heights Community Congress v. Hilltop Realty, Inc., 629 F.Supp. 1232 (N.D.Ohio 1983), aff'd in part, rev'd in part 774 F.2d 135 (6th Cir.1985).^[15] Neither used the testing protocol established in the HUD studies or modified in Hilltop as a standard for developing their testing protocols. There was ambiguity in the test instructions as to the target housing area; some forms targeted the "City of Florissant", while others targeted the "Florissant area". Some test forms contained an initial "telephone contact form", others did not. Some test forms contained a debriefing form to be completed by the Test Coordinator, some did not.

More worrisome was the inconsistencies in the execution of the tests. Several of the testers failed to follow the fabricated script; i.e. significantly altering their personal background, the reason for contacting the agent, changing the type of house sought. Some failed to complete the test form or left parts blank. The testers differed as to their response to follow-up calls by the agents; some viewed additional homes, some did not. On some test forms, the number of homes shown were not the same for both testers of a "couple" team. In the 1996 tests, one agency was audited by repeated contact with the same agent (who happened to be the owner) while the other agencies were audited by repeated contact with different agents.^[16] The debriefing of testers varied greatly. Especially in 1997, there was little or no consistency in the timing or method of debriefing. In fact, at least one tester was never debriefed.

The plaintiffs argue strenuously that there is a distinction between "research based testing" and "enforcement based testing". They contend that their testing was "enforcement based testing" and thus, social science research standards, HUD audit standards, and the standards espoused in Hilltop are inapplicable. The Court finds this argument meritless.

Firstly, it is clear that both the 1996 and 1997 tests were commissioned by Florissant to investigate and study possible fair housing discrimination taking place in Florissant. Neither of the tests were an effort to gather evidence of housing discrimination based upon any actual complaint by any identified person. Instead, in 1996, the City of Florissant commissioned and partially financed the EHOC to investigate allegations of housing discrimination in the sale of single family residences in the area.^[17] "From the middle of January and in the months of February and March, 1996, EHOC conducted an `audit' or study of the housing practices in the City of Florissant." Defendant's Exhibit Ci — "A Study of Real Estate Housing Patterns in the City of Florissant, Missouri" by The Metropolitan St. Louis Equal Housing Opportunity Council (1996), pg. 1264. The EHOC specifically states that "[t]he study was especially constructed to monitor the activities of Florissant's housing industry in the sale and purchase of single family residences." Exhibit Ci, pg. 1264. More specifically,

"The Florissant Housing Study was constructed and designed to examine three major objectives. First, there was an effort to determine the patterns and practices of the real estate industry operating within the City of Florissant as a whole. Second, the study was designed to determine how individual agents responded to clients in general. Third, the study was designed to determine how citizens interacted with prospective home buyers in various neighborhoods in the City of Florissant. In response to these objectives, audits were conducted at the offices of the concerned Realtors and also at the sales sites of real estate listings throughout the City of Florissant."

Defendant's Exhibit Ci., pg. 1265.

The 1997 test was clearly a follow-up study based upon the perceived results of the 1996 test. "In April of 1997, the City of Florissant entered into a contract to conduct those follow-up tests at the five real estate agencies who were previously alleged to be practicing unfair treatment and steering in the Florissant area." Defendant's Exhibit Cj—"A Study of Real Estate Housing Patterns in the City of Florissant, Missouri — Summer 1997" by the Metropolitan St. Louis Equal Housing Opportunity Council.

Secondly, the plaintiffs offered no evidence which supported their contention that there is a distinction between enforcement and social science fair housing testing. Whether the test results are used as the basis of HUD research study, a social science study, or as evidence in a lawsuit, the techniques employed by testers in gathering the data is the same. In reviewing the Fair Housing Act and the HUD studies, John Yinger^[18] recognized that whether an audit is used for social science research or for enforcement purposes, the basis methodology for gathering data is the same.

"This study [referring to the 1979 study] employed a research technique called a fair housing audit, in which two identically qualified people, one White and the other a member of one of the groups defined in the Fair Housing Act, called a protected class, successively visit a landlord or real estate broker to inquire about advertised housing. The two teammates then record what they were told and how they were treated. Discrimination can be said to exist if the auditors from the protected class are systematically treated less favorably than the White auditors. As discussed below, this technique is also used for enforcement purposes, in which case it is usually called a test, not an audit."

Defendant's Exhibit Cf — Younger, John, Cityscape: A Journal of Policy Development and Research, Vol. 4, Number 3 (1999), "Sustaining the Fair Housing Act", pg. 95.

More importantly, the plaintiffs failed to articulate a set of standards which govern the validity and reliability of an "enforcement test". In fact, plaintiffs' experts Clifford Schrupp and Shanna Smith opined that there are no standards that govern the validity and reliability of fair housing tests, including the EHOC tests. According to Schrupp the only standards by which he adjudged the EHOC tests were the result of conversations with fair housing attorneys: 1) make sure the testee be an owner or agent of the owner of real property covered under the FHA; 2) make sure the tester is available for court testimony (either live or by affidavit) and that all relevant documentation has been preserved; and 3) that the test involve gathering the type of evidence indicative of an alleged violation of a provision of the FHA.

There simply was no standardized protocol utilized in the design and execution of the EHOC tests. The Court is not dismissing the fact that tester evidence is probably the best means by which to demonstrate fair housing violations. However, the production of such evidence must be done under some set of standards which assure a reasonable level of validity and reliability. In this case, the EHOC failed to utilized standards available to it in designing and executing the tests; i.e. HUD standards, the standards set out in Hilltop, or even generally recognized social science standards. The plaintiffs have failed to demonstrate by a preponderance of the evidence^[19] that the techniques employed in designing and executing the EHOC tests in determining that defendant engaged in racial steering are sufficiently reliable to satisfy Rule 702. Although the Court makes this determination as to the tests themselves, the Court does not conclude that the unreliability of the testing methodology alone renders the plaintiffs' experts testimony inadmissible; however, it does have relevance when considering their methodology and conclusions.

This leads the Court to the second part of its Daubert analysis. Having found that the EHOC tests had serious design flaws giving rise to questionable data, the Court now addresses each expert's proffered testimony.

Shanna Smith

Ms. Smith is the Executive Director of the National Fair Housing Alliance. She has been involved in housing discrimination testing in various cities for several years. She has worked with HUD on fair housing issues and is familiar with both HUD studies. In her expert report and the hearing she offered two expert opinions: 1) that race can be identified conclusively over the telephone; and 2) that the defendant's experts used the "wrong" standards to analyze the data gathered by the testers and summarized by EHOC staff.

As to racial voice identification, Ms. Smith's opinion is primarily based upon her own experience through her involvement in other fair housing discrimination testing. She is not a linguist nor trained in voice analysis. There was no evidence that she has any background in voice therapy. Although she has published works on fair housing, she has not published any work which addresses her theory as to racial voice identification. In preparation for the Daubert hearing she contacted a Stanford professor on his work with racial voice identification.^[20] She is familiar with articles written by a number of other individuals but concedes that there are no written reports linking racial voice identification with fair housing testing. As Executive Director of the National Fair Housing Alliance, she has not conducted any type of standardized testing for racial voice identification but feels that one particular article entitled "Perceptual and Phonetic Experiments on American English Dialect Identification" by Thomas Purnell, William Idsardi, and John Baugh is strongly supportive of the theory that a person's race can be determined by their manner of speech during a telephone conversation.

Ms. Smith's opinion regarding racial voice identification can best be summed up by her statement that telephone identification for race and ethnicity has general acceptance within the community that she deals with and that her own experience has shown that people can guess race or nationality by voice alone. This "general acceptance" can at times be sufficient to consider certain expert evidence admissible; however, not in the present case. Ms. Smith fails to point to any real authority which demonstrates this "general acceptance" other than her own belief. Furthermore, she has demonstrated no working knowledge of the dialects in this part of the country. Not only does she lack specific training in voice identification especially with regards to race, she is totally unfamiliar with the St. Louis region as to its various dialects. She has never lived in St. Louis (or Florissant), she is unfamiliar with the so-called "St. Louis" accent. She had no knowledge as to whether a "regional dialect" exists. She never talked with any of the testers in this case to ascertain whether any tester's voice was strongly indicative of race. She has never heard any of the conversation of the testers and other people involved in this case. In her work with the insurance industry, she concedes that when testing for discrimination as to homeowners' insurance, testers are instructed to use a "black voice" and to state that they are calling from an area widely considered to be a "black neighborhood" or a "white voice" and calling from a "white neighborhood". She was vague as to what exactly constitutes a "black voice" and/or "white voice".

As for the Purnell, Idsardi, and Baugh article, it concludes that in all probability race can be identified by voice, but such a theory is not supported by the Federal Rules of Evidence. In Footnote I of the article, the authors acknowledge that while the Federal Rules of Evidence allow for voice identification (as to a particular person), there is no support in the rules for any evidence that a person's race can be identified by voice, especially over a telephone.

Ms. Smith's expert testimony regarding racial voice identification fails to meet the Daubert criteria for scientific validity and reliability.

Regarding her expert opinion as to the "wrong" standards being applied by the defendant's experts in reviewing the EHOC test data, this testimony too fails to meet the Daubert criteria. Ms. Smith is not a social scientist or statistician. She did not review a single test report form, tester deposition, agent deposition, or test debriefing report in this matter. Her opinion that the EHOC tests were "enforcement tests" is not supported by any independent evidence. She failed to cite any authority that an "enforcement test" should be governed by a different set of standards as those utilized by the defendant's experts. In fact, she could offer no standard by which to adjudge the validity and reliability of the EHOC tests. Her proposed rebuttal testimony regarding the defendant's experts' methodology is inadmissible.

Clifford Schrupp

Since 1973, Mr. Schrupp has been the Executive Director of the Fair Housing Center in Detroit. He is familiar with the 1989 HUD study and the techniques used. He has been using testers in cases such as the present one since the 1960s. He has supervised many investigations such as the present one.

His expert opinion is focused on the "mistaken" approach taken by the defendant's experts in analyzing the EHOC test data. He basically contests their position that the testing needed to follow the standards of a social science audit. He believes that the EHOC tests are on solid foundation for "enforcement testing" because they followed a protocol developed through legal consultation. The testing was done in accordance with certain legal precepts: 1) the testee should be someone falling within the purview of the FHA; 2) the tester's evidence (both via testimony and documentation) should be preserved for admission as evidence in court; and 3) the test should focus on gathering evidence of an alleged violation of a provision of the FHA. Mr. Schrupp had no opinion as to whether defendant engaged in racial steering.

Mr. Schrupp has no formal degree in any field of social science. He is not a statistician nor a lawyer. His only formal degree is in theology. He admits that he is not an "expert" and characterized himself as a "fact witness". As for the standards he espouses for an "enforcement test" he fails to point to any authority, treatise, journal, or any written body of knowledge that supports his "legal" standards for the testing done in this case. Mr. Schrupp's theory on fair housing testing has not been subject to any peer review or publication, its known potential rate of error has not been tested, and it has not been shown to be generally accepted as reliable by the relevant social science community. This lack of a reliable foundation for his opinion is equally applicable to his opinion criticizing defendant's experts' reports. The Court can appreciate Mr. Schrupp's many years of dedicated service in the fair housing area; however, his testimony is not being offered for its expertise in the fair housing area in general. He himself states that he is no expert with regard to fair housing testing. Mr. Schrupp's testimony and report are inadmissible as they fail to meet the Daubert criteria for scientific validity and reliability.

William Peterman

Dr. Peterman has his bachelor and masters degrees in geophysical science (meteorology) and his doctorate degree in urban geography. He has primarily been involved in urban planning projects for the last decade. He has taught statistics courses, especially graduate courses in quantitative methodology. He has been involved in several studies regarding fair housing issues such as mortgage discrimination and other discriminatory lending practices. He has never designed, implemented, controlled or supervised a fair housing test on real estate agents.

Dr. Peterman performed three (3) analytical tests on the data set^[21] provided by EHOC, and based upon the information provided by the EHOC staff, Dr. Peterman concluded that defendant had engaged in racial steering. Dr. Peterman's conclusions were based upon his assumption that the EHOC data set was valid and upon being told "data was collected properly" by EHOC staff. He did not review any of the original underlying data; i.e., the tester depositions, the agent depositions, the actual test report forms, the debriefing forms. For purposes of his analysis, he assumed that the testing protocol had isolated race as the variable factor. He conceded that if the underlying data was flawed, then his analysis of the data supplied to him by the plaintiffs lacked reliability.

With respect to the first statistical test, a "T-Test", Dr. Peterman conceded both in his deposition and at the hearing that he applied this test incorrectly. Although conceding this point, he was reluctant to alter his expert opinion in light of this error.

The next test he used is the Chi-Square test, another statistical test. Plaintiffs contend that the data gathered in the EHOC tests was not "matched pair" data.^[22] However, Dr. Peterman testified several times that it was in fact "matched pair" data.^[23] He admitted at the hearing that the application of the Chi-Square test to this type of data has not been tested, nor subjected to peer review or publication, its known rate of error has not been determined (as applied to "matched pair" data), nor has it been generally accepted in the relevant scientific community for use with "matched pair" data.

The last test he used is the "Nearest Neighbor Test", another statistical test. This test, as explained in his expert report, is used to "investigate the nature of the location of homes as plotted on Map 2." It is used to "study the significance of the geographic distribution of various properties". Dr. Peterman admitted that this test could not be used to determine the existence of racial steering. It simply demonstrates the distance between known points on a map; in this case, allegedly the houses shown. Dr. Peterman conceded that the "Nearest Neighbor Test" is widely accepted in the field of geography, not in any field of social science.

The Court determines that Dr. Peterman's testimony and expert report are inadmissible for failure to meet the Daubert criteria on several grounds. Firstly, the Court cannot simply dismiss his failure to review the underlying data for accuracy. Granted that statisticians may often rely on "secondary data" (data collected by others) and not "primary data" (data collected by the researcher himself or herself), in this case, he relied upon "tertiary data".^[24] Given the importance of the accuracy of the underlying data, a fact he admitted, it would have been prudent for him to examine the test report forms and the debriefing forms to assure himself of the accuracy of the underlying data.

Secondly, he admittedly used an incorrect test because he was not familiar with statistical testing in the area of fair housing. He did not realize that he had used a wrong test until he read an article by John Younger which evidently put him on notice for the first time that the T-Test was not suitable for the type of data collected in the EHOC tests.^[25] His unfamiliarity with the erroneous application of this test belies his expertise in statistical testing in fair housing analysis.

Finally, the Court finds the Nearest Neighbor Test to be irrelevant to the issues in this case. The plaintiffs assert that defendant engaged in racial steering in violation of the FHA. No expert conclusion regarding racial steering can be derived from this test. Dr. Peterman's testimony will not "assist the triar of fact" in any significant issue in this case. The jurors themselves can see the pins on the map, determine if they match the information on the testers' reports, and decide if the locations of the houses shown display any particular pattern indicative of racial steering.

Although Dr. Peterman could be considered qualified as an expert in statistical analysis of geographic data, the Court finds that he lacks sufficient knowledge through training, education, or experience to be considered qualified to offer an expert opinion regarding matched pair data analysis and/or whether the EHOC tests provide evidence of racial steering. See, Smith v. Rasmussen, 57 F.Supp.2d 736, 766 (N.D.Iowa 1999) (citing cases in which persons were found to be an expert in one field but not qualified to testify as an expert in another field); see also, Uribe v. Sofamor, S.N.C., 1999 WL 1129703 (D.Neb., August 16, 1999) (in a "bone screw" case, plaintiffs medical expert found to be qualified as an expert on the topic of spinal bone fusion but not on the topic of bone screw fixation devices or instrumented spinal surgery). Furthermore, Dr. Peterman's opinions do not "fit" this case because his conclusions are based upon unreliable data furnished by EHOC and his own invalid and unreliable methodology assessing this data. Under Daubert, the expert testimony proffered in a case must be sufficiently tied to the facts of the case so as to aid the jury in resolving a factual dispute. Id., 509 U.S. at 591, 113 S.Ct. 2786. This "tie" was referred to as "fit"; i.e. the expert testimony must not only be scientifically reliable and valid but also "fit" the facts of the case in which it is offered. Id., 509 U.S. at 591, 113 S.Ct. 2786.

"While Daubert states that a trial court's focus must be on the experts' underlying principles and methodology rather than on his conclusions, the Supreme Court has since clarified that `conclusions and methodology are not entirely distinct from one another,' and that `[t]rained experts commonly extrapolate from existing data.' Accordingly, `[a] court may conclude that there is simply too great an analytical gap between the data and the opinion offered.'" Jaurequi, at 1082, n. 3 quoting General Electric Co. v. Joiner, 522 U.S. 136, 118 S.Ct. 512, 519, 139 L.Ed.2d 508 (1997); see also, Concord Boat Corp. v. Brunswick Corp., 207 F.3d 1039, 1055 (8th Cir.2000).

Susan Scovill^[26]

Ms. Scovill is a lawyer and presently the Director of Fair Housing for Housing Opportunities Made Equal (HOME) in Richmond, Virginia. She has worked for the Virginia Department of Commerce as its fair housing administrator. She is certified to train real estate agents and brokers in fail housing. She has conducted fair housing classes for real estate agents and brokers in various parts of the United States and for HUD.

She was asked by the plaintiffs to give an opinion as to what constitutes standard and appropriate fair housing education for real estate sales agents and brokers; and give an opinion as to the quality and effectiveness of defendant's fair housing training in particular. She reviewed the training materials of defendant, the depositions of defendant's management officials in charge of fair housing training, the depositions of defendant's managers, agents and brokers involved in this case, and the applicable statutes and regulations on fair housing.

She opined that a good and effective fair housing training program for real estate agents and brokers should include: 1) a clearly communicated message from the company that it is committed to the letter and spirit of the fair housing laws; 2) that the fair housing training should be provided by an instructor with substantive knowledge of fair housing laws and fundamental principles; and 3) training that results in agents who have a fundamental understanding of the basic fair housing principles. Based upon her methodology regarding good and effective fair housing training, Scovill concluded that defendant's fair housing training education program was deficient and failed to provide means of ensuring that agents understood and applied the information available to them regarding fair housing compliance.

Defendant faults her findings regarding defendant' fair housing education and program contending that she really did not look into the particulars of its program. However, defendant fails to really question her methodology. It does not challenge her experience and background as lacking with regards to a foundation for the validity and reliability of her opinion. It simply believes that her testimony is inadmissible because she lacks information regarding the background of the defendant's instructors, the names of the instructors, and the content of the defendant's core curriculum. These objections may attack the strength of her testimony but they don't sufficiently challenge the validity and reliability of the methodology underlying her opinion.

There is, however, one point raised by the defendant which goes to the heart of the dispute over Ms. Scovill's testimony. Defendant correctly states that improper training has not been pleaded as a cause of action in this case; i.e. there is no claim that defendant's real estate agents were improperly trained. There is no provision under the FHA which provides that inadequate training of real estate agents as to fair housing compliance is a violation of the FHA. Plaintiffs contend that her testimony is relevant because real estate agents should have training in fair housing compliance and "[i]t appears that in this defendant may argue that its agents have training about fair housing compliance and, therefore, could not have engaged in violations of the fair housing laws."

At this time, the Court determines that Ms. Scovill has extensive experience and expertise in training real estate agents and brokers in compliance with the requirements of the FHA. Under Rule 702 she has the specialized knowledge which qualifies her to offer her expert opinion of fair housing education and training for real estate agents in general, and to offer her opinion on defendant's education and training program. However, Ms. Scovill's testimony will only be admissible in rebuttal to any testimony offered by defendant as to the integrity of its fair housing education and training program, and compliance by its agents.

David Graeven

Dr. Graeven has a Ph.D. in sociology and has supervised hundreds of research projects involving research design. He has over thirty-five (35) years experience in social science research. His only exposure to housing issues was in the early 1970s when he conducted a study in rent discrimination in Haywood, California. He has testified in U.S. courts as an expert on social science research and research design on several occasions. He no longer teaches and is presently employed with a jury research firm.

Dr. Graeven reviewed the methodology employed by EHOC staff and testers in designing and executing the tests. He reviewed the depositions of the testers, materials supplied by EHOC regarding its tester training program, and the depositions of EHOC trainers. In preparing his expert report, Dr. Graeven did not assess the credibility of the EHOC staff and/or testers. He opined that in order to conduct a competent and reliable study, whether it be for research or enforcement, certain fundamental precepts of social science must be followed. Foremost, in order to determine validity, one must be able to measure reliability and dependability of data against certain social science research standards including but not limited to: testers must be properly trained (some recognizable measure of consistency), all data must be included, all variables except one (in this case, race) must be controlled to isolate the one variable as making the difference in results. Particular to this case, Dr. Graeven further noted that the test results must be properly and consistently analyzed through debriefing, and the process of collecting data via the testers must be consistent. He opined that the HUD studies' protocol should have been considered by EHOC because it has generally been accepted as a valid method for investigating alleged racial housing discrimination.

Dr. Graeven concluded that the EHOC test protocols were severely flawed as to design and execution. EHOC staff failed to use any standardized testing procedure other than to implement the suggestions of attorneys as to preservation of evidence for trial. There were marked differences in training of the testers in 1997 from training in 1996. There were inconsistencies in the information sought on the test forms, such as the targeted residential area being either the "City of Florissant" or the "Florissant area". He found that testers executed the tests in a variety of ways; i.e. some adhered to the script, some "ad-libbed" vital background information. Some testers made initial contact by telephone, while others did not. Some testers did not complete the test form. Debriefing of testers became significantly inconsistent in that debriefing in 1996 was done by the Test Coordinator completing a debriefing form; however, in 1997, debriefing took a variety of forms; i.e. over the telephone, in person, shortly after completion of the test, an extended period of time after the test, with the test report form in hand, and without the test report in hand.

Plaintiffs argue that Dr. Graeven's expert testimony regarding the plaintiffs' testing methodology should be inadmissible because he used the "wrong" standards to adjudge the EHOC testing. As stated before, the plaintiffs have failed to articulate the standards that should be applied to assess the validity and reliability of the EHOC testing methodology, even if the tests would be considered for "enforcement purposes". Dr. Graeven's proposed social science research testing standards are widely accepted in the social science research community and have been subjected extensively to peer review and publication. His testimony is based upon approved testing procedures. The Court finds that Dr. Graeven's testimony and expert report meet the criteria set down under Daubert and its progeny for scientific validity and reliability. Furthermore, the Court finds such testimony to be relevant to the facts in this case and of assistance to the jury in adjudging such facts. Dr. Graeven's testimony and expert report is admissible under Rule 702.

Mike Tiktinsky

Dr. Tiktinsky has doctorate degrees in psychology and mathematical/statistical psychology. He has been involved in behavioral sciences data analysis since 1965; working primarily in field research and statistical analysis. He has been a party to several field research projects involving matters of discrimination and has testified frequently as a data analyst in discrimination cases. He has given numerous seminars and workshops on field research protocol.

Dr. Tiktinsky reviewed the EHOC testing methodology and the analytical methodology utilized by plaintiffs' experts. He reviewed Dr. Graeven and Dr. Peterman's expert reports at length. He reviewed numerous articles on measuring discrimination in housing studies. He reviewed the HUD studies and the Hilltop decision. He reviewed not only the data set compiled by EHOC staff, but the testers' report forms from which the data set was derived.

He concluded that the EHOC failed to follow the systematic protocol of the HUD studies and failed to follow the systematic protocol of established field research. As to the HUD studies, he does not embrace them as the standardized protocol for racial housing discrimination testing; however, he recognizes that the HUD studies followed a generally reliable testing protocol which would have provided some measurement of validity and reliability to the EHOC tests. He concurred with Dr. Graeven as to the EHOC test design flaws and execution inconsistencies.

He opined that the methodology employed by the plaintiffs' experts in generating their opinions was scientifically invalid and unreliable. He particularly faulted Dr. Peterman for failing to review the underlying data for accuracy and for simply relying on Zwirner and counsels' assertion that the testers' data was property collected. Based upon his review of the underlying data and the data set compiled by the EHOC staff, he concluded that little, if any, correlation existed. Using widely accepted social science research methodology in comparing raw data with summarized data, he found marked differences. He found that the number of properties listed by the tester or tester couple did not always equal the number recorded in the EHOC data set. He found on one or more occasions, testers working as a couple did not record the same number of properties shown or referred. He found that the test coordinators regularly recorded on the debriefing forms a different number of houses shown or referred than the tester(s) actually listed on the test report form.

Furthermore, he found that Dr. Peterson's methodology for analyzing the EHOC test data lacked a systematic approach as to the inclusion and exclusion of data. He could discern no real reason for discarding sonic data (one couple did not complete the test report form; in one test, only the "husband" went to see the houses), while keeping in other data (one tester did not complete the test report form until two years after the test was completed).

Finally, Dr. Tiktinsky opined that the EHOC tests were a "matched pairs" test and as such, should not have been subjected to a Chi-Square Test. He testified that there are no published works, in the social science area, that accept the Chi-Square Test for matched pair data; in fact, he cited several publications which found that the Chi-Square Test should not be used to analyze a matched pair data set.

As to Ms. Smith's racial voice identification theory, he conceded that it might be possible to identify a person's race by only their voice; however, he was unaware of any documentation subscribing to this concept for use in fair housing cases.

Plaintiffs assert that Dr. Tiktinsky's testimony and expert report is inadmissible for several reasons. They contend that he is improperly attacking the credibility of the testers. They contend that he used the "wrong" standards by which to adjudge the EHOC testing methodology and the methodologies used by their experts. These arguments are meritless. Dr. Tiktinsky's testimony regarding the reliability of the EHOC's evidence is a challenge to the reliability of the testing protocol. It goes to the core of the reliability of the plaintiffs' experts' analytical methodology. As Dr. Peterman stated, "garbage in, garbage out". Dr. Tiktinsky's methodology regarding comparability of raw data; i.e. the comparability of the matched pair tests is extremely relevant to the assumption by Dr. Peterman that the underlying data was valid and reliable; his methodology and conclusions drawn rested entirely on this assumption. As for the "wrong" statistical analysis, the Court has already stated its reasons for rejecting this argument. More significant is the plaintiffs' failure to identify the particular statistical analysis that they believe Dr. Tiktinsky incorrectly applied in his analysis. They do not quarrel with the scientific validity and reliability of his methodology; they simply disagree with its application in this case.

Dr. Tiktinsky's analytical methodology has been used extensively on matched pair data, has been subjected to peer review and publication, is widely accepted in the field of social science research and nonparametric statistics, and has a clearly defined rate of error. Dr. Tiktinsky is aptly qualified as an expert to give his opinion in this case as to the testing methodology and analytical methodology present in this case. His evidence is not only reliable but relevant to the issues present and would be of assistance to the jury in adjudging the facts. His testimony and expert report meet the criteria set down by Daubert and its progeny, and thus, is admissible under Rule 702.

Donald MacPherson

Mr. MacPherson is a lawyer who works primarily in the real estate field. Since 1976 he has been a licensed real estate agent and broker. He has been a corporate owner of REMAX Realty Co. since 1996. He has served as the President of the Missouri Association of Realtors and on the Board of Directors of the National Association of Realtors. While serving on the Board of Directors of the National Association of Realtors, he has sat on the Professional Standards Committee. He also sits on the Missouri Real Estate Commission.

Mr. MacPherson's proffered testimony and expert report is concerned with the professional and ethical standards real estate agents follow in Missouri. The plaintiffs' primary objection to his testimony and expert report is that he will be telling the jury what the law is with regard to the FHA, and what constitutes compliance with the Act. Upon review of his expert report and his proffered testimony, the Court determines that Mr. MacPherson's testimony only informs the jury as to the professional and ethical standards established by Missouri's real estate professional organizations. He offers no opinion, legal or otherwise, as to what constitutes compliance with state or federal fair housing laws. Thus, he possesses specialized knowledge which is relevant to issues present in this case, and will assist the jury in adjudging the facts. He is qualified to testify as an expert pursuant to Rule 702.

Plaintiffs have failed to demonstrate by a preponderance of the evidence that Smith, Schrupp, and Peterman's methodologies for evaluating the design, execution, and data derived from the EHOC tests are sufficiently reliable to establish defendant's legal liability for violations of the FHA. The methodologies have not been shown to be sufficiently reliable to provide a sound basis for the conclusions drawn.

This is not a wholesale dismissal of racial steering testing protocol. It may be current standardized social science research methodologies, if adhered to closely, might satisfy the Daubert and Kumho requirements and meet the standards of Rule 702 for admissibility, but the plaintiffs have failed to show that such standards were met in this case.^[27]

On the contrary, defendant's experts have demonstrated by a preponderance of the evidence that their analytical methodologies for evaluating the design, execution and data derived from the EHOC tests are scientifically valid and reliable for admissibility under Rule 702.

In light of the above findings, the Court makes the following conclusions regarding the proffered testimony of Smith, Schrupp, Peterman, Scovill, Graeven, Tiktinsky, and MacPherson:

1) Smith, Schrupp, and Peterman may not testify as to the validity and reliability of the design of the EHOC tests.

2) These same witnesses may not testify as to the validity and reliability of the execution of the EHOC tests, including the training of the testers.

3) These same witnesses may not testify as to the validity and reliability of the data derived from the EHOC tests, nor testify to any opinion that such data definitively shows that the defendant engaged in racial steering.

4) These witnesses may not testify to any opinion regarding a different testing protocol for "research-based" testing and "enforcement-based" testing.

5) These witnesses may not testify to any opinion regarding the "correct" analytical methodologies applicable in this case.

6) Smith may not testify to any opinion regarding voice identification as to race in general or by any agent of the defendant.

7) Scovill may testify and give her expert opinion in rebuttal regarding fair housing education and training of real estate agents in general, and the adequacy of the defendant's fair housing education and training of its real estate agents.

8) Graeven and Tiktinsky may testify and give their expert opinions on the design, execution, and data derived from the EHOC tests.

9) Graeven and Tiktinsky may not testify to any opinion as to the credibility of any testimony by any tester or agent of the defendant.

10) MacPherson may testify as to the professional and ethical standards established by Missouri's real estate professional organizations.

11) MacPherson may not testify or give his legal opinion as to any agent's compliance with either the FHA or Missouri's fair housing laws.

NOTES

[1] These motions will be addressed in a separate order and memorandum.

[2] A challenge to experts brought pursuant to Rules 104(a), 403, and 702 Fed.R.Evid. and under the auspices of Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993) is now commonly referred to as a "Daubert motion".

[3] The Court, as do the parties, treat this motion as a "Daubert motion".

[4] The Court apprised counsel and the parties of the decision to grant the defendant's earlier motion to exclude affidavits and/or testimony of John Farley and Juliet Saltman by letter, dated April 5, 2000. Said letter was docketed as document # 112.

[5] Although MacPherson was the subject of a motion in limine which did not raise specifically a Daubert challenge, the Court determined that in the interest of efficiency it would be best to have him testify and consider the motion in limine as a Daubert challenge.

[6] Equal Housing Opportunity Commission

[7] Plaintiff EHOC is a non-profit organization which promotes fair housing in the St. Louis metropolitan area. In furtherance of this purpose, the EHOC develops and implements educational programs to inform the public of its fair housing rights and obligations. It also investigates housing practices in the St. Louis metropolitan area. The EHOC was incorporated by its Board of Directors in 1992. Its primary source of funding is the U.S. Department of Housing and Urban Development (HUD) Fair Housing Initiatives Program.

[8] The City's suspicions were not based upon any tangible evidence other than one anonymous phone call raising a concern that homes in his or her neighborhood were being shown only to persons of one race. At no time during this hearing or in any papers filed with the Court has any person been identified as a victim of alleged housing discrimination in the City of Florissant.

[9] Florissant is one of several municipalities which are located in the North County area of metropolitan St. Louis.

[10] Defendant's Exhibit Ci: "A Study of Real Estate Housing Patterns in the City of Florissant, Missouri" by the Metropolitan St. Louis Equal Housing Opportunity Council.

[11] Whether or not the 1997 test results showed racial steering by the other real estate agencies is not at issue in this case and is of no concern to the Court.

[12] The Court has been unable to locate any caselaw which addresses a Daubert challenge to the type of evidence presented here. The plaintiffs contend that since "tester evidence" has always been accepted by the courts, pre-Daubert and post-Daubert, it should simply be accepted at face value here. The Court disagrees. Firstly, just because no one has ever challenged the type of evidence or methodology present here is not a justifiable reason to ignore the challenge; and secondly, as stated before, contrary to the plaintiffs' assertions, this is not a challenge to the testers' factual evidence (what questions the agents asked, what information the testers gave, the number of houses the testers were shown, etc.), but a challenge to the validity and reliability of the tests' design, execution, and analysis.

[13] This two-phase examination was used by a Nebraska district court in a post-Daubert case in which the psychological evaluation protocol, upon which the expert testimony of a psychiatrist and two psychologists relied, was examined by the district court for scientific validity and reliability prior to examination of the analytical methodology of the experts. Gier, et. al. v. Educational Service Unit No. 16, 845 F.Supp. 1342 (D.Neb.1994), aff'd 66 F.3d 940 (8th Cir.1995).

[14] In 1979 and 1989, HUD carried out national studies to investigate housing discrimination in the United States. There have been no further HUD studies since 1989.

[15] In Hilltop, the district court examined the "HUD-type audit" and found it to be a generally valid method of detecting discriminatory housing practices especially with regard to race. Id., at 1248. However, it made some modifications to the audit protocol to make it more reliable and accurate; such as, taking into account all the homes shown and listings offered by an agent to the tester in each test. Id., at 1248.

[16] In auditing the Betts Real Estate Agency, Mr. Betts was contacted by both white and black tester(s). There is no evidence before the Court that in auditing defendant, Mr. Gordon Gundaker was contacted by any tester.

[17] There was absolutely no evidence provided to the Court that any particular person at any time had accused any real estate agency in Florissant, especially the defendant, of any type of housing discrimination.

[18] There is no dispute that Mr. Yinger is a recognized expert in the field of fair housing and was referenced by experts on both sides.

[19] There is scant post-Daubert caselaw which addresses the burden of proof in a Rule 702 analysis. However, in Daubert, the Supreme Court stated that the admissibility of expert scientific testimony should be established by a preponderance of proof. Daubert, 509 U.S. at 593, n. 10, 113 S.Ct. 2786. The Fifth Circuit of Appeals has elaborated on this standard by stating that "[t]he proponent need not prove to the judge that the expert's testimony is correct, but she must prove by a preponderance of the evidence that the testimony is reliable." Moore v. Ashland Chemical Co., 151 F.3d 269, 276 (5th Cir.1998); see also, Tanner v. Westbrook, 174 F.3d 542, 547 (5th Cir.1999).

[20] She identified this person as Dane Archer, but failed to identify him in her professional resume or expert report.

[21] The data set included the mapping of the houses shown as indicated on the testers' forms, and a computer-generated data base of the specific vital statistics taken from the forms such as race of the tester, race of the agent, addresses of houses shown, number of houses shown, etc.

[22] This argument is somewhat disconcerting since both of the EHOC published studies clearly refer to the tests as being "matched pairs of tests". Defendant's Exhibit Ci (the 1996 EHOC study report), pg. 1264; Defendant's Exhibit Cj (the 1997 EHOC study report), pg. 1310.

[23] The transcript of Dr. Peterman's testimony, pgs. 28-29; 62-63; and 72.

[24] Under standard statistical concepts, in this case, the primary data would have been data Dr. Peterman collected himself, secondary data would have been data collected by the testers, and tertiary data is the maps and data set compiled by the EHOC staff.

[25] The Court acknowledges that plaintiffs are now disregarding the T-Test analysis both in testimony and Dr. Peterman's report. However, it still impresses upon the Court the fact that Dr. Peterman was not aware of the erroneous use of this test until he read an article by someone else recognized as an expert in fair housing matters, including testing analysis.

[26] Ms. Scovill's proffered testimony is before the Court through her deposition and expert report. Due to great travel expense, the Court allowed the plaintiffs to offer her deposition testimony and expert report in lieu of live testimony at the Daubert hearing.

[27] In Gier, supra, the district court found that although the evaluation protocol (Child Behavior Checklists and clinical interviews) might be admittedly standardized and accepted as reliable for use with non-mentally impaired children, the same could not be said for their use with mentally impaired children when investigating possible abuse.