Association of Mexican-American Educators v. California

231 F.3d 572, 2000 Cal. Daily Op. Serv. 8675, 2000 Daily Journal DAR 11533, 2000 U.S. App. LEXIS 27040, 79 Empl. Prac. Dec. (CCH) 40,298, 84 Fair Empl. Prac. Cas. (BNA) 474

Opinions

Opinion by Judge GRABER; Partial Concurrence and Partial Dissent by Judge REINHARDT; Partial Concurrence and Partial Dissent by Judge FERNANDEZ; Partial Concurrence and Partial Dissent by Judge KLEINFELD; Partial Concurrence and Partial Dissent by Judge GOULD; Dissent by Judge TASHIMA.

GRABER, Circuit Judge:

Plaintiffs are a class of Mexican-American, Asian-American, and African-American educators and would-be educators in California. They appeal from an adverse judgment in their action against the State of California and its agency, challenging (1) the district court’s holding that the California Basic Education Skills Test (“CBEST”), which is a prerequisite to employment in a variety of positions in the California public schools, violates neither Title VI nor Title VII of the Civil Rights Act of 1964, and (2) the district court’s use of a technical advisor at trial. Defendants cross-appeal from the district court’s rulings, on summary judgment, that Title VI and Title VII apply in this case. Defendants also appeal from the district court’s order denying their request for costs. For the reasons that follow, we hold that Title VII applies to the CBEST; that the CBEST was validated properly; that the district court permissibly used a technical advisor; and that the district court did not abuse its discretion by refusing to award costs to Defendants. Accordingly, we affirm both the judgment in Defendants’ favor and the order denying them costs.

FACTUAL AND PROCEDURAL BACKGROUND

Effective February 1, 1983, the California legislature amended the California Education Code to prohibit the California Commission on Teacher Credentialing (“CCTC”) from issuing “any credential, permit, certificate, or renewal of an emergency credential to any person to serve in the public schools unless the person has demonstrated proficiency in basic reading, writing, and mathematics skills.” Cal. Educ.Code § 44252(b). At the same time, the legislature authorized the state’s Superintendent of Public Instruction to “adopt an appropriate state test to measure proficiency in these basic skills.” ■ Cal. Educ.Code § 44252(c). The Superintendent adopted the CBEST and, in May 1983, CCTC assumed responsibility for administering and revising the test.

The CBEST is a pass-fail examination consisting of three sections: reading, writing, and mathematics.¹ The reading and mathematics sections each contain 50 multiple-choice questions, 40 of which are scored. The writing section consists of two essays. The CBEST was revised in 1995. At that time, questions that tested “higher order” mathematical skills, such as geometry, were eliminated from the mathematics section of the test.

To pass the CBEST, a candidate must receive a “scaled” score of 123. Accordingly, a candidate passes by averaging 41 points on each of the three sections (out of a score range of 20 to 80). A scaled score of 41 on the reading section translates into a raw score of 28 out of 40 questions correct; on the mathematics section, a scaled score of 41 equates to a raw score of 26 out of 40 correct. Each of the two essays is graded by two readers, who give raw scores of between one and four points per essay. Thus, the range of possible scores for the writing section is between four and 16 points. A raw score of 12 points translates into a scaled score of 41 points. The CBEST employs a “compensatory scoring” model, under which a candidate passes the test with a scaled score lower than 41 on a particular section, so long as his or her total scaled score is at least 123.

A passing score on the CBEST is required for all public elementary and secondary school teachers in California. See Cal. Educ.Code §§ 44256, 44257, 44259. A passing score also is required for many nonteaching employees of the California public schools, including administrators, see id. § 44270, school counselors, see id. § 44266, and school librarians, see id. § 44269.

Since the CBEST’s inception, minority candidates have disproportionately received failing scores. The named Plaintiffs are three nonprofit organizations that represent the interests of minority educators, and eight individual minority candidates. They brought this action against the State of California and the CCTC to challenge the validity of the test under Title VI and Title VII, on behalf of themselves and all others similarly situated. The district court certified the following class:

All Latinos, African-Americans and Asians who have sought or are seeking California public school credentials and certificated positions who have been, are being, or will be adversely affected in their ability to obtain credentials and certificated positions by [CBEST] results.

In their complaint, Plaintiffs sought to enjoin the use of the CBEST, alleging that the test has a disproportionate, adverse impact on minority candidates and that Defendants have failed to adopt screening procedures with a less adverse impact.

The parties filed cross-motions for summary judgment, addressing the applicability of Titles VI and VII. In August 1993, the district court granted partial summary judgment to Plaintiffs, concluding that both Titles VI and VII apply to the CBEST. See Association of Mexican-American Educators v. California, 836 F.Supp. 1534 (N.D.Cal.1993) (“AMAE I ”). Following a bench trial, the district court held that: (1) Plaintiffs had demonstrated that the CBEST has a disparate impact on minorities; (2) the studies that were submitted at trial demonstrated that the test was a valid measure of job-related skills; (3) the level at which the passing scores were set reflected reasonable professional judgments about minimum levels of basic knowledge; and (4) Plaintiffs had failed to demonstrate that other, equally effective screening devices existed. The court entered a judgment in Defendants’ favor. See Association of Mexican-American Educators v. California, 937 F.Supp. 1397 (N.D.Cal.1996) (“AMAEII”).

Defendants then presented the district court with a bill for taxable costs pursuant to Federal Rule of Civil Procedure 54(d)(1). That cost bill totaled $216,443.67. In an order dated February 12, 1997, the district court denied the cost bill in its entirety.

On appeal, Plaintiffs argue (1) that the district court erred in concluding, after trial, that the CBEST was validated properly and (2) that the court violated Federal Rule of Evidence 706 by relying on the advice of an expert who was not subject to cross-examination and did not prepare a report. Defendants cross-appeal with respect to the district court’s conclusions, on summary judgment, that Titles VI and VII apply. Defendants also appeal from the district court’s order denying costs.²

DISCUSSION

I. Title VII

A. Title VII Applies to the CBEST 3

Defendants appeal from the district court’s summary judgment in favor of Plaintiffs on the issue of the applicability of Titles VI and VII. We review de novo the district court’s grant of summary judgment. See Robi v. Reed, 173 F.3d 736, 739 (9th Cir.), cert. denied, — U.S.-, 120 S.Ct. 375, 145 L.Ed.2d 293 (1999).

As a threshold matter, we note that, because we ultimately hold that the CBEST was validated properly (see Part 1.B., below), we could decline to decide whether Titles VI and VII apply. Were we to do so, we simply would assume for the sake of argument that the statutes apply and move immediately to the question of validation. Although that might appear to be an expedient approach, we decline to follow it for three reasons. First, as a matter of logic, the applicability of Title VI or Title VII is a predicate to any discussion of validation. Validation would not be required, and indeed would not even be relevant, if neither Title VI nor Title VII applies. Second, as a matter of fairness, these parties deserve an answer not only to the bare question of who wins this case, but also to the underlying question of the applicability of federal civil rights law to the CBEST. The state, in particular, has proceeded for years on the assumption that those laws apply to its administration of the CBEST and has expended considerable effort and expense in attempting to comply with federal law in this area. If that effort was unnecessary, the state deserves to know, so that it may act accordingly in the future. Third, as a matter of judicial economy, our answer to the statutory question can avoid future litigation by other parties. We turn, then, to a discussion of Title VII’s application.

Title VTI of the Civil Rights Act of 1964 provides:

It shall be an unlawful employment practice for an employer-

(1) to fail or refuse to hire or to discharge any individual, or otherwise to discriminate against any individual with respect to his compensation, terms, conditions, or privileges of employment, because of such individual’s race, color, religion, sex, or national origin[J

42 U.S.C. § 2000e-2(a)(l). Title VII applies “to governmental and private employers alike.” Dothard v. Rawlinson, 433 , 332 n. 14, 97 S.Ct. 2720, 53 L.Ed.2d 786 (1977).

Plaintiffs and Defendants do not have a direct employment relationship. Rather, Plaintiffs are employees and potential employees of individual school districts in California. That fact does not end our inquiry, however. A direct employment relationship is not a prerequisite to Title VII liability. Although “there must be some connection with an employment relationship for Title VII protections to apply,” that “connection with employment need not necessarily be direct.” Lutcher v. Musicians Union Local 47, 633 F.2d 880, 883 (9th Cir.1980).

Among other things, we have held that an entity that is not the direct employer of a Title VII plaintiff nevertheless may be liable if it “ ‘interferes with an individual’s employment opportunities with another employer.’ ” Gomez v. Alexian Bros. Hosp., 698 F.2d 1019, 1021 (9th Cir.1983) (quoting Lutcher, 633 F.2d at 883 n. 3). In Gomez, we held that the defendant hospital could be held liable under Title VII for its discriminatory treatment of the plaintiff, notwithstanding the fact that the plaintiff was employed by a third party, if the defendant had interfered with the plaintiffs employment by that third party. See id. at 1021.

In so holding, we followed the opinion of the District of Columbia Circuit in Sibley Memorial Hospital v. Wilson, 488 F.2d 1338, 1340-41 (D.C.Cir.1973).⁴ In Sibley, the plaintiff was a male private-duty nurse. When a patient in the defendant hospital requested a private nurse, the hospital arranged through a registry service to have a private nurse provided. That nurse attended the patient at the hospital, but was paid directly by the patient. The plaintiff alleged that the defendant allowed male nurses like him to attend male patients only, but allowed female nurses to attend both male and female patients. See id. at 1339-40.

The D.C. Circuit concluded that the defendant could be held liable under Title VII even though it was not the plaintiffs direct employer. The court reasoned that, although the defendant did not employ the plaintiff, it exercised considerable power over his ability to form employment relationships with third parties. The court noted that Congress intended, through Title VII, to prohibit entities that possessed such power from “foreelos[ing], on invidious grounds, access by any individual to employment opportunities otherwise available to him.” Id. at 1341. The court further stated:

To permit a covered employer to exploit circumstances particularly affording it the capability of discriminatorily interfering with an individual’s employment opportunities with another employer, while it could not do so with respect to employment in its own service, would be to condone continued use of the very criteria for employment that Congress has prohibited.

Id. Finally, the court held that the defendant’s control over the premises on which the plaintiff provided his services, as well as its control over the plaintiffs access to patients, created a “highly visible nexus with the creation and continuance of direct employment relationships between third parties” that brought the defendant’s actions within the scope of Title VII. Id. at 1342.

The D.C. Circuit’s holding in Sibley was rooted in the text of Title VII. The Court reasoned that, although Title VII applies to “employees,” Congress extended the protections of the statute to “any individual” who suffers discrimination: “nowhere are there words of limitation that restrict references in the Act to ‘any individual’ as comprehending only an employee of an employer.” Id. at 1341. As we did in Gomez, we agree that the D.C. Circuit’s interpretation of the statutory text is the proper one in view of “Congress’ directive to read Title VII broadly so as to best effectuate its remedial purposes.” Duffield v. Robertson Stephens & Co., 144 F.3d 1182, 1192 (9th Cir.), cert. denied, 525 U.S. 982, 119 S.Ct. 445, 142 L.Ed.2d 399 (1998).

We also note, as did the Sibley court, see 488 F.2d at 1342, that Congress explicitly made Title VII applicable outside the “direct employment” context by including employment agencies and labor organizations in the statute’s coverage. See 42 U.S.C. § 2000e-2(b), (c). In particular, employment agencies — defined as parties “regularly undertaking with or without compensation to procure employees for an employer or to procure for employees opportunities to work for an employer,” 42 U.S.C. § 2000e(c) — may not “refuse to refer ... or otherwise discriminate against” any individual under Title VII. 42 U.S.C. § 2000e-2(b). This provision of Title VII, like the others, applies to states. See Dothard, 433 U.S. at 332 n. 14, 97 S.Ct. 2720; Dumas v. Town of Mt. Vernon, 612 F.2d 974, 980 (5th Cir.1980), overruled on other grounds by Larkin v. Pullman-Standard Div., Pullman Inc., 854 F.2d 1549 (11th Cir.1988), rev’d sub nom. Pullman-Standard, Inc. v. Swint, 493 U.S. 929, 110 S.Ct. 316, 107 L.Ed.2d 307 (1989). We do not suggest that Defendants are, strictly speaking, an “employment agency” under Title VII, although they perform an analogous function. Rather, we mention this statutory provision as evidence that Congress intended to close any loopholes in Title VU’s coverage and to extend the statute’s coverage to entities with actual “[ejontrol over access to the job market,” Sibley, 488 F.2d at 1341, whether or not they are direct employers.

In concluding that Title VII applies in this case, the district court held that Defendants “interfere[dj” with Plaintiffs’ employment opportunities with local school districts in California by requiring, implementing, and administering the CBEST. See AMAE I, 836 F.Supp. at 1551. We agree.

Our conclusion is dictated by the peculiar degree of control that the State of California exercises over local school districts. In California, public schools are “a matter of statewide rather than local or municipal concern; their establishment, regulation and operation are covered by the [state] constitution and the state Legislature is given comprehensive powers in relation thereto.” Hall v. City of Taft, 47 Cal.2d 177, 179, 302 P.2d 574, 576 (1956). The California legislature “has plenary authority over the education of California’s youth.” San Francisco NAACP v. San Francisco Unified Sch. Dist., 484 F.Supp. 657, 662 (N.D.Cal.1979). It is ‘Veil settled that the California Constitution makes public education uniquely a fundamental concern of the state” and that “the degree of supervision ... retained by the State over the common school system is high indeed.” Butt v. California, 4 Cal.4th 668, 685, 689, 15 Cal.Rptr.2d 480, 842 P.2d 1240, 1251, 1254 (1992).

The state’s involvement is not limited to general legislative oversight but, rather, affects the day-to-day operations of local public schools. “Unlike most states, California school districts have budgets that are controlled and funded by the state government rather than the local districts.” Belanger v. Madera Unified Sch. Dist., 963 F.2d 248, 251 (9th Cir.1992). As the California Supreme Court noted in Butt, California statutes regulate “district organization, elections, and governance; educational programs, instructional materials, and proficiency testing; sex discrimination and affirmative action; admission standards; compulsory attendance; school facilities; rights and responsibilities of students and parents; holidays; school health, safety, and nutrition; teacher credentialing and certification; rights and duties of public school employees; and the pension system for public school teachers.” Butt, 4 Cal.4th at 689, 15 Cal.Rptr.2d 480, 842 P.2d at 1254 (citations omitted). The state also “dictates when students may be expelled or suspended, and ... exerts control over the textbooks that are used in public schools.” Belanger, 963 F.2d at 253 (citations omitted).

Indeed, the state is so entangled with the operation of California’s local school districts that individual districts are treated as “state agencies” for purposes of the Eleventh Amendment. See Freeman v. Oakland Unified Sch. Dist., 179 F.3d 846, 846 (9th Cir.1999). The fact that the districts are entitled to assert Eleventh Amendment immunity underscores the state’s unusually high degree of involvement in the operation of local schools.

Against that background of “plenary” state control, we have no difficulty concluding that the State of California is in a theoretical and practical position to “interfere” with the employment decisions of local school districts. And by requiring, formulating, and administering the CBEST, the state has “interfered” to a degree sufficient to bring it within the reach of Title VII. Through the CBEST, the state has created a limited list of candidates from which local public school districts may hire. Private schools may hire candidates who have not passed the CBEST; but California’s public schools, which are under the state’s control in almost every aspect of their operations, may not. Thus, in addition to controlling local districts’ budgets and textbooks and regulating the duties of public school employees, the state dictates whom the districts may and may not hire. That degree of control over districts’ hiring decisions subjects Defendants to the coverage of Title VII in this case.

The relationship between the State of California and California’s local school dis-triets is analogous to the relationship between a corporate parent and its wholly owned subsidiaries. “In the absence of special circumstances, a parent corporation is not liable for the Title VII violations of its wholly owned subsidiary.” Watson v. Gulf & W. Indus., 650 F.2d 990, 993 (9th Cir.1981). In Watson, this court held that the parent corporation was not subject to Title VII because the case presented no “special circumstances.” Id. But the court went on to explain that, “[i]f there was any evidence that [the parent] participated in or influenced the employment policies of [the subsidiary], ... then we would be presented with a very different case.” Id. Ours is that “very different case.” The “parent” state has participated extensively in, and influenced, the employment policies and practices of the “subsidiary” local school districts; therefore, the state is covered by Title VII.

Defendants contend, however, that they are not subject to Title VII because the CBEST is merely a licensing examination. The administration of such examinations, they argue, is not covered by Title VII. As support for that argument, they cite several cases that have held that governmental agencies are not subject to Title VII with regard to their licensing activities. See Haddock v. Board of Dental Exam’rs, 777 F.2d 462 (9th Cir.1985); Fields v. Hallsville Indep. Sch. Dist., 906 F.2d 1017 (5th Cir.1990); George v. New Jersey Bd. of Veterinary Med. Exam’rs, 794 F.2d 113 (3d Cir.1986).⁵

The cases on which Defendants rely are not controlling for two reasons. First, the state’s high level of involvement in the operation of local public schools distinguishes this case from those that Defendants cite. In those cases, licensing was the entire connection between the plaintiffs and the defendants; here, the CBEST is but one aspect of pervasive state control. Second, the CBEST is not merely an ordinary licensing examination; it applies only to public school employees. In other words, the State of California is acting pursuant to its proprietary, as well as its police, power.

There is no overarching “licensing” exception to Title VII. The cases that Defendants cite stand for a related but narrower proposition — that Title VII does not apply when the only connection among the licensing agency, the plaintiff, and the universe of prospective employers is the agency’s implementation of a general licensing examination. In such cases, to borrow the words of the Sibley court, the agency does not have a “highly visible nexus with the creation and continuance of direct employment relationships between third parties,” such as would subject it to Title VII under an “interference” theory. Sibley, 488 F.2d at 1342.

In Haddock, the plaintiff conceded that his only connection to the defendant Board of Dental Examiners was that the Board had given him an examination that he failed. This court concluded that such a connection, by itself, was insufficient to subject the Board to Title VII liability. See 777 F.2d at 464. The plaintiff apparently did not argue that the Board had “interfered” with his employment under the principle adopted in Gomez and Sibley; if he did, the opinion does not mention it.

In Fields, the plaintiffs argued that the defendants had violated Title VII through their administration of the Texas Examination for Current Administrators and Teachers (TECAT), a compulsory certification examination. See 906 F.2d at 1019. The district court granted summary judgment in favor of the state defendants, concluding that they did not have an employment relationship with the plaintiffs. On appeal, the plaintiffs challenged that conclusion, arguing that the state defendants actually controlled their employment even though the plaintiffs nominally were employed by local districts. In rejecting that argument, the Fifth Circuit noted that “[t]he only evidence presented by [the plaintiffs] suggesting control is the Texas State Board of Education’s administration of the TECAT exam and its ability to decertify teachers who fail the exam.” Id. (emphasis in original). Of particular relevance to this case is footnote three of the opinion, which reads:

In a footnote in [the plaintiffs’] brief on appeal, they present evidence regarding state funding of facilities, payment of salaries and selection of textbooks. As this evidence was not before the district court, it is not part of the summary judgment record on appeal.

Id. at 1019 n. 3. The court suggested that the outcome might be different if there were such evidence of the state’s right to control the work of the teachers. See id. at 1019-20.

Finally, in George, the plaintiff alleged that the defendant Board of Veterinary Medical Examiners had violated Title VII by administering a licensing examination that discriminated against him on the basis of national origin. See 794 F.2d at 114. The Third Circuit affirmed the district court’s dismissal of the plaintiffs action. The court distinguished Sibley, on which the plaintiff had relied, stating:

In the Sibley Memorial Hospital case the relationship of the hospital to the employment by its patients of private duty nurses secured for them by the hospital was very close, whereas in the present case there was nothing even remotely resembling an employer-employee relationship between the Board and the plaintiff.

Id.

To summarize, the circumstances here demonstrate a level of control and interference far greater than that in the “mere licensing” cases on which Defendants rely. The State of California exerts a high degree of control over the operation of local public school districts. That control is evidenced both by the record and by California law.

Defendants cite George for the further proposition that state licensing examinations are acts of state police power, to which Title VII does not apply. But in George, the Board was acting only pursuant to the state’s police power to protect the public from incompetent veterinarians and was not attempting to control the hiring practices of, or the performance of work for, any specific employer. By contrast, the CBEST does not apply across-the-board to all who wish to teach in California, as (for example) a veterinary licensing examination applies to all who wish to ■practice the profession of veterinary medicine within the state’s borders. Rather, the CBEST applies only to those who wish to teach for the public school system — a system over which the State of California exerts plenary control, including regulation of employees’ duties.

We conclude, therefore, that administration of the CBEST is not solely an exercise of the state’s police power. Rather, it is an exercise of both the state’s police power and its proprietary power; and it is the exercise of proprietary power that subjects the state to the coverage of Title VII in this case.

We hold that the CBEST examination is subject to the provisions of Title VII. We turn next to the question whether the CBEST violates the provisions of that Act.

B. The District Court Did Not Clearly Err in Concluding that the CBEST Was Properly Validated.⁶

“[Discriminatory tests are impermissible unless shown, by professionally acceptable methods, to be predictive of or significantly correlated with important elements of work behavior which comprise or are relevant to the job or jobs for which candidates are being evaluated.” Albemarle Paper Co. v. Moody, 422 U.S. 405, 431, 95 S.Ct. 2362, 45 L.Ed.2d 280 (1975). In evaluating employment tests that are alleged to have a racially disparate impact, we first consider whether the plaintiff has established a prima facie case by demonstrating that the test causes a disparate impact on the basis of race. Here, the district court concluded that Plaintiffs had established a prima facie case. See AMAE II, 937 F.Supp. at 1403. Defendants do not challenge that conclusion on appeal.

Because Plaintiffs have established a prima facie case, the burden shifts to Defendants to demonstrate that the CBEST was validated properly.⁷ See Albemarle Paper, 422 U.S. at 425, 95 S.Ct. 2362. In its detailed and careful opinion, the district court concluded that Defendants had met their burden and that the test had been validated properly based on three studies: (1) the 1982 Wheeler and Elias study; (2) the 1985 Practitioners’ Review; and (3) the 1995 Lundquist study. See AMAE II, 937 F.Supp. at 1411. Plaintiffs challenge that conclusion.

Although this court has not discussed in detail the appropriate standard of review for a district court’s ruling on test validation, we have applied the “clearly erroneous” standard. Clady v. County of Los Angeles, 770 F.2d 1421, 1434 (9th Cir.1985). The other circuits that have addressed this issue likewise have applied the “clearly erroneous” standard. See, e.g., Melendez v. Illinois Bell Tel. Co., 79 F.3d 661, 669 (7th Cir.1996); Bernard v. Gulf Oil Corp., 890 F.2d 735, 743 (5th Cir.1989); Hamer v. City of Atlanta, 872 F.2d 1521, 1526 (11th Cir.1989). The question whether a test has been validated properly is primarily a factual question, which depends on underlying factual determinations regarding the content and reliability of the validation studies that a defendant utilized. Consistent with Clady, we review for clear error the district court’s determination in this case that the CBEST was validated properly.

To demonstrate that the CBEST was validated properly, Defendants are required to “show that it has ‘a manifest relationship to the employment in question.’ ” Clady, 770 F.2d at 1427 (quoting Griggs v. Duke Power Co., 401 U.S. 424, 432, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971)). In cases in which a scored test, like this one, is challenged, we require that the test be “job related” — that is, “that it actually measures skills, knowledge, or ability required for successful performance of the job.” Contreras v. City of Los Angeles, 656 F.2d 1267, 1271 (9th Cir.1981). In making a determination about job-relatedness, we follow a three-step approach:

The employer must first specify the particular trait or characteristic which the selection device is being used to identify or measure. The employer must then determine that the particular trait or characteristic is an important element of work behavior. Finally, the employer must demonstrate by “professionally acceptable methods” that the selection device is “predictive of or significantly correlated” with the element of work behavior identified in the second step.

Craig v. County of Los Angeles, 626 F.2d 659, 662 (9th Cir.1980) (quoting Albemarle Paper, 422 U.S. at 431, 95 S.Ct. 2362).⁸

We will analyze each of those three steps in turn. In addition, we will consider Plaintiffs’ argument that the passing score on the writing component of the CBEST is set too high.⁹

1. Specific Traits or Characteristics

The first step of our inquiry is to identify the trait or characteristic that the test is designed to measure. See Craig, 626 F.2d at 662. Here, the district court found that the test was being used to measure “basic skills in reading, writing, and mathematics,” AMAH II, 937 F.Supp. at 1411, and Plaintiffs do not dispute that finding.

2. Important Elements of Work Behavior

Next, we consider whether basic skills in reading, writing, and mathematics are “important elements] of work behavior,” Craig, 626 F.2d at 662, for the public school jobs for which the test is required. The district court found that the tested skills were important to the jobs at issue. See AMAH II, 937 F.Supp. at 1419. Plaintiffs challenge that finding on three grounds. First, they argue that the 1985 Practitioners’ Review failed to identify any particular work behaviors or job duties and thus could not be used to assess whether the CBEST measured important elements of work behavior. Second, they argue that Lundquist’s 1995 study failed to distinguish “important” skills from skills that are less important. Third, they argue that Defendants failed to demonstrate that the CBEST is job-related for the particular positions for which it is required. We address each of those arguments in turn.

Plaintiffs first argue that the 1985 Practitioners’ Review, conducted by Dr. Richard Watkins, was inadequate because it failed to identify specific job duties to which the CBEST skills could be correlated. We conclude that the district court did not clearly err in finding that the 1985 study adequately identified the “element[s] of work behavior,” Craig, 626 F.2d at 662, that the CBEST is designed to measure.

The district court found that the Review comprised the “pooled judgments” of knowledgeable persons, such as incumbents in the jobs, “about the relevance of the skills tested on the CBEST to the jobs for which it is required, an appropriate form of a job analysis under the professional standards of the time.” AMAE II, 937 F.Supp. at 1419. Specifically, the Practitioners’ Review consulted 234 teachers, administrators, and other public school employees, 36 percent of whom were members of minority groups. See id. at 1413. “The participants took part in nine review panels, in which they judged the relevance of both the skills assessed by the CBEST and the test items themselves.” Id. They were asked to rate how relevant each of the CBEST skills would be to the work of four groups: (1) elementary school teachers; (2) secondary school teachers; (3) librarians, counselors, and attendance officers; and (4) school administrators. See id. The possible ratings ranged from “not relevant” to “very relevant.” Id.

Thus, the Practitioners’ Review was designed to learn from teachers, administrators, and other school employees the categories of skills that they considered relevant to their own jobs. The skills measured by the study tracked the categories of skills measured by the CBEST, and the skills were described in some detail on the rating forms used by the panel members.. For example, the broad skill category “Mathematical concepts and relationships” was further described as follows:

Questions in this category test the understanding of basic concepts, such as the meaning of certain terms (area, for example), order among numbers, relationships shown by graphs, elementary probability, and the like. Questions in this category may be from arithmetic, algebra, or elementary geometry.

The study’s participants were guided by detailed instructions relating to each skill category and were told to rank the importance of each skill for both teaching and nonteaehing jobs. The study therefore satisfies the requirement from Craig that the employer determine whether a “specific trait or characteristic is an important element of work behavior.” Craig, 626 F.2d at 662. The district court did not clearly err in concluding that the 1985 Practitioners’ Review was “an appropriate form of a job analysis under the professional standards of the time.” AMAE II, 937 F.Supp. at 1419.

We next consider Plaintiffs second challenge under the “important elements” prong of Craig. Plaintiffs do not challenge the 1995 Lundquist study’s methodology for identifying job-related skills. They do, however, challenge that study’s method for determining which skills are “important” to particular jobs.

Dr. Lundquist polled experts and interviewed and observed educators in order to develop a list of activities and skills used by educators. See AMAE II, 937 F.Supp. at 1414. She then polled 1,330 teachers and administrators, asking them to rate the importance of those activities and skills on a four-point scale from 0 (“not applicable”) to 3 (“critical”). Activities and skills were retained only if “at least 80 percent of the survey respondents rated the activity or skill as applicable to the job and the mean importance rating was 1.5 or higher.” Id. at 1414 (emphasis in original). Applying those standards led to elimination of a number of the activities and skills from Dr. Lundquist’s list. After conducting additional studies, which are described in detail in the district court’s opinion, see id. at 1415-17, Dr. Lundquist then formulated new specifications for all three sections of the test. In response to those specifications, Defendants revised the CBEST before they administered the August 1995 test.

Plaintiffs focus on the fact that Dr. Lundquist retained activities and skills on her list if they received a “mean importance rating” of 1.5 on a scale that designated 2 as “important” and 1 as “minor.” By using a mean rating of 1.5, Plaintiffs argue, Dr. Lundquist retained skills and activities that were rated as “less than important” by the study’s participants. Therefore, their argument proceeds, the study violated the requirement from Craig that only “important” work skills be measured.

The district court rejected Plaintiffs’ argument, finding that “Dr. Lundquist’s decisions reflect manifestly reasonable professional judgments.... With respect to the 1.5 mean, as Dr. Lundquist testified at trial, a 1.5 rounds up to 2.0. It must be remembered that the mean rating of 1.5 was coupled with an 80 percent endorsement criterion, which is quite stringent.” AMAE II, 937 F.Supp. at 1418 n. 35.

We agree that it is theoretically possible to imagine a circumstance that illustrates Plaintiffs’ concerns on this point. For example, suppose that 80 percent of the study’s participants agreed that a particular skill was relevant, but 75 percent of them rated that skill’s importance as “minor.” If the remaining 25 percent rated the skill as “critical,” then the skill would be retained despite the fact that a majority of the study’s participants rated its importance as “minor.” Although that scenario is possible, such a skewed distribution of responses is unlikely. Plaintiffs present only a theoretical possibility that such “highly relevant but unimportant” skills remained on Dr. Lundquist’s list. Further, as the district court noted, Dr. Lund-quist conducted additional “importance” reviews of the mathematics section of the test.

Validation studies “are by their nature difficult, expensive, time consuming and rarely, if ever, free of error.” Cleghorn v. Herrington, 813 F.2d 992, 996 (9th Cir.1987). Plaintiffs’ argument demonstrates, at most, that Dr. Lundquist’s study may not be totally free of error. But the argument does not persuade us that the district court clearly erred in relying on Dr. Lund-quist’s study.

Finally, Plaintiffs argue that Defendants failed to conduct job-specific studies to determine that the CBEST is “job related for the position^] in question.” 42 U.S.C. § 2000e-2(k)(l)(A)(i). The CBEST is not intended to measure all the skills that are relevant to all the jobs for which it is required. (Indeed, it does not purport to measure all the skills of any of the jobs for which it is required.) Rather, the CBEST is intended to establish only a minimum level of competence in three areas of basic educational skills. The question is whether the validation studies in this case have satisfied the requirement that those skills be “job related” for all the positions in question. The district court found that the validation studies adequately analyzed the CBEST in terms of both the teaching and nonteaching jobs for which the test is required. See AMAE II, 937 F.Supp. at 1418-19. The district court did not clearly err in so finding.

Both the 1985 and the 1995 validation studies contained adequate consideration of the specific positions for which the CBEST is required. The 1985 Practitioners’ Review defined the positions that it analyzed as (1) elementary school teachers, (2) secondary school teachers, (3) librarians, counselors, and attendance officers, and (4) school administrators. All participants in the study were asked to judge the relevance of the CBEST skills by category for those jobs. Because the study’s participants were asked to determine the relevance of the basic skills measured by the CBEST to the disparate groups of positions for which the test is required, we cannot say that the district court clearly erred in finding the job analysis in the Practitioners’ Review to be sufficiently specific and particularized.

The 1995 Lundquist study, as noted, identified job activities through observation, interviews, and reviews of specialized literature. See id. at 1414. Dr. Lund-quist then pared her list of job skills and activities through surveys of educators and arrived at a list of “common skill requirements” that were relevant for both teachers and administrators. See id. at 1414-15. Her study reports:

Basic skill ratings were examined for administrators to determine if the same skill sets applied to both teacher and administrator jobs. Results showed all but one skill item (a math item) retained for teachers also applied to the administrator group. Thus, the basic skill requirements identified for teachers were found to be job-related for administrators as well, and the same test specifications may be used to test basic skills for teachers and administrators.

Dr. Lundquist’s study classified jobs for which the CBEST is required as either “teacher” or “administrator” and determined that the CBEST was valid for both groups of positions. Accordingly, the 1995 study considered the validity of the CBEST across the range of jobs for which the test is required. The district court accepted the study’s conclusions and found that the CBEST had been validated adequately “with respect to teaching and non-teaching jobs.” Id. at 1418. On this record, that finding is not clearly erroneous.

In sum, we hold that the district court did not clearly err in finding that the skills measured by the CBEST are “important element[s] of work behavior” with regard to the jobs for which the test is required. Craig, 626 F.2d at 662.

3. Actual Measurement of Skills

The final step in this court’s three-step analysis from Craig is to determine whether Defendants have demonstrated by “professionally acceptable methods that the selection device is predictive of or significantly correlated with the element of work behavior” that it is designed to measure. See id. (internal quotation marks omitted). The district court concluded that “the CBEST actually measures ... basic skills [in reading, writing, and mathematics].” AMAE II, 937 F.Supp. at 1411. Plaintiffs claim that the district court simply accepted the “facial” validity of the CBEST without any evidence that it actually measures the basic skills that it purports to test.

This court held in Contreras that “a key requirement of [the] third step [from Craig ], a requirement essential to proof of job relatedness generally, is that the validation method be professionally acceptable.” 656 F.2d at 1282. Here, there is evidence in the record from an expert, Dr. William A. Mehrens, that supports the district court’s findings on this issue. Dr. Mehrens reported:

ETS [Educational Testing Service] personnel wrote some of the original items and assisted the test development committees in writing other items. ETS is well known and respected as a developer of standardized tests. They have well trained item writers and an impressive internal set of guidelines they follow with respect to item writing.

When asked whether “the CBEST development [was] appropriate with respect to writing and evaluating the items,” he reported:

It has been. Many of the items came from an existing ETS pool. Others were written specifically for CBEST by members of the test development committee in concert with ETS test development specialists. The individuals on the committees worked with specialists from ETS to further develop and define the content specifications, to review an existing ETS pool test item, to write new test items, and to review the items submitted by fellow committee members. In addition, the committees studied all of the data from the field testing, made recommendations for revisions as they felt necessary, and reviewed all final test results.

There also is additional evidence in the record that the test questions were “matched” to the skills that they were intended to measure. The district court referred to the “Curriculum Matching Project, in which two ETS employees ... matched CBEST test specifications to material found in textbooks purportedly used in the California public schools.” AMAE II, 937 F.Supp. at 1412 n. 21. The district court was somewhat critical of this study, but noted that “the study did support the overall conclusion that the kinds of skills tested on the CBEST can be found in elementary and secondary school textbooks.” Id.

In short, there is evidence — even if not overwhelming evidence — that the development and evaluation of the CBEST were appropriate and that the test measures the types of skills that it was designed to measure. We therefore hold that the district court did not clearly err in concluding that the test questions had been shown by professionally acceptable methods to be “predictive of or significantly correlated with the element of work behavior” that they were designed to measure. Craig, 626 F.2d at 662 (internal quotation marks omitted).

In sum, we hold that the district court did not clearly err in concluding that the CBEST was validated properly.¹⁰

4. Standards for Passing Scores

Plaintiffs also argue that the 12-out-of-16 passing score on the writing section of the CBEST is too high. According to Plaintiffs, the 1982 Wheeler and Elias study demonstrates that the proper passing score is 9 or 10 out of 16.

An employer is not required to validate separately the selection of particular passing scores on an employment test. See id. at 665. Rather, the EEOC’s Guidelines more generally provide: ‘Where cutoff scores are used, they should normally be set so as to be reasonable and consistent with normal expectations of acceptable proficiency within the work force.” 29 C.F.R. § 1607.5(H). This court previously has applied that standard. See, e.g., Craig, 626 F.2d at 665. In analyzing the Guidelines’ scoring requirement, the Second Circuit has stated that an employer “might establish a valid cutoff score by using a professional estimate of the requisite ability levels, or, at the very least, by analyzing the test results to locate a logical ‘break-point’ in the distribution of scores.” Guardians Ass’n of New York City Police Dep’t, Inc. v. Civil Serv. Comm’n of New York, 630 F.2d 79, 105 (2d Cir.1980).

Here, the district court found that “the passing scores on the CBEST reflect reasonable judgments about the minimum level of basic skills competence that should be required of teachers.” AMAE II, 937 F.Supp. at 1420. The evidence before the court revealed that the California Superintendent of Public Instruction, who was responsible for establishing the cutoff scores, relied on polling data created as part of the Wheeler and Elias study in setting the cutoff for the writing section of the test. As part of that study, 44 readers reviewed approximately 6,800 CBEST essays and made recommendations regarding the cutoff between passing and failing scores. The readers unanimously agreed that a raw score of 12 out of 16 was a “passing” score. Approximately 80 percent of the readers agreed that a score of 11 out of 16 could be a “passing” score. On that basis, the Superintendent established a passing score of 12 out of 16, with an absolute minimum of 11 out of 16 under the “compensatory scoring” system.

Those cutoff scores represent a “logical breakpoint” between passing and failing scores. Plaintiffs argue that the breakpoint should have been set at 9 or 10 out of 16, because a majority of the readers opined that 10 out of 16 was a “passing” score. But the Superintendent was not required to set the score at the lowest level that a majority of the readers considered to be “passing.” Rather, he was required to set a cutoff that was logical, reasonable, and consistent with the data before him. He chose to set the cutoff at a level that all the readers agreed was “passing,” and to set an absolute minimum at a level that 80 percent of the readers thought was “passing.” The district court found that the Superintendent’s decision to set the cutoff score at that level was consistent with the EEOC’s Guidelines. We conclude that the district court did not clearly err in so finding.

11. Title VI¹¹

Because we have concluded that Title VII applies to the CBEST, we need not consider whether Title VI also applies. See 42 U.S.C. § 2000d. Plaintiffs proceed on a disparate impact theory under both Title VI and Title VII. Thus, our discussion of the merits under Title VII, and of the validation of the CBEST, would resolve Plaintiffs’ claims on the merits under Title VI as well. Accordingly, we decline to issue an advisory opinion on the applicability of Title VI.

III. The District Court’s Appointment of a Technical Advisor¹²

Plaintiffs also argue that the proceedings were tainted by the influence of Dr. Stephen Klein. Dr. Klein was appointed by the district court as a technical advisor, but was not called as an expert witness, was not subject to cross-examination, and did not furnish an expert’s report.

In those rare cases in which outside technical expertise would be helpful to a district court, the court may appoint a technical advisor like Dr. Klein. See Reilly v. United States, 863 F.2d 149, 166 (1st Cir.1988); see also General Elec. Co. v. Joiner, 522 U.S. 136, 149, 118 S.Ct. 512, 139 L.Ed.2d 508 (1997) (Breyer, J., concurring) (endorsing the appointment of special masters and specially trained law clerks to assist district courts with scientific or technical evidence). The court appointed Dr. Klein in an order that specifically identified him as a technical advisor.

Plaintiffs argue that the court committed legal error under Federal Rule of Evidence 706(a) by neither requiring Dr. Klein to submit a report nor allowing him to be cross-examined. The short answer to Plaintiffs’ argument is that Rule 706 applies to court-appointed expert witnesses, but not to technical advisors like Dr. Klein. See Reilly, 863 F.2d at 155.

At one point in the proceedings, the district court told the parties that it intended to call Dr. Klein to testify near the conclusion of the trial and that the court would permit cross-examination at that time. Had the court called him, Dr. Klein would have testified as an expert witness, and Federal Rule of Evidence 706 would have applied. But the court never called Dr. Klein, and there is no indication in the record that the court relied on Dr. Klein as a source of evidence. On this record, Dr. Klein acted only as a court-appointed technical advisor, and the district court did not err in refusing to allow cross-examination or to require an expert’s report.

In his dissent, Judge Tashima agrees with us that district courts retain inherent authority to appoint technical advisors in appropriate cases; that this was an appropriate case for a technical advisor; that Rule 706(a) does not apply; and that we are reviewing for an abuse of discretion.

His disagreement rests on his analysis of how we should respond to the relative paucity of information in the record about Dr. Klein’s interaction with the district court. In our view, the absence of any evidence even suggesting an impropriety on the part of the district court militates against a conclusion that the court abused its discretion. Although it is at least possible, as Judge Tashima suggests, that “Dr. Klein may have impermis-sibly influenced the court’s ultimate finding,” diss. op. at 611, we instead assume that the district court did its job properly when we lack evidence to the contrary.¹³

Judge Tashima also proposes a list of procedures for district courts to follow when appointing technical advisors. Even assuming that those procedures are appropriate, the district court did not have the benefit of Judge Tashima’s dissent before this trial, and we will not fault the court for failing to foresee his recommendations. We are not willing to find an abuse of discretion and to undo this entire trial because the district court did not follow a set of guidelines that are required nowhere in the rules or relevant case law.

IV. The District Court’s Refusal to Award Costs¹⁴

Defendants appeal from the district court’s order denying their cost bill in the amount of $216,443.67. Federal Rule of Civil Procedure 54(d)(1) provides that “costs other than attorneys’ fees shall be allowed as of course to the prevailing party unless the court otherwise directs.” By its terms, the rule creates a presumption in favor of awarding costs to a prevailing party, but vests in the district court discretion to refuse to award costs. See National Info. Servs., Inc. v. TRW, Inc., 51 F.3d 1470, 1471 (9th Cir.1995).

That discretion is not unlimited. A district court must “specify reasons” for its refusal to award costs. Subscription Television, Inc. v. Southern Cal. Theatre Owners Ass’n, 576 F.2d 230, 234 (9th Cir. 1978). On appeal, we determine whether the reasons that the district court has specified are appropriate and whether, considering those reasons, the court abused its discretion in denying costs. See National Info. Servs., 51 F.3d at 1471-72.

In past cases, this court has approved the following reasons for refusing to award costs to a prevailing party: the losing party’s limited financial resources, see National Org. for Women v. Bank of Cal., 680 F.2d 1291, 1294 (9th Cir.1982); see also Wrighten v. Metropolitan Hosps., Inc., 726 F.2d 1346, 1358 (9th Cir.1984); Moore v. Hughes Helicopters, Inc., 708 F.2d 475, 486 (9th Cir.1983); and misconduct on the part of the prevailing party, see National Info. Servs., 51 F.3d at 1472. Further, in Stanley v. University of Southern California, 178 F.3d 1069, 1079-80 (9th Cir.), cert. denied, — U.S.-, 120 S.Ct. 533, 145 L.Ed.2d 413 (1999), we held that the district court abused its discretion in denying a losing civil rights plaintiffs motion to re-tax costs without considering (1) the plaintiffs limited financial resources; and (2) “the chilling effect of imposing such high costs on future civil rights litigants.”

Here, the district court gave four reasons for denying costs to Defendants: (1) the case “involvefs] issues of substantial public importance,” specifically “educational quality, interracial disparities in economic opportunity, and access to positions of social influence”; (2) there is great economic disparity between Plaintiffs, who are individuals and “small nonprofit educational organizations,” and the State of California; (3) the issues in the case are close and difficult;¹⁵ and (4) Plaintiffs’ case, although unsuccessful, had some merit, as evidenced by the 1995 modification of the CBEST to eliminate “higher order” mathematics questions.

Defendants argue that the district court’s reasons for denying costs were improper. According to Defendants, this court’s opinion in National Information Senices establishes that the only proper reason for denying costs to a prevailing party is to punish misconduct by that party. We disagree.

As noted, this court previously has held that district courts may consider other, nonpunitive reasons for denying costs to a prevailing party. National Information Senices does appear to suggest that such a denial is proper only as a means of punishing a prevailing but undeserving litigant. See also Zenith Ins. Co. v. Breslaw, 108 F.3d 205, 207-08 (9th Cir.1997) (following National Information Senices). But that suggestion is inconsistent with earlier opinions of this court, see, e.g., National Org. for Women, 680 F.2d at 1294, opinions that National Information Services, as a panel’s opinion, could not (and did not purport to) overrule.

More importantly, we see no basis for limiting district courts’ discretion in the manner that Defendants suggest. The rule itself contains no such limitation; it provides simply that costs shall be allowed to the prevailing party unless the district court “otherwise directs.” The requirement that district courts give reasons for denying costs flows logically from the presumption in favor of costs that is embodied in the text of the rule; if a district court wishes to depart from that presumption, it must explain why “so that the appellate court will be able to determine whether or not the trial court abused its discretion.” Subscription Television, 576 F.2d at 234. But the limitation on district courts’ discretion that Defendants advocate curtails that discretion in a manner and to a degree that are inappropriate in view of the broad wording of the rule. We now overrule National Information Services to the extent that it held that only misconduct may support the denial of costs to a prevailing party.

Federal Rule of Civil Procedure 54(d)(1) establishes that costs are to be awarded as a matter of course in the ordinary case. Our requirement that a district court give reasons for denying costs is, in essence, a requirement that the court explain why a case is not “ordinary” and why, in the circumstances, it would be inappropriate or inequitable to award costs. Misconduct on the part of the prevailing party is one factor that might render a case “extraordinary.” But it is not the only such factor. Here, the reasons that the district court gave for denying costs reflect what is clear at a glance: This is an extraordinary, and extraordinarily important, case. Plaintiffs are a group of individuals and nonprofit organizations. The record demonstrates that their resources are limited. They have brought an action that presents issues of the gravest public importance, and the action affects tens of thousands of Californians and the state’s public school system as a whole. The issues in the case are close and complex. Although Plaintiffs have not prevailed in this action, their claims are not without merit. Indeed, as the district court pointed out, Defendants substantially altered the CBEST during the pendency of this litigation. Finally, costs in this case are extraordinarily high. In keeping with our decision in Stanley, we note that divesting district courts of discretion to limit or to refuse such overwhelming costs in important, close, but ultimately unsuccessful civil rights cases like this one might have the regrettable effect of discouraging potential plaintiffs from bringing such cases at all.

We do not mean to suggest that the presumption in favor of awarding costs to prevailing parties does not apply to defendants in civil rights actions. Nor are we attempting to create an exhaustive list of “good reasons” for declining to award costs. We simply hold that the reasons that the district court gave for refusing to award costs in this case were appropriate under Rule 54(d)(1) and that, considering those reasons, the court did not abuse its discretion in refusing to award costs to Defendants.

CONCLUSION

For the reasons stated, we hold that Title VII applies to the CBEST; that the CBEST was validated properly; that the district court permissibly used a technical advisor; and that the district court did not abuse its discretion by refusing to award costs to Defendants. Accordingly, we affirm both the judgment in Defendants’ favor and the order denying them costs.

AFFIRMED.

1

. The CBEST is described in more detail in the district court’s opinion. See Association of Mexican-American Educators v. California, 937 F.Supp. 1397 (N.D.Cal.1996).

2

. This appeal originally was heard by a three-judge panel of this court; the court later agreed to rehear the case en banc and withdrew the panel’s opinion. See Association of Mexican-American Educators v. California, 195 F.3d 465 (9th Cir.1999), withdrawn, 208 F.3d 786 (9th Cir.2000).

3

. Chief Judge Hug, and Judges Schroeder, Reinhardt, Fernandez, Rymer, and Thomas join in this part of the majority opinion.

4

. Other cases following the "interference” model from Sibley include: Charlton v. Paramus Bd. of Educ., 25 F.3d 194, 202 (3d Cir.1994); Christopher v. Stouder Mem’l Hosp., 936 F.2d 870, 876-77 (6th Cir.1991); Pardazi v. Cullman Med. Ctr., 838 F.2d 1155, 1156 (11th Cir.1988); and Shehadeh v. Chesapeake & Potomac Tel. Co., 595 F.2d 711, 722 (D.C.Cir.1978).

5

. To the same effect, see also Woodard v. Virginia Bd. of Bar Exam’rs, 598 F.2d 1345 (4th Cir.1979).

6

. Chief Judge Hug, and Judges O’Scannlain, Fernandez, Rymer, and Kleinfeld join in this part of the majority opinion.

7

. In cases in which a defendant establishes that a test is validated properly, the burden shifts back to the plaintiff to show the existence of other selection devices that also would "serve the employer’s legitimate interest in efficient and trustworthy workmanship,” but that are not discriminatory. Albemarle Paper, 422 U.S. at 425, 95 S.Ct. 2362 (internal quotation marks omitted). Here, Plaintiffs' challenge is limited to the validation of the test; they do not attempt on appeal to meet their burden of demonstrating the existence of preferable selection devices, assuming that the CBEST is validated.

8

. Also relevant to our inquiry are the Equal Employment Opportunity Commission’s (''EEOC”) Uniform Guidelines on Employee Selection Procedures (“Guidelines”), which are codified at 29 C.F.R. pt. 1607. Although the Guidelines are not legally binding, they are “entitled to great deference.” Albemarle Paper, 422 U.S. at 431, 95 S.Ct. 2362 (internal quotation marks omitted). Failure to comply with the Guidelines, although not automatically fatal to an employment test, “diminishes the probative value of the defendants’ validation study.” Clady, 770 F.2d at 1430 (internal quotation marks omitted). The studies on which Defendants rely were content validity studies. See 29 C.F.R. § 1607.5. Such studies establish whether the content of a test approximates the knowledge, skills, or abilities that an applicant will use on the job. See 29 C.F.R. § 1607.14. "Evidence of the validity of a test or other selection procedure by a content validity study should consist of data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.” 29 C.F.R. § 1607.5(B).

9

. Our discussion of validation owes much to the original panel majority’s excellent treatment of these issues.

10

. We affirm the district court’s conclusion that the CBEST was validated properly based on the second and third studies: the 1985 Practitioners’ Review and the 1995 Lundquist study. Because we conclude that those tests adequately support the district court’s finding of job-relatedness, we do not discuss in detail the 1982 Wheeler and Elias study, on which the district court also relied. All three studies reviewed the same basic version of the CBEST: the version that was given between 1983 and August 1995, when the test was revised. There is no allegation that the test changed significantly during that period or that the test was invalid until 1985 but valid thereafter. Accordingly, it is sufficient for us to conclude that the test was validated properly by the 1985 and 1995 studies and, because we so conclude, we do not decide whether the 1982 study provides an additional source of validation.

11

. Chief Judge Hug, and Judges Schroeder, Reinhardt, O’Scannlain, Fernandez, Rymer, Kleinfeld, Thomas, and Gould join in this part of the majority opinion.

12

. Chief Judge Hug, and Judges Schroeder, Reinhardt, O’Scannlain, Fernandez, Rymer, Kleinfeld, Thomas, and Gould join in this part of the majority opinion.

13

. The present case is distinguishable from those that Judge Tashima cites, see dissenting op. at 619 n. 9. In all but the first and last of the cited cases, the lower tribunal failed to make findings of fact that were required by existing law. The first cited case was a death-penalty habeas corpus case in which the court remanded for an evidentiary hearing on the merits of several claims that had been dismissed, improperly, on procedural grounds. The panel's decision to remand was dictated by its application of the specific rules and procedures that govern habeas corpus in capital cases. In the last case, the district court wholly failed to address a potentially disposi-tive legal issue.

By contrast, here the district court had before it a nonhabeas civil case, in which it decided all issues presented and made all required findings of fact. The only question before us is whether the district court’s use of a technical advisor was an abuse of discretion.

14

. Chief Judge Hug, and Judges Schroeder, Reinhardt, Thomas, and Gould join in this part of the majority opinion.

15

. Although we have not previously approved that reason for denying costs, other circuits have. See Teague v. Bakker, 35 F.3d 978, 997 (4th Cir.1994); White & White, Inc. v. American Hosp. Supply Corp., 786 F.2d 728, 733 (6th Cir.1986). Other reasons for denying costs that circuit courts have approved include: the "nominal” or partial nature of the prevailing party’s recovery, see Richmond v. Southwire Co., 980 F.2d 518, 520 (8th Cir.1992); Howell Petroleum Corp. v. Samson Resources Co., 903 F.2d 778, 783 (10th Cir.1990); and the good faith of the losing party, see Teague, 35 F.3d at 997; White, 786 F.2d at 730. The Seventh Circuit also has suggested, in dictum, that the denial of costs might be appropriate in cases that present “landmark issues of national importance.” Popeil Bros. v. Schick Elec., Inc., 516 F.2d 772, 776 (7th Cir.1975); see also Delta Air Lines, Inc. v. Colbert, 692 F.2d 489, 490 (7th Cir.1982). That dictum is similar to the district court’s first reason for denying costs in this case, namely, the exceptional public importance of the issues presented.