Mr. Justice Lindsley
delivered the opinion of the Court.
On July 28, 1952, defendant in error, herein designated as plaintiff, filed his complaint in the nature of mandamus under Rule 106 (a) (2) R.C.P. Colo., in which he alleged the respective capacity of himself, the plaintiffs in error, herein designated as defendants, and complained that the civil service promotional examination taken by the respective assistant chiefs of the Fire Department of the City and County of Denver for the position of Deputy Chief of said Fire Department on May 12, 1951, was null and void, and that the certification of defendant Becker as first upon the eligible register, pursuant to said examination and his later appointment as Deputy Fire Chief, was illegal and void.
Defendants answered, denying the illegality of the examination and the illegal appointment of Becker as Deputy Fire Chief, asserted that plaintiff was guilty of laches, and defendant Becker asserted a special defense of estoppel, stating that he had changed his position in good faith in that he entered upon his duties of Deputy Fire Chief on or about June 12, 1951, and had continued in that position until the filing of the complaint herein.
The issues raised, the applicable provisions of the charter of the City and County of Denver, and the evidence supporting the contentions of the parties, are fully dealt with in the finding of facts and conclusions of law of the
able trial judge, whose “Findings (and Judgment are as follows:
“On this day, the Court now being sufficiently advised in the premises, rendered and caused to be entered of record herein the following
‘FINDINGS AND JUDGMENT’
“On March 18, 1951, the Civil Service Commission of the City and County of Denver announced that it would hold an examination on May 12, 1951, for the purpose of creating an eligible list for promotional appointment to Deputy Chief of the Fire Department. The announcement set forth the subjects and weights for the examination as follows:
Oral test 60%
Merit 20%
Seniority 20%
“The examination was held on the appointed day. Nine persons took the examination including the plaintiff Cassio Frazzini and the defendant Arthur J. Becker. After the examination, and based upon the grades given the applicants, the Commission issued an eligible list or register for the position of Deputy Chief, and the name of defendant Arthur J. Becker led all the rest. The name of plaintiff Frazzini was second on the list. The grade given to Becker was 90.6% and to Frazzini was 90.00%. In ascertaining these grades Becker was given five points for veteran’s preference and Frazzini ten points for veteran’s preference.
“Upon requisition from the Manager of Safety and excise, Becker was certified for appointment, and on June 12, 1951, he was appointed to the office of Deputy Chief of the Fire Department of Denver, which position he now holds under that appointment. Plaintiff, at the time of taking the examination and now, occupies the position of Assistant Chief of the Fire Department. He has been a member of the Denver Fire Department since 1931.
“The examination was wholly oral. No written tests were given the applicants. This is the first time a civil
service examination in Denver has been given without some part of it at least being in written form, and as stated above, the oral test was weighted at 60%.
“The oral examination was conducted by a board of five men, selected by the Commission. These men were A. J. Galli, a deputy fire chief from San Francisco; Edward Grenfell, chief of the fire department, Portland, Oregon; Harry Seligson, a professor from Denver University; W. P. Chase, personnel and research department, Lowry Field, Denver; and Earl L. Pinker, personnel director for Gates Rubber Company, Denver.
“Prior to the examination, each member of the board was given instructions in writing contained in four separate documents, identified here as Plaintiff’s Exhibits B, C, D and E.
“The oral examination consisted of two parts, called ‘The Group Oral’ and ‘The Individual Interview.’ At The Group Oral the candidates were seated around a table with identifying number cards at their places and the examiners occupied seats around the room. The examiners were instructed to change seats around the room. The examiners were instructed to change their seats from time to time during this part of the examination so that each examiner would have ample opportunity to observe each candidate, face to face. The candidates discussed various topics assigned to them and the examiners merely listened to the discussion and observed the various candidates as they talked. The examiners were instructed not to interrupt the candidates during this phase of the test.
“After the Group test, each candidate, one at a time, after fifteen minutes lallowed for preparation, gave a talk on a subject of his own selection to the five members of the board. Only the one candidate talking was before the board at any one time during this part of the test. Following this, each candidate was personally and individually interviewed by the five members of the board. The board asked each candidate the same five questions
prepared in advance. The questions were in substance as follows:
1. ‘What qualifications do you think you possess for the position of deputy chief? Explain it.’
2. ‘If you had it to do over, would you have chosen the fire department for a career? Why, and explain.’
3. ‘Under what conditions would you order the sounding of a greater alarm?’
4. ‘In the past, the deputy chief acted more or less as a senior district chief from the central district; in revising the rules of the department would you recommend any changes? If so, what; and if not, why not?’
5. What are your ideas for improving the public relations of the fire department?’
“While the examiners were instructed to ask each candidate the same five questions, the examiners apparently were not prohibited by specific direction from the Commission from asking questions and receiving answers other than the five mentioned above, and so far as the record shows, there is no way of knowing what additional questions were asked any particular candidate if any; and if there were such, what effect the answers had on the over-all grade given the candidate.
“After the Individual Interview test, the examiners graded the candidates by filling in the rating sheet, Plaintiff’s Exhibit D. Each candidate was rated by a letter on each of eight personality factors; first on the result of the Group Oral test and second on the Individual Interview test. The rating key was as follows:
S Superior
G Good
P Passable
NA Not acceptable
Very poor
“From the ratings thus recorded each examiner then gave each candidate an over-all estimate of value signified by a letter, the key for this rating being as follows:
A — I believe this applicant is best qualified for this position.
B — I believe this applicant is qualified for this position.
C — -I believe this applicant is passable for this position.
D — I doubt that this applicant is qualified for this position.
E — I believe that this applicant is the least qualified for this position.
“Plaintiff Frazzini’s rating by the five examiners was A-A-B-C-D, and Becker’s, A-B-B-B-C.
“The Commission had prepared a table (Plaintiff’s Exhibit F) from which a mathematical equivalent for all combinations of the letter ratings could be made, and from this table Frazzini’s grade appeared to be 80 and Becker’s 83.
“The first paragraph of the general instruction sheet given to the examiners by the commission before the examination started is as follows: ‘The purpose of the group oral and individual interview of candidates for the position of Deputy Chief, Fire Department, is to evaluate, as nearly as possible, certain personality factors and traits which are believed to be essential for the type of work to be performed. All who have been admitted to the oral hiave passed a written examination on subject matter pertinent to the work. Therefore, it is necessary to rate candidates only on expression of their own ideas and their reactions to the ideas of others. In rating, the examining board should discount as much as possible the accuracy of any facts that candidates may present.’
“As stated above, the examiners rated each candidate on eight separate personality factors: These appear in separate columns on the rating sheet as follows:
1. Appearance
2. Voice and language
3. Manner and poise
4. Comprehension and presentation of ideas
5. Diplomacy
6. Adaptability
7. Maturity of judgment
8. Emotional stability
“Plaintiff’s Exhibit C explains the meaning of the eight personality factors and instructs the examiners what to consider under each factor in determining the rating to be given the various applicants. Plaintiffs Exhibits I, J, K, L, M, N, O, P, Q and R are the original rating sheets prepared and turned in by the five examiners, each examiner using two sheets. On these sheets the candidates are designated by number in the left-hand column. Plaintiff was No. 4 and Becker No. 5. There is no indication or identification on these sheets which reveals the name of the particular examiner who filled out any particular rating sheet, and the candidates themselves throughout the entire examination before the oral board were known only by number.
“After the results of the examination were announced but before Chief Becker received the appointment of deputy chief, plaintiff filed with the Commission a protest, claiming that the efficiency or merit rating given him was erroneous and inequitable and should be adjusted; and, further, that the over-all grade of !D’ given him by one of the examiners was unfair, erroneous and inequitable.
“On May 31, 1951, the Commission heard the protest, at which hearing plaintiff and his attorney were present, and later the protest was denied by the Commission and the eligible register previously announced was affirmed and declared to be the eligible register from which certification for appointment should be made. * * * Plaintiff alleges that the examination, the eligible register certified as the result of the examination, and the appointment of defendent Becker as Deputy Fire Chief are illegal and void, are contrary to the Charter of Denver and the Rules and Regulations of the Commis
sion, and that the Commission exceeded its jurisdiction and abused its discretion, because:
1. The examination consisted entirely of oral statements and no means was furnished or employed for objective grading of said oral statements.
2. The examination wias entirely subjective in nature and contained ño objective standards as to knowledge or fitness of the applicant which could be challenged or reviewed by other examiners of equal ability and experience.
3. The grading of the examination did not take into account the knowledge of the applicant upon the subject of fire fighting or the proper organization and direction of a city fire department, and was not a test or examination of knowledge at all.
“Other reasons were pleaded which were not argued at the trial. * * *
“The defenses of laches 'and estoppel will be discussed first. These are equitable doctrines and their application is controlled by equitable considerations. Each case where these defenses are invoked must be decided upon the facts and circumstances before the court in that particular case, and while general principles are applied, no hard and fast rules are controlling as is often the case where purely legal as distinguished from equitable doctrines are being tested.
“Mere delay in asserting a right is not conclusive and is not in itself sufficient to constitute a bar to the maintenance of an action. Delay in the institution of a suit which has not worked injury, prejudice or disadvantage to those claiming laches or estoppel, will not bar the action. Defendant Becker was a member of the fire department when he took the examination for deputy chief, and had been serving as such for a good many years. He did not give up- private employment or business in order to become a member of the fire department when the examination here under scrutiny was held. If his appointment as deputy chief should be declared invalid, he
will automatically again assume the rank of assistant chief of the fire department, the position he occupied before his promotion, and as assistant fire chief he will be eligible to take any future promotional examination which may be held, should he desire to do so. The only way in which he can be said to have suffered injury or prejudice, is that presumptively the salary of deputy chief is greater than that of assistant chief. What the actual difference is in dollars and cents is not in the evidence before the court. In Plaintiff’s Exhibit H, the book of rules of the Commission as of 1949, this difference is shown to be $33.06 per month.
“In reliance upon the continuation of the larger salary he perhaps had made certain financial commitments and perhaps has changed his manner of living to some extent, and if his appointment is set aside, he may or may not be required to refund the difference in the two salaries which he has received since his appointment as deputy chief.
“In my judgment, and it is so held, the defenses of laches and estoppel are not available to the defendants in this action, and the issue as to those defenses is found in favor of the plaintiff and against the defendants.
“Excerpts from applicable sections of the Charter of Denver are as follows:
“Sec. 223. ‘The Commission shall control all examinations and may * * * designate a suitable number of persons * * * to be examiners * * *.’
“Sec. 224. ‘The Commission shall have power to make and enforce rules * * *.’
“Sec. 225. ‘The rules shall provide for * * * open and competitive examinations as to fitness * * *; promotion on the basis of merit, experience and record.’
“Sec. 228. ‘All examinations shall be impartial and relate only to matters which will test the fitness of the persons examined for the • service they wish to enter * * * 3
“Sec. 233. Promotions, ‘The Commission shall provide
for promotion in the classified service on the basis of ascertained merit and seniority in service and standing upon examinations * * *. All examinations for promotion shall be competitive among such members of each department as desire to submit themselves to examination. * * * The method of examining, the rules governing the same and the method of certifying shall be the same, as near as may be, as provided for applicants for original appointment * * *.’
“Section 233, las quoted, requires examinations for promotions to be competitive, and plaintiff claims here that the examination under attack was not competitive as that term has been defined by the courts.
“In a case often cited, Fink v. Finegan, 1 N.E. 2d 462, the New York Court said: ‘A test or examination, to be competitive, must employ an objective standard or measure. Where the standard or measure is wholly subjective to the examiners it differs in effect in no respect from an uncontrolled opinion of the examiners and cannot be termed competitive. * * * An examination cannot be classed as competitive unless it conforms to measures or standards which are sufficiently objective to be capable of being challenged land reviewed, when necessary, by other examiners of equal ability and experience.’
“While the facts in the case cited are not on all fours with this case, the quoted portion of the opinion is an accepted general statement as to what is meant by la competitive examination in considering laws and rules governing civil service and persons who submit to examinations in the classified service of state or city.
“I am of the opinion that the manner in which the examination for deputy fire chief involved here was promulgated and set up by the Commission, if its instructions had been carried out by the examiners, would have met all tests prescribed by charter, rule and adjudicated cases; but I am convinced that the way in which the grades were determined by the examiners, as shown by
their rating sheets and the testimony of two of them, makes the examination vulnerable to the attack made on it by plaintiff and requires an adjudication by this court declaring it a nullity.
“In one phase of the personal interview test, it appears that some questions might have been asked some candidates which were not put to others, and there is no possible way of determining what weight might have been given to answers made to such questions by the individual examiners. The examiners were instructed to ask each candidate the same five questions and to grade them on their answers.
“The final rating sheets, plaintiff’s Exhibits I to R, upon which the examiners gave an over-all estimate of value, which estimate in turn was given a mathematical value by the Commission according to a previously prepared table, land which mathematical figure was the basis for 60% of the applicant’s total grade, were prepared and designed so that it could be readily ascertained by anyone examining the sheets exactly what grades or ratings each candidate received from each examiner on each and all of the various factors being considered.
“Some, but not all, of the defects appearing on the rating sheets which in my judgment make the examination invalid are the following:
“One examiner gave no grade or rating whatever to any of the candidates on the group test.
“One examiner gave only one of the candidates a grade on the personality trait designated on the sheet as ‘Adaptability,’ and the one who was rated on this trait was not the plaintiff.
“One examiner gave two candidates an ‘S.’ meaning superior, on all factors and on both parts of the test, that is, the Group and the Individual tests, but gave one candidate an over-all estimate of ‘A’ and the other one ‘B.’
“One examiner rated an applicant ia ‘C’ based on twelve ‘G’s’ and four ‘P’s,’ but gave another applicant a ‘B’ based on nine ‘G’s,’ six ‘P’s’ and one ‘NA.’
“Other similar examples of arbitrary grading and rating without regard to measures and standards can be found upon close scrutiny of the examiners’ rating sheets which they delivered to the Commission and upon which the mathematical final grade was computed.
“For the reasons stated it is the finding and judgment of the court that the examination given for the office of deputy chief of the fire department is a nullity, illegal and void and cannot be allowed to stand, and the eligible list and appointment which followed the examination must also fall.
“The issues joined are found in favor of the plaintiff and against the defendants, the plaintiff is entitled to the relief prayed for. A new promotional examination for the office of deputy chief of the fire department should be held without unnecessary delay.”
Subsequent to the above judgment defendant Becker obtained additional counsel, who filed on his behalf a motion for relief from the judgment under Rule 60 (b), R.C.P. Colo., upon the grounds of mistake and excusable neglect, setting forth alleged facts, all directed to the defenses of laches and change of position or estoppel of said Becker as lalleged in the answer to the complaint. In the facts set forth in the affidavit it was pointed out that defendant Becker’s salary advance was about $50.00 per month instead of $33.06, as found by the trial court; that his retirement pension of one-half pay, to which he was eligible, would be correspondingly increased; that if a new examination were given he would lose 5% seniority benefit over the plaintiff and that six other men would be eligible for veterans’ preference, and seven additional men would have the maximum rating for seniority.
With the court’s permission, Frazzini filed a counter affidavit alleging that upon a new examination Becker’s position might be improved due to the seniority ratings of the persons eligible to take the examination.
Upon hearing, the court denied the petition of Becker for relief from the judgment, in effect re-affirming its
judgment upon the issue of laches and estoppel, and we are not inclined, upon this review, to disturb this ruling and judgment.
Fundamentally this is not a contest between Friazzini and Becker who placed second and first respectively in the examination, but the real question is whether a legal examination was held and all the candidates who participated therein were afforded equal opportunity to compete in such examination.
The matter now left for determination is whether such examination as was held is so subjective in character as to be in effect a non-competitive examination and illegal; and if it is determined that the character of the examination was competitive and legal, then did the examiners follow their instructions and actually conduct a legal examination?
An analysis of the decision in
Fink v. Finegan,
270 N.Y. 356, 1 N.E. (2d) 462, cited in the trial court’s findings and judgment, discloses that there was no finding and announced notice by the commission that executive ability and force were necessary qualities for the position of police surgeon and medical officer in the fire department, or that these qualities would be tested or that they could not be tested objectively; that the examiners injected this phase of the examination and rejected the candidates thereon.
Following the Fink v. Finegan decision, supra, in
Sloat v. Board of Examiners of Board of Education of the City
of
New York,
274 N.Y. 367, 9 N.E. (2d) 12, 112 A.L.R. 660, the court upheld the examination. The petitioner, a school teacher who passed a written, practical and physical examination, failed in an oral “teaching and interview” test because of voice difficulties, the court held, as appears at page 373: “The mandate of the Constitution for the ascertainment of merit and fitness, so far as practicable, by competitive examination, may not be transformed into an interdict against the examinations which are best adapted for the demonstration
of fitness. It would be impossible to formulate a standard by which such qualities may be defined or measured with entire objectivity. The law does not require the impossible or forbid the reasonable. The record discloses that here the examiners have based determination upon their estimates of qualities which, it is reasonably clear, affect the merit and fitness of a teacher and that this estimate is derived from tests calculated reasonably to show those qualities.”
Courts in subsequent New York cases, in attempting to rationalize
Fink v. Finegan, supra,
and
Sloat v. Board of Examiners, supra,
announce the rule that personality tests should employ objective standards insofar as practicable.
Bridgman v. Kern,
257 N.Y. App. Div. 420, 13 N.Y.S. 2d 249;
Bridgman v. Kern,
282 N.Y. 375, 26 N.E. (2d) 99;
Farrell v. Kern,
17 N.Y.S. 2d 934;
Andresen v. Rice,
277 N.Y. 271, 14 N.E. 2d 65;
Cowen v. Reavy,
283 N.Y. 232, 28 N.E. (2d) 390;
Barnett v. Fields,
92 N.Y.S. 2d 117;
Weinberg v. Fields,
114 N.Y.S. 2d 238;
Quinn v. Streeter,
22 N.Y.S. 2d 546.
As the testing of executive ability in certain types of civil service positions became more important in the minds of various commissions a series of California decisions were rendered culminating in the California rule laid down in
Almassy v. Los Angeles County Civil Service Commission,
34 Cal. 2d, 387, 210 P. (2d) 503. The facts in this case, the pattern of the examination, the computation of the grades, and the attacks thereon are similar and nearly parallel to the instant case.
The Los Angeles charter provided for competitive examinations and that records thereof be kept. In upholding the legality of the examination the court said, page 510, 210 P. (2d): “What, then, is meant by ‘competitive examination’? In a competitive examination, the candidates match their qualifications each against the others, and the final determination is made by rating and comparison. It is open to all who are eligible. * * *
“To be sure, there must be measurable standards es
lablished for determining the general proficiency of the candidates for the particular position, but the fact that such standards were subjective rather than objective does not prevent the classification of the ‘oral interview’ as a ‘competitive’ examination. The various aspects of the personality factors, as previously determined by the commission to be ‘necessary and important’ to the position to be filled, were itemized for consideration by the examiner as the measuring rod and guidepost in his evaluation process as to the personal fitness of the candidate, so that it cannot be said that there was no effective common criterion of base topics underlying the comparative ratings.” Citing
King v. Emmons,
128 Ohio St. 216, 190 N.E. 468.
The Almassy case, supra, was approved in
Stoor v. City of Seattle,
44 Wash. (2d) 405, and
Curtis v. City of Seattle,
by the Washington Supreme Court, 267 P. (2d) 902.
In
Civil Service Board of City of Phoenix v. Warren,
74 Ariz. 88, 244 P. (2d) 1157, the court in commenting upon the fact that the city ordinance required a record of questions and answers in a civil service examination held that the reasoning in the opinions in
Sloat v. Board of Examiners, supra,
and
Almassy v. Los Angeles County Civil Service Commission, supra,
was not persuasive, in face of the mandate of the ordinance and invalidated the examination. The ordinance to which we herein refer is absent in the instant case.
In the present case we hold that the examination proposed by the commission was in compliance with the charter of the City and County of Denver; the determination that personality factors were to be a part of the examination was for the commission.
Getty v. Witter,
107 Colo. 302, 111 P.(2d), 636;
Hewitt v. State Civil Service Commission,
114 Colo. 561, 167 P.(2d), 961;
Sloat v. Board of Examiners, sdpra,
and
Almassy v. Los Angeles County Civil Service Commission, supra;
that the proposed examination was competitive in that the candi
dates were to participate against each other equally before the examiners in answering questions of like character and nature, and that they had equal opportunity in the oral discussion to compete each against the other under like and similar conditions; that the method of grading set up in the examination by the commission was sufficiently objective and definite to be capable of being challenged and reviewed by examiners of equal ability and experience, and that there was sufficient record available for a review by the courts.
Having determined that the
proposed
examination as set up by the commission was legal, it then becomes a question of fact whether the examiners in conducting the examination followed the instructions of the commission. The record amply supports the finding of the trial court that the examiners failed in this respect; towit: Although instructed to do so, one examiner did not grade or rate any of the candidates on the group test.
One examiner graded only one of the candidates on the personality trait designated on the examination sheet as “Adaptability” and the one who was rated on this trait was not the plaintiff nor the defendant Becker.
One examiner rated an applicant a “C” based on twelve “G’s” and four “P’s,” but gave another applicant a “B” based on nine “G’s,” six “P’s” and one “NA” (not acceptable), manifestly an inequitable over-all estimate.
Other similar examples of arbitrary grading and rating without regard to measures and standards can be found upon close scrutiny of the examiners’ rating sheets which were delivered to the commission.
Therefore we hold the examination was not legally conducted, resulting in depriving those participating therein of a truly competitive result.
At this point we call attention to the fact that by the very nature of an examination of this character, the likelihood of abuse, favoritism and human error are inherent, and the greatest care and caution should be exercised by the commission in preparing and conducting
such examination; moreover, if personality factors are to be considered in appraising the qualifications of applicants, the announcement of the examination should specifically so state.
Consequently we approve the finding of the trial court holding the examination invalid. The judgment is therefore affirmed.