Duffly, J.
Since 2005, Jill Kavaleski has sought employment as a police officer with the Boston police department (department). The department has, on three occasions, extended conditional offers of appointment to Kavaleski, each of which was [681]*681contingent upon her successful completion of a psychological screening process. On each occasion, department psychiatrists found Kavaleski psychologically unqualified for the job, and the department “bypassed” her for appointment as a police officer. See G. L. c. 31, § 27. This case arises from the third such bypass, which Kavaleski appealed to the Civil Service Commission (commission). See G. L. c. 31, § 2 (b). After an evidentiary hearing, the commission concluded that the department had failed to meet its burden of establishing a reasonable justification for bypassing Kavaleski, and ordered that her name be restored to the department’s list of individuals certified for appointment. The department filed an appeal in the Superior Court, see G. L. c. 31, § 44, arguing that, in reaching its decision, the commission had erroneously relied on expert testimony from an unrelated proceeding. A Superior Court judge ruled that the commission had erred and vacated the commission’s order. We granted Kavaleski’s application for direct appellate review.
We agree that the commission erred in the manner in which it considered expert testimony from another proceeding. Because the commission’s decision was supported by substantial evidence independent of this extraneous evidence, however, we conclude that the error did not prejudice the department. Accordingly, we reverse the Superior Court judge’s order.
Background.1 Kavaleski is a lifelong resident of Boston. She has received two graduate degrees from a local university, and has for many years been employed by the city of Boston’s veterans’ services department.2 She has never been diagnosed as having, and has never received treatment for, any psychiatric or psychological disorder or condition.
In 2002, Kavaleski applied to be a police officer in New York. She passed that State’s civil service examination, a background investigation, and psychological screening, and was offered a position with the New York City police department. She declined that offer. In 2005, she applied for a position as a police officer with the department.
[682]*682As is required of all applicants, Kavaleski completed the department’s lengthy application packet, which requires extensive disclosures about many aspects of an applicant’s life. She also submitted the required letters of reference, cooperated with a background investigation, and took a civil service examination administered by the Commonwealth’s human resources division (HRD). Kavaleski passed the examination, and according to the commission, her references were “of the highest order.” The commission summarized Kavaleski’s references as describing “a dedicated and passionate person committed to public or community service, who exercises responsibility, good judgment and common sense in the completion of her tasks.”
In 2006, Kavaleski twice received a conditional offer of appointment from the department, but on each occasion was “bypassed”3 after being deemed psychologically unqualified by department psychiatrists. In early 2007, the department extended a third conditional offer of appointment to Kavaleski.4 The sole condition of the third offer was, again, that Kavaleski successfully complete the department’s psychological screening process.5
The department’s psychological screening process has three [683]*683“phases.” In “Phase I,” candidates for employment must take two standardized tests: the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) and the Personality Assessment Inventory (PAI).6 In “Phase II,” candidates meet with one of the department’s psychiatrists for a thirty-minute clinical interview (first-level interview). Before the interview, the psychiatrist reviews the candidate’s MMPI-2 and PAI test results as well as material from the department’s background investigation, the candidate’s medical history, and information provided by the candidate in a biographical questionnaire. During the interview, the psychiatrist conducts a “mental status examination,” explores any areas of concern raised by the testing and biographical data, and evaluates possible areas of “psychological vulnerability as it pertains to the essential functions of the police officer position. ” If this process raises no “suitability issues,” the psychiatrist will report to the department in writing that the candidate is psychologically suitable for appointment as a police officer.
If the psychiatrist identifies areas requiring further inquiry, he or she will prepare a written report outlining the specific concerns and refer the candidate to “Phase IH” of the screening process, a “second opinion psychiatric interview” (second-level interview). A different psychiatrist conducts the second-level interview. The second psychiatrist reviews the report from the first-level interview, as well as the test results and background material reviewed initially by the first psychiatrist. The second psychiatrist then conducts an “in-depth clinical interview” and makes a final written recommendation to the department regarding any “psychological/behavioral issues that would interfere with the applicant’s performance of the essential job functions” of being a police officer.
The entire screening process operates in accordance with [684]*684rules promulgated by HRD. Those rules define the medical standards that a municipal police officer in the Commonwealth must meet, and sort disqualifying medical or psychiatric7 conditions into two categories. A “Category A” condition is one “that would preclude an individual from performing the essential job functions of a municipal police officer or present a significant risk to the safety and health of that individual or others.”8 A “Category B” condition is one that, “based on its severity or degree, may or may not preclude an individual from performing the essential job functions of a municipal police officer, or present a significant risk to the safety and health of that individual or others.”9
As noted, at the time of her appeal, Kavaleski had undergone the department’s psychological screening process three times, and had completed the MMPI-2 and PAI during each round of screening. Both tests were scored automatically, using Kavales-ki’s responses to create a computer-generated report of her psychological “profile.” According to the MMPI-2 reports from Kavaleski’s first two rounds of testing, she produced “invalid” profiles because her responses were “too defensive to permit an adequate assessment of her psychological adjustment.” In both rounds of screening, the first and second psychiatrists who evaluated Kavaleski after reviewing these “invalid” profiles reported that Kavaleski was defensive, guarded, or “interpersonally stiff.”10 The psychiatrists also took note of Kavaleski’s ap[685]*685pearance, describing her as thin, with hair that was “messy” or “unkempt.”
In the third round of testing, Kavaleski’s responses produced a “valid” MMPI-2 profile. The computer-generated profile indicated that Kavaleski had “[ijndorsed” certain test questions, known as “critical items,” in the areas of acute anxiety, somatic symptoms, anxiety and tension, and deviant beliefs.11 A similar computer-generated report based on Kavaleski’s responses to the PAI noted that Kavaleski presented a “[l]ow risk” in the “[pjsychological rating risk factor” category, and that she had indorsed critical items relating to drug problems, anxiety, persecution, and aggressive attitude.12
Dr. Marcia Scott, who had interviewed Kavaleski in a previous round of screening, conducted Kavaleski’s first-level interview on March 20, 2007. Scott reported that Kavaleski was “less guarded” than she had been in previous interviews, and was “able to respond appropriately to relevant personal questions.” Scott also made various observations about Kavaleski’s weight and appearance, noting Kavaleski’s “almost cache[c]tic body”13 and “messy” hair. Scott concluded her report by stating that Kavaleski “is a steady controlled person but has very limited self-awareness, little understanding of her motivations or emotional limitations and inflexible approaches to both internal and external stresses.” Scott stated that Kavaleski’s “capacity to evaluate situations and make effective judgments” would [686]*686impair her ability to work as a police officer, and referred Kavaleski for a second-level interview with Dr. Julia M. Reade.14
Reade, who conducted each of Kavaleski’s three second-level interviews, met with Kavaleski approximately three months later, on June 30, 2007. She described Kavaleski as “thin, but not unhealthy looking,” and again noted that “her hair was messy.” Reade stated that she had reviewed materials from Kavaleski’s two previous rounds of psychological screening, and included in her report the critical items that Kavaleski had indorsed during the latest round of MMPI-2 and PAI testing. Reade described Kavaleski’s demeanor during the interview as “impassive” and concluded her report as follows:
“In summary, despite her continued effort to be more open and flexible, Ms. Kavaleski continues to present as a psychologically inflexible, interpersonally stiff woman whose extreme defensiveness limits her capacity to reflect on her own decision-making, responses, actions or impact on others. Her concrete cognitive style is equally limiting and is likely related to what appears to be a charactero-logic rigidity. These limitations would interfere with Ms. Kavaleski’s ability to manage the duties of a Boston [p]olice officer.”
Based on Reade’s report, the department notified HRD that it intended to bypass Kavaleski because she had failed to meet the psychological criteria for employment as a police officer. HRD accepted the department’s stated reasons, and on August 31, 2007, Kavaleski appealed to the commission pursuant to G. L. c. 31, § 2(b).
A hearing was held before the commission on April 3, 2008, at which Kavaleski represented herself. The commission accepted numerous exhibits in evidence and heard testimony from Reade and Kavaleski. By a divided vote, the commission ruled that the department had not met its burden of establishing a reasonable justification for bypassing Kavaleski. The commission noted that a disqualifying psychiatric condition has not “been found to exist in [Kavaleski], nor has the [department] [687]*687specifically asserted any such condition.” The commission ordered the department to place Kavaleski’s name “at the top of the eligibility list for original appointment to the position of [pjolice [o]fficer ... so that she shall receive at least one opportunity for consideration from the next certification for appointment as a [department] police officer.” The commission also ordered that, should the department choose to require Kavaleski to submit to further psychological screening, it must use psychiatrists other than those who had previously been involved in screening or evaluating her.
In reaching its decision, the commission quoted several written findings of fact that it had made in deciding the case of Roberts vs. Boston Police Dep’t, Civil Serv. Comm’n, No. G1-06-321 (Sept. 25, 2008) (Roberts).15 Like the present case, Roberts involved a psychological bypass by the department based on the candidate’s interviews with Scott and Reade. The candidate in that case had called psychologists Dr. Mark S. Schaeffer and Dr. James C. Beck16 to testify as expert witnesses, and the Roberts commission quoted extensively from their testimony in its written decision.17
Schaeffer testified in Roberts that interpreting MMPI-2 and PAI results “fall[s] within the professional discipline of psychology, as opposed to medicine and psychiatry.” In discussing the psychiatrists’ evaluations of Kavaleski in the present case, the commission quoted Schaeffer’s testimony from Roberts as well as the Roberts commission’s finding in that case that “all the expert witnesses who testified in [Roberts] agree that a qualified psychologist is the recommended professional with the necessary expertise to which a psychiatrist generally defers when it comes to the subject of psychological testing” (emphasis in original). The commission then noted that none of the [688]*688psychiatrists involved in interviewing Kavaleski had consulted a specially-trained psychologist to interpret her test results.18 Based in part on these findings, the commission determined that Reade’s conclusions about Kavaleski’s psychological fitness for employment as a police officer were not credible.
The department filed an appeal in the Superior Court, pursuant to G. L. c. 31, § 44, arguing that the commission had improperly relied on Roberts. Concluding that the commission had erroneously relied on testimony introduced in the Roberts case in reaching its decision in the present case, a Superior Court judge allowed the department’s motion for judgment on the pleadings and vacated the commission’s decision.
Standard of review. When a candidate for appointment appeals from a bypass, the commission’s role is not to determine whether that candidate should have been bypassed. Rather, the commission determines, “on the basis of the evidence before it, whether the appointing authority [has] sustained its burden of proving, by a preponderance of the evidence, that there was reasonable justification” for the decision to bypass the candidate. Brackett v. Civil Serv. Comm’n, 447 Mass. 233, 241 (2006), citing G. L. c. 31, § 2 (b). “Reasonable justification in this context means ‘done upon adequate reasons sufficiently supported by credible evidence, when weighed by an unprejudiced mind, guided by common sense and by correct rules of law.’ ” Brackett v. Civil Serv. Comm’n, supra, quoting Selectmen of Wakefield v. Judge of First Dist. Court of E. Middlesex, 262 Mass. 477, 482 (1928). In determining whether the department has shown a reasonable justification for a bypass, the commission’s primary concern is to ensure that the department’s action comports with “[b]asic merit principles,” as defined in G. L. c. 31, § 1.19 See Massachusetts Ass’n of Minority Law Enforcement Officers v. Abban, 434 Mass. 256, 259 (2001). The com[689]*689mission “finds the facts afresh” in conducting this inquiry and is not limited to the evidence that was before the department. Beverly v. Civil Serv. Comm’n, 78 Mass. App. Ct. 182, 187 (2010).
Pursuant to G. L. c. 31, § 44, we review the commission’s decision to determine whether it was in conformity with the standards set forth in G. L. c. 30A, § 14 (7). See Massachusetts Ass’n of Minority Law Enforcement Officers v. Abban, supra at 263. We may set aside or modify the commission’s decision if we conclude that “the substantial rights of any party may have been prejudiced” by a decision that is based on an error of law, unsupported by substantial evidence, or otherwise not in accordance with the law. G. L. c. 30A, § 14 (7). Because it is the department that appealed from the commission’s decision, the department bears the burden of establishing that the decision is invalid. Brackett v. Civil Service Comm’n, supra at 242. That is a “heavy burden,” Massachusetts Ass’n of Minority Law Enforcement Officers v. Abban, supra at 263-264, since we give “due weight to the experience, technical competence, and specialized knowledge” of the commission in deciding these matters. G. L. c. 30A, § 14 (7). “This standard of review is highly deferential to the agency on questions of fact and reasonable inferences drawn therefrom.” Brackett v. Civil Service Comm’n, supra, quoting Flint v. Commissioner of Pub. Welfare, 412 Mass. 416, 420 (1992).
Discussion. The department advances two related arguments to support its claim that the commission’s decision should be reversed. As stated, it contends that the commission erred as a matter of law in relying on testimony from Roberts to attack Reade’s credibility. As a corollary, the department maintains [690]*690that there was no properly admitted expert evidence to contradict Reade’s testimony and that, therefore, the commission erred in substituting its own assessment of Kavaleski’s psychological fitness for employment as a police officer. We agree that the commission should not have considered the expert testimony that was introduced in Roberts, but not for the reasons advanced by the department. In addition, we conclude that the commission was entitled to discredit Reade’s testimony without hearing testimony from an opposing expert.
General Laws c. 30A, which governs proceedings before the commission, sets forth the extent to which an agency may rely on, and take notice of, materials other than those supplied by the parties. General Laws c. 30A, § 11 (4), provides, in relevant part:
“All evidence, including any records, investigation reports, and documents in the possession of the agency of which it desires to avail itself as evidence in making a decision, shall be offered and made a part of the record in the proceeding, and no other factual information or evidence shall be considered . . . .”
A related provision, G. L. c. 30A, § 11 (5), authorizes agencies to “take notice of any fact which may be judicially noticed by the courts,” as well as any “general, technical or scientific facts within their specialized knowledge.” However, “[pjarties shall be notified of the material so noticed, and they shall be afforded an opportunity to contest the facts so noticed.” Id. See Assessors of Boston v. Ogden Suffolk Downs, Inc., 398 Mass. 604, 605-606 (1986).
The critical component of these statutory provisions is that parties be afforded notice of and an opportunity to respond to the evidence on which an agency relies in rendering a decision. See, e.g., Doe, Sex Offender Registry Bd. No. 89230 v. Sex Offender Registry Bd., 452 Mass. 764, 782 (2008) (agency erred in relying on psychiatric manual where petitioner not notified or afforded opportunity to refute that evidence); New York Cent. R.R. v. Department of Pub. Works, 354 Mass. 332, 336 (1968) (facts not properly before department where petitioner did not have opportunity to contest them). Thus, our concern with the commission’s decision is not that the commission considered [691]*691testimony from a different commission proceeding, which it permissibly may do. See Doherty v. Retirement Bd. of Medford, 425 Mass. 130, 140 (1997) (upholding State agency’s reliance on transcripts from Federal criminal proceedings where transcripts bore “reasonable indicia of reliability”).20 Contrast Assessors of Boston v. Ogden Suffolk Downs, Inc., supra at 606 (agency could not permissibly rely on determination of property values made in prior proceeding involving same party where prior decision was not supported by contemporaneous findings). Rather, the commission erred in failing to alert the department that it would be looking to Schaeffer’s and Beck’s testimony in Roberts, and considering it as evidence in the present case, thus depriving the department (and Kavaleski) of an opportunity to contest and respond to that evidence. Contrast Doherty v. Retirement Bd. of Medford, supra (agency introduced portions of transcript of prior criminal trial during hearing; defense counsel permitted to respond and to introduce other portions of transcript to question witness’s credibility).
Although we conclude that the commission erred by considering testimony from Roberts without notice to the parties and an opportunity to respond, that does not end our inquiry. Pursuant to G. L. c. 30A, § 14 (7), we also determine whether, as a result of that error, “the substantial rights of any party may have been prejudiced.”
We are satisfied that the department was not prejudiced by the commission’s reliance on expert testimony from Roberts, because the commission’s decision did not depend on that testimony. Although, as the Superior Court judge noted, the extraneous evidence “factored into” the commission’s decision to discredit Reade’s testimony regarding Kavaleski, the commission did not decide Kavaleski’s appeal on that basis alone, and there was other substantial and reliable evidence in the record, independent of the testimony from Roberts, to support the commission’s decision.21 See Assessors of Boston v. Ogden Suffolk Downs, Inc., supra at 607-608 (even if expert witness’s [692]*692testimony not credited, other “reliable evidence” was sufficient to uphold agency’s decision). The commission concluded that Reade’s opinion was the result of arbitrary predispositions against Kavaleski, and that Reade (and, thus, the department) had based her determination on “unsubstantiated and subjective” criteria that lacked adequate factual support. This conclusion is supported by substantial evidence in the record. See G. L. c. 30A, § 14 (7). See also Embers of Salisbury, Inc. v. Alcoholic Beverages Control Comm’n, 401 Mass. 526, 528 (1988) (agency’s decision must be supported by substantial evidence).
“ ‘Substantial evidence’ means such evidence as a reasonable mind might accept as adequate to support a conclusion.” G. L. c. 30A, § 1 (6). “The standard is more stringent than abuse of discretion, and less than preponderance of the evidence; ‘an agency’s conclusion will fail judicial scrutiny if “the evidence points to no felt or appreciable probability of the conclusion or points to an overwhelming probability of the contrary.” ’ ” Duggan v. Board of Registration in Nursing, 456 Mass. 666, 674 (2010), quoting Cobble v. Commissioner of the Dep’t of Social Servs., 430 Mass. 385, 390-391 (1999).
The commission concluded that the department had bypassed Kavaleski based on mere conjecture, and that the evidence did not show that a disqualifying psychiatric condition “has been found to exist in [Kavaleski].” The commission emphasized the “indefiniteness and inconclusiveness” of the language employed by all the psychiatrists involved in Kavaleski’s screenings. It observed that, in each of her interviews with Kavaleski, Reade had relied heavily on the earlier assessments of the first-level psychiatrists, repeating the “descriptive phrases” and “alleged observations or determinations” that had been made in prior reports. These earlier reports did not affirmatively state that Kavaleski suffered from any psychiatric disorder or condition but, rather, offered vague assessments that Kavaleski’s profile [693]*693was “not inconsistent” with the presence of certain traits, that an eating disorder “may be present,” and that it was “not possible to rule out” other emotional problems. The commission found that Reade’s “testimonial tenor, tone and content indicated that she entirely adopted and affirmed” these earlier assessments, “attribut[ing] great weight and reliability to them,” and that, by doing so, Reade had evidenced a predisposition to finding Kavaleski psychologically unsuitable. These findings and conclusions were based on substantial evidence other than that derived from Roberts.
Further, the commission expressed concern regarding Reade’s repeated references to Kavaleski’s physical appearance and “messy hair,” as well as Scott’s use of “extreme language” in referring to Kavaleski’s appearance as cachectic. See note 13, supra. The commission found, based on Kavaleski’s appearance at the hearing, that those descriptions were “clear misrepresentation[s]” and an “indication of some bias or some other improper consideration” by the department.22 Observing that neither Reade nor the department asserted that Kavaleski would be unable to perform the essential functions of the job of a police officer, the commission determined that no disqualifying condition “has been found to exist in [Kavaleski], nor has the [department] specifically asserted” that one exists. It concluded:
“Instead of determining the existence of a specific disqualifying condition, according to specified standards and proscribed process, Dr. Reade looks for her own subjectively determined qualifying traits. Dr. Reade’s screening process is arbitrary and capricious, in contradiction of the basic merit principles of [G. L. c. 31]. The accuracy and reliability of the psychological screening process, as applied to [Kavaleski], is incapable of substantiation.
“After considering all the credible and reliable evidence [694]*694in the record, I conclude that the Boston [p]olice [department did not have sound and sufficient reasons for bypassing [Kavaleski] for selection as a police officer in the [c]ity of Boston.”
The department argues that, in making these findings, the commission impermissibly “substituted its own lay person opinion of Kavaleski” for that of Reade and the other experts who had examined Kavaleski. This argument misstates the commission’s role as fact finder. “The commission, and not the court, is the sole judge of the credibility and weight of the evidence before it.” School Comm. of Brockton v. Massachusetts Comm’n Against Discrimination, 423 Mass. 7, 15 (1996). See Boston Police Superior Officers Fed’n v. Civil Serv. Comm’n, 35 Mass. App. Ct. 688, 695 (1993).
The commission was entitled to discredit Reade’s assessment of Kavaleski even though Kavaleski offered no expert testimony of her own. See, e.g., Daniels v. Board of Registration in Med., 418 Mass. 380, 392 (1994), quoting Commonwealth v. DeMi-nico, 408 Mass. 230, 235 (1990) (“[t]he law should not, and does not, give the opinions of experts on either side of . . . [a]n issue the benefit of conclusiveness, even if there are no contrary opinions introduced at the trial”). See also Boston Gas Co. v. Assessors of Boston, 334 Mass. 549, 579 (1956) (“That a person qualifies as an expert does not endow his testimony with magic qualities”). The commission properly explained on the record its reasons for rejecting portions of Reade’s testimony. Contrast Robinson v. Contributory Retirement Appeal Bd., 20 Mass. App. Ct. 634, 639-640 (1985), citing New Boston Garden Corp. v. Assessors of Boston, 383 Mass. 456, 470 (1981) (where there is uncontradicted expert testimony on subject beyond common knowledge and experience of fact finder, agency may not reject that testimony without providing basis for rejection in record).
Similarly, the commission was entitled to reject the department’s assertion that Reade’s evaluation was sufficient to disqualify Kavaleski. The commission appropriately recognized that Reade’s function in the psychological screening process was narrowly circumscribed. Her sole task was to determine [695]*695whether Kavaleski had a psychiatric condition that would prevent her from performing, even with reasonable accommodation, the essential functions of the job.23 See G. L. c. 151B, § 4 (16). The record supports the commission’s conclusions that Reade’s opinions were “substantially subjective determinations” that were “insufficiently factually supported,” and that Reade did not provide a single “convincing situational example” to support her conclusion that Kavaleski’s “defensiveness” and “char-acterologic rigidity” would interfere with police work in an “objective real-world context.”24
Because the commission’s conclusions were independent of its improper reliance on Roberts and are supported by substantial evidence in the record, the department has failed to meet its burden of establishing that the commission’s decision was invalid pursuant to G. L. c. 30A, § 14 (7).
Conclusion. The order allowing the department’s motion for judgment on the pleadings is reversed, and the judgment is set aside. The case is remanded to the Superior Court for entry of an order affirming the commission’s decision.
So ordered.