MICHEL, Circuit Judge.
Henry L. Chisolm, Jr. (“Chisolm”) appeals the decision of the United States Court of Federal Claims granting the government’s Motion for Judgment on the Administrative Record. Chisolm, v. United States, 49 Fed.Cl. 614 (2001). Because we hold that under all the circumstances of this unusual case the Air Force Board for Correction of Military Records (“Corrections Board”) acted in an arbitrary and capricious manner in disregarding con[396] trary evidence without analysis and accepting as dispositive the evaluation board president’s opinion that Chisolm’s missing Officer Performance Report (“OPR”) would not have made a difference in the Management Level Evaluation Board’s (“MLEB”) final recommendation to the selection board, the trial court’s decision is vacated and remanded.
I
Chisolm was a Judge Advocate General (“JAG”) officer in the Air Force, serving as the Deputy Staff Judge Advocate (“SJA”) of the U.S. Air Force Academy, and was eligible for promotion to lieutenant colonel in 1993. Following Air Force procedures, Chisolm’s superiors prepared a Promotion Recommendation Form (“PRF”) for him prior to the promotion selection process. A PRF is a document designed to assist the central selection board in determining which candidates should be promoted. It is largely narrative in nature, but also contains a promotion recommendation — either “Definitely Promote” (“DP”), “Promote” (“P”), and “Do Not Promote.” Chisolm’s PRF was drafted by his first-level supervisor, Colonel DeAustin (“DeAustin”), and reviewed, approved and signed by Chisolm’s senior rater, Colonel Dorger (“Dorger”).
Each senior rater is allotted a fixed number of DP recommendations. When such a senior rater does not have any DP recommendations to award, his officers compete with others in a pool before an MLEB, which ultimately determines the officer’s promotion recommendation. A central selection board then makes the decision. Normally, all DP candidates are promoted and a minority of P candidates are as well. Because Dorger did not have the quota to award Chisolm a “Definitely Promote” recommendation, Chisolm’s record was forwarded to a Non-Line Evaluation Board (“NLEB”), a management-level evaluation board for non-line officers such as JAG officers.1 By coincidence, DeAustin was a member of this evaluation board. He noticed that Chisolm’s most recent OPR,2 which he himself had prepared, was not in the record before the Board. The president of the NLEB, Brigadier General Gideon, tried unsuccessfully to locate the missing report. And, ultimately the NLEB evaluated Chisolm’s record and made a recommendation to the selection board without the OPR. Thus Chisolm was compared to other candidates when his record was incomplete but theirs was complete. Chisolm received a “Promote” recommendation from the NLEB and was non-selected by the central selection board.
Chisolm appealed to the Corrections Board claiming that his 1993 promotion non-selection was invalid because the NLEB had relied on an incomplete record.3 The Air Force agreed that Chisolm’s record before the NLEB was incomplete and the Corrections Board forwarded his complete record (now containing the missing OPR) to the evaluation board president, Brigadier General Gideon (“Gideon”), for a second review, per Air Force Regulation 31-11. After reviewing Chisolm’s complete record, Gideon concluded that the five-[397] member NLEB still would have given him only a “Promote” recommendation even if it had considered the missing OPR. In a letter written almost a year after the initial review, Gideon identified three reasons why he felt that the missing OPR would not have changed the NLEB’s recommendation:
a. [Chisolm] was a Deputy SJA (Staff Judge Advocate) since June 1988 — a long time without advancement,
b. After rave reviews on his first OPR from McGuire AFB, the next two seemed rather ordinary — DP records typically do not show this regression,
c. His PRF, written by the Additional Rater on the missing OPR who knew of his most recent performance, was lackluster compared to those accompanying most DP recommendations.
Based upon Gideon’s review, the Corrections Board concluded that Chisolm had presented insufficient relevant evidence to satisfy his burden of demonstrating probable material “error or injustice,” 10 U.S.C. § 1552(a)(1) (1994) (“The Secretary ... may correct any military record ... when the Secretary considers it necessary to correct an error or injustice.”) and" therefore denied his request that his promotion recommendation be changed to “Definitely Promote.” Chisolm requested reconsideration three times based on additional evidence he submitted but with the same result: the “Promote” recommendation was upheld.
Following the Corrections Board’s final denial of his application for correction, Chisolm filed suit for military back pay in the Court of Federal Claims. The court granted the government’s Motion for Judgment on the Administrative Record, holding that in referring the matter to Gideon the Corrections Board merely acted pursuant to Air Force regulation and, therefore, did not act in an arbitrary and capricious manner in adopting Gideon’s opinion that the missing OPR would not have made a difference in the NLEB’s final recommendation to the selection board. See Chisolm, 49 Fed.Cl. at 619. Chisolm timely appealed the trial court’s decision.4 We have appellate jurisdiction under 28 U.S.C. § 1295(a)(3).
II
A.
We review legal determinations of the Court of Federal Claims such as a judgment on the administrative record de novo. Heisig v. United States, 719 F.2d 1153, 1158 (Fed.Cir.1983). This means we apply the same standard of review the trial court did. Accordingly, we will not disturb the Corrections Board’s decision unless it was arbitrary, capricious, contrary to law, or unsupported by substantial evidence. See Wronke v. Marsh, 787 F.2d 1569, 1576 (Fed.Cir.1986) (stating that the appellant was bound by the Army Board for Correction of Military Records’ determination that he was unsuitable as an officer unless he established that the determination was arbitrary, capricious, contrary to law, or unsupported by substantial evidence). “The arbitrary and capricious standard of review is difficult for an appellant to satisfy .... ” Lampe v. Sec’y of Health and Human Servs., 219 F.3d 1357, 1360 (Fed.Cir.2000). However, given the particular circumstances of this case, we hold that this standard was met by the evidence Chisolm submitted to the Corrections [398] Board and then the trial court because neither made any findings as to why the later-submitted evidence did not overcome the evidence on which the Corrections Board relied.
We note at the outset that “[b]e-cause ‘decisions as to the composition, training, equipping and control of a military force are essentially professional military judgments,’ the substance of such decisions ... is frequently beyond the institutional competence of courts to review.” Lindsay v. United States, 295 F.3d 1252, 1257-58 (Fed.Cir.2002) (quoting Gilligan v. Morgan, 413 U.S. 1, 10, 93 S.Ct. 2440, 37 L.Ed.2d 407 (1973)). And we have held that “such unreviewable determinations include a conclusion that removal of a defective [Officer Evaluation Report] would not have made any difference to an officer’s prospects for ... promotion.” Id. (citing Fluellen v. United States, 225 F.3d 1298, 1304 (Fed.Cir.2000)). We have also explained, however, that “[a] claim of procedural violations may present a justiciable controversy ...” when it involves statues or regulations. Id.
This dispute stems from the NLEB’s consideration of Chisolm’s Record of Performance (“ROP”) despite the absence of the most recent OPR in assessing his promotion potential compared to others in the same pool with complete records. In order for a promotion recommendation to be changed, AFR 31-11 requires the president to certify that “compared to other records reviewed during the evaluation board, [the reviewee’s] record would have been competitive for the revised PRF assessment if the circumstance which caused the original PRF assessment had not existed.” AFR 31-ll(x)(1)(b). Thus, absent such certification, Chisolm’s promotion recommendation could not be changed. Gideon did not make this certification.5 The Corrections Board’s responsibility then was to review his actions, i.e., to determine whether Gideon correctly concluded that when compared to other records the NLEB reviewed, Chisolm’s complete record would not have been competitive for a DP recommendation. We cannot review the NLEB’s substantive decision; however, we can and do review the process the Corrections Board followed when, per regulation, it determined whether Gideon’s report was correct that the missing OPR would not have made a difference in determining Chisolm’s promotion recommendation. See Lindsay, 295 F.3d at 1257-58.
The purpose of the Corrections Board is to provide means to “correct[] military records to remedy error or injustice.” 32 C.F.R. § 865.0 (2001); see 10 U.S.C. § 1552(a)(1) (1994). In rendering a decision, the Corrections Board must determine “[w]hether the applicant has demonstrated the existence of a material error or injustice that can be remedied effectively through correction of the applicant’s military record.” 32 C.F.R. § 865.4(l)(4). Implicit in such a determination is a meaningful consideration of the evidence presented by the applicant. That is, simply stating that the evidence presented was “considered” is not sufficient when that evidence is in direct conflict with the evidence on which the Corrections Board relied for its ultimate decision. Having reviewed the procedure the Corrections Board followed in analyzing the conflicting [399] evidence before it, we hold that the Corrections Board did not properly review this evidence and therefore could not meaningfully ascertain whether a probable material error or injustice had taken place. Id. Specifically, it was procedural error for the Corrections Board to fail to explain how the DeAustin, Dorger, and Hosmer letters, which contradict Gideon’s report, would not alter a determination that when “compared to other records reviewed during the evaluation board” Chisolm’s record was not “competitive for the revised [DP] PRF assessment,” AFR 31—11(x)(1)(b), in light of the reasons Gideon expressed. That is because while we cannot review the NLEB’s substantive conclusion, the Corrections Board must do so.
B.
Although the Air Force concedes it was improper for a promotion recommendation to be rendered by the NLEB when the candidate’s file lacked the most recent OPR, it maintains it acted in accordance with Air Force regulations when the Corrections Board forwarded Chisolm’s complete file for review by the NLEB president and then relied on his assessment that the missing OPR would not have made a difference in the NLEB’s promotion recommendation.
Air Force Regulation 31-11 provides in part
The president of the MLEB [here an NLEB] reviews the circumstances surrounding the issuance of the new PRF. Using whatever means are required, the MLEB president should certify [if correction is to be made] that compared to other records reviewed during the evaluation board, [the candidate’s] record would have been competitive for the revised PRF assessment if the circumstance which caused the original PRF assessment had not existed.
AFR 31—11(x)(1)(b) (1990) (emphasis added). Thus, we agree the Corrections Board did comply with the regulation by having Gideon review Chisolm’s complete file and render an opinion as to whether the missing OPR would have made a difference in the NLEB’s promotion recommendation. However, that the regulation provides that the evaluation board president may use “whatever means are required” to make his certification has no effect on what the Corrections Board must do upon receipt of the evaluation board president’s review. In this case, in addition to the evaluation board president’s report, the Corrections Board received numerous documents, including one from another member of Chisolm’s NLEB, supporting the contention that the missing OPR could in fact have made a difference, i.e., contradicting Gideon’s opinion. (Letter from Col. DeAustin to AFBCMR of 3/1/95, at ¶ 1). Also of significance, the Corrections Board received two other documents contradicting Gideon’s conclusion that Chisolm’s long tenure as Deputy SJA should be viewed as a negative. (Letter from Dorger to Sec. of the Air Force of 9/25/95, at 1-2; Letter from Hosmer [former Air Force Academy Superintendent] to Sec. of the Air Force of 8/28/95, at 1).
Gideon’s first of three reasons that the missing OPR would not have affected the NLEB’s promotion recommendation was because Chisolm had been a Deputy SJA from 1988-1993, longer than a normal tour. As Gideon phrased it, this was “a long time without advancement,” indicating that Chisolm’s five-year tenure as a Deputy SJA was a weak point in his record. However, the evidence submitted to the Board included letters that clearly indicate this conclusion was not sound:
[T]here is ... reference in a review by Brig Gen Gideon of Major Chisolm’s long tenure at the Air Force Academy (6 [400] years at the time of the board). That fact needs some clarification. First, the Academy is not a normal base, as a DRU it is a major command equivalent in terms of military justice issues. I doubt there was another major in the Air Force who was serving as the deputy SJA for a major command as Major Chisolm was or who was working in the range and level of issues Major Chisolm was. Second, some of the length of the service at the Academy was for the benefit of the Air Force. We changed Staff Judge Advocate in 1992 (Major Chisolm’s normal rotation time), and it would have been imprudent to change over both the SJA and deputy during a particularly sensitive period in Academy history (in terms of legal issues). Third, given that Major Chisolm was working issues of national significance, involving [¶] USAF, DOD, and congress, [sic] it is not clear to me that there was a real advancement opportunity for him until he was promoted. Almost any move in the Air Force would have been a step down at a critical point in his career.
Dorger Letter at 1-2; see also Hosmer Letter at 1 (explaining “[Chisolm] willingly supported the decision to extend him in a deputy position because of untimely turnover of Academy JAGs, despite the risk— now reality — that extended duty as a deputy could harm him in the eyes of a promotion board”). In response to the DeAustin letter of March 1995, the Corrections Board merely stated that “[insufficient relevant evidence has been presented to demonstrate the existence of probable error or injustice to warrant upgrading the PRF ... [and that the Corrections Board was] persuaded that the applicant was provided fair and equitable consideration for a ‘DP’ promotion recommendation and was not treated any differently from the other similarly situated individuals.” In re Chisolm, No. 94-03180 at 2 (Second Addendum to Record of Proceedings AFBCMR, Apr. 19, 1996). The Corrections Board neither then nor later discussed the content of any of the three letters. The Corrections Board thus failed to specifically address the information contained in DeAustin’s, Dorger’s, and Hosmer’s letters. Yet this information places the soundness of Gideon’s report directly at issue.6 Accordingly, we hold this failure to expressly analyze the evidence was procedural error.
DeAustin himself was a member of the NLEB making the promotion recommendation. In the same letter questioning Gideon’s interpretation of Chisolm’s Deputy SJA tenure, DeAustin explicitly states “I disagree with General Gideon’s conclusion that failure to include Major Chisolm’s 5 June 1993 Officer Performance Report (OPR) did not affect the Board’s decision .... ” (DeAustin Letter at 1). Thus, the Corrections Board had before it an express statement by another NLEB member that the missing OPR would have made a difference to the evaluation board’s majority in determining the appropriate promotion recommendation for Chisolm. So, one member of the NLEB, the president, indicated that the missing OPR would not have made a difference, and another member of the NLEB, DeAustin, who had written the missing report, indicated that it would have made a difference. The Corrections Board thus had two out of five members expressing an opinion. These opinions were diametrically opposed. Each member had an equal vote. [401] Thus, only three votes, out of five, were necessary to award a DP recommendation. We conclude that under the circumstances where the other three members were not consulted, the Corrections Board acted in an arbitrary and capricious manner in accepting Gideon’s opinion over DeAustin’s without any analysis of the DeAustin letter or the other two letters later submitted to it. See Houston v. United States, 156 Ct.Cl. 38, 297 F.2d 838, 842 (1962) (explaining the arbitrary and capricious standard is met when a court acts on “a mere whim or caprice”).
The harm flowing from this procedural error is amplified when the composition of the NLEB is closely examined in relation to Chisolm. Chisolm was a JAG officer. And, four of the five members were JAG officers. The only member of the NLEB with no connection to JAG and, understandably less understanding of JAG duties and assignments, was Gideon. Yet the Corrections Board accepted his opinion regarding whether the missing OPR would have affected the NLEB’s promotion recommendation absent any express analysis. We hold that where the Corrections Board’s responsibility was to determine whether a material error or injustice existed in Chisolm not receiving a DP recommendation, no analysis of the three contradictory letters constituted a procedural error by the Corrections Board.
Similarly, the Corrections Board’s treatment of the Dorger letter explaining another shortcoming of Gideon’s opinion is cause for concern. In Gideon’s report, he states that Chisolm’s “PRF, written by the Additional Rater on the missing OPR, who knew of his most recent performance, was lackluster compared to those accompanying most DP recommendations.” The Corrections Board noted that “the senior rater who signed the contested PRF provide[d] no support for upgrading the promotion recommendation.” In re Chisolm, No. 94-03180 at 2 (Addendum to Record of Proceedings AFBCMR, May 31, 1995). That was relevant because ordinarily the senior rater writes the Corrections Board in support of an applicant. Meanwhile, however, Dorger had retired and moved, so Chisolm did not initially submit a letter from Dorger. Later, when Dorger was found, he sent a strongly supportive letter to the Secretary of the Air Force, stating “I certainly supported his [Chisolm] getting a ‘Definitely Promote’ for the lieutenant colonel board .... ” (Letter From Dorger to Sec. of the Air Force of 9/25/95, at 1). Chisolm submitted this letter to the Corrections Board. However, in its next action, the Corrections Board merely stated that Dorger was entitled to his opinion that Chisolm deserved a DP recommendation, but since Dorger did not have any DP’s to distribute, it was up to the NLEB president to certify whether Chisolm’s complete record would have warranted a DP recommendation. The Corrections Board opinion did not even mention Dorger’s disagreement with Gideon’s appraisal of the PRF as “lackluster.” Indeed, Dorger stated that the PRF was “not intended to be [lackluster in comparison to others]. It was meant to be strong and competitive.” Dorger Letter at 1. This goes to the heart of the matter: it disputes Gideon’s characterization of Chisolm’s complete record of performance. That the Corrections Board summarily discounted this letter further supports our determination that the Corrections Board committed a procedural error, thereby acting in an arbitrary and capricious manner. It simply adopted Gideon’s opinion without any explanation of why it was not overwhelmed by the contrary evidence.
We are equally troubled by the Board’s decision to adopt Gideon’s opinion without any evidence indicating that Gideon ever [402] compared Chisolm’s complete record of performance with the records of the competing candidates before the evaluation board nearly one year earlier. Gideon simply states that “[Chisolm’s] total ROP would not change substantially in relation to his peers’ records which the board reviewed.” It is unclear to us, indeed, how such a conclusion could be reached without review of his peers’ records. We recognize that Gideon has experience evaluating other officers for promotion. And true, Gideon can make general representations as to what normally is found in the records of DP-recommended candidates. However, the language of Gideon’s opinion that “[Chisolm’s] PRF ... was lackluster compared to those accompanying most DP recommendations” (emphasis added) indicates that Gideon was comparing Chisolm’s PRF to those of all promotion candidates he has considered in the past and not the relevant pool of officers — those Chisolm was competing against at the time the NLEB initially reviewed his file. Further, it is difficult to see how Gideon possibly could have “certified] that compared to other records reviewed during the evaluation board, [Chisolm’s] record would have been competitive for the revised PRF [DP] assessment if the circumstance which caused the original PRF [P] assessment had not existed,” AFR 31-11(x)(1)(b), without also reviewing the records of the candidates to whom Chisolm must be compared. As the government points out, the regulation does not explicitly require an NLEB president to personally review other officers’ records, but the implication is quite obvious, as he “should certify” that a promotion candidate would have been competitive for the higher recommendation had the circumstances which caused the lower recommendation not existed. Such an implication is particularly acute where the corrective review is, as here, almost one year later. How could Gideon remember the records of Chisolm’s competitors? In so concluding, however, we do not mean to suggest that an MLEB president or the Corrections Board must always track down and personally review the records of the other individuals who were evaluated along with the individual claiming an error, as such a task would be difficult and the regulation (AFR 31—11(x)(1)(b)) does not require it. Nor need the Corrections Board always consult all members of an evaluation board. But, the facts here make this case exceptional.
While Gideon’s analysis may have been sound, given what he knew,7 the Corrections Board’s was not, for it knew more.
There is nothing in the regulation that limits the Corrections Board to considering only the MLEB president’s opinion. See AFR 31-11(x)(1)(b). Indeed, in a situation such as this, where the president is a line officer on a non-line evaluation board and apparently unfamiliar with the reasons a promotion candidate may have remained in a position longer than his peers (such as a candidate’s unusual skill in his position) and the Corrections Board is given three letters which expressly contradict the evaluation board president’s opinion, the Corrections Board’s failure to require input from the other three NLEB members and to expressly analyze the conflicting evidence constitute a faulty adjudicatory process. Thus, we hold that the Corrections Board acted in an arbitrary and capricious manner in adopting Gideon’s report without any express analysis of the three conflicting letters.
[403] The Corrections Board’s treatment of this case lacked analysis of the considerable evidence before it which contradicted Gideon’s report. Not once in three subsequent opinions did the Corrections Board address the substance of this evidence. All the Corrections Board did was repeat the legal conclusion that “insufficient relevant evidence has been presented to demonstrate the existence of probable error or injustice.” Such treatment of seemingly powerful evidence is arbitrary and capricious where, as here, the Corrections Board must determine whether the contents of these letters demonstrate that a probable material error or injustice existed in Gideon’s determination that Chisolm’s record would not have been competitive, even considering the missing OPR.
Ill
The trial court states “[t]hus, the AFBCMR did not act arbitrarily and capriciously in relying on General Gideon’s opinion rendered pursuant to Air Force regulations. The AFBCMR reviewed plaintiffs application and all the evidence of record and found ‘[ijnsufficient relevant evidence ... to demonstrate the existence of probable error or injustice.’ ” Chisolm, 49 Fed.Cl. at 619. This logic fails for the same reason the government’s arguments do: just because Chisolm’s complete ROP was forwarded for Gideon’s review in accordance with regulation, does not mean that the Corrections Board need not analyze the documentary evidence supplied by Chisolm when it was determining whether a probable material error or injustice was established. The regulations do not limit the Corrections Board’s actions to submitting the complete ROP to the evaluation board president. When considerable evidence contradicting the evaluation board president’s opinion exists, the Corrections Board must do more than simply accept the president’s opinion; it must resolve the conflicts.
The trial court’s decision thus suffers from the same deficiency as the Corrections Board decision: it merely mentions the contradictory evidence without any analysis, instead choosing to merely state that the Corrections Board “considered” this evidence. Id. Accordingly, under the exceptional facts of this case, we find that the trial court erred in holding that the Corrections Board did not act in an arbitrary and capricious manner when the Board failed to discuss the import of the applicant’s evidence.
IV
Because we find that the Corrections Board acted in an arbitrary and capricious manner by failing to expressly analyze the three letters offered, serially, by Chisolm, we vacate the Court of Federal Claims’ decision and remand this case to the trial court, instructing it to, in turn, remand to the Corrections Board for further proceedings consistent with this opinion.
Again, we make no determination on the substantive effect, if any, of the letters on revision of Chisolm’s promotion recommendation. On remand, a thorough, comparative analysis of the Dorger, DeAustin, and Hosmer letters and Gideon’s report must be conducted to determine whether a probable material error or injustice was shown. This includes an explanation of the relevance of the letters and, if the Corrections Board renders the same decision absent further investigation, how, given the shortcomings of Gideon’s report indicated by these letters, there was nevertheless sufficient evidence to show that Chisolm failed to establish probable error or injustice. Of course, the Corrections Board is free to gather additional evidence, such as the opinions of the remaining three evaluation board members as to whether [404] Chisolm’s missing OPR would have made a difference in their votes on a promotion recommendation.