ORDER RE PLAINTIFF’S MOTION TO SUPPLEMENT THE ADMINISTRATIVE RECORD (DOC. 71)
LAWRENCE J. O’NEILL, District Judge.
I. INTRODUCTION
Plaintiff Pinnacle Armor, Inc. (“Pinnacle”) produces armor designed to protect buildings, vehicles, and the human body. Among Pinnacle’s primary customers are local law enforcement agencies that often utilize federal subsidies to purchase body armor. Availability of at least one such subsidy is conditioned upon certification that the body armor was manufactured in compliance with the most recent standards set by the National Institute of Justice (“NIJ”), an arm of the U.S. Department of Justice (“DOJ”). In its Verified First Amended Complaint (“FAC”), Pinnacle alleged that NIJ’s decision to revoke certification for one of Pinnacle’s products: (1) violated Pinnacle’s procedural due process rights under the Fifth Amendment; and (2) was “arbitrary and capricious” in violation of the Administrative Procedure Act (“APA”), 5 U.S.C. § 706(2)(A): Doc. 6.1
On March 11, 2008, 2008 WL 686857, the district court dismissed both claims, holding that Pinnacle’s interest in NIJ certification is not a protected property right under the due process clause, and that NIJ’s certification decision is exempt from review under the APA because the certification process is “committed to agency discretion by law,” 5 U.S.C. § 701(a)(2). Doc. 22. Pinnacle appealed. Doc. 24. The Ninth Circuit affirmed on the Fifth Amendment claim, but reversed and remanded on the APA claim, directing the district court to conduct further proceedings. Doc. 35, filed May 26, 2011, Pinnacle Armor, Inc. v. United States, 648 F.3d 708 (2011).
On May 31, 2012, Defendant, the United States of America, moved to dismiss the remaining APA claim pursuant to Fed. R.Civ.P. 12(b)(1), arguing that subsequent regulatory activity rendered moot any dispute about its decision under a now-superseded regulation. Doc. 53. That motion was denied without prejudice. Doc. 61.
[1231]*1231On November 26, 2012, Defendant’s third motion to dismiss.this case as moot was denied, as was Defendant’s motion for summary judgment that Plaintiff lacked standing to sue. Doc. 69. Defendant also moved for summary judgment on the merits of the APA claim, arguing that it lawfully revoked certification for Dragon Skin. Doc. 64-1. In response, Plaintiff requested deferral of any merits ruling to allow time for Plaintiff to obtain additional evidence. The Court denied Plaintiffs generic request for discovery. Doc. 69 at 22. Although the Court found the" 56(d) showing to be “weak,” the Court nevertheless deferred decision on the merits-to “permit a closer examination of whether the Revised Administrative Record [(“RAR”)], Doc. 60, should be supplemented to include documents already in Plaintiffs possession.” Doc. 69 at 22.2 .
II. DISCUSSION
A. Legal Standard.
The APA limits the scope of judicial review to the administrative record. 5 U.S.C. § 706 (directing the court to “review the whole record or those parts of it cited by a party”); Fla. Power & Light Co. v. Lorion, 470 U.S. 729, 743-4, 105 S.Ct. 1598, 84 L.Ed.2d 643 (1985) (“The task of the reviewing court is to apply the appropriate APA standard of review ... to the agency decision based on the record the agency presents to the reviewing court.”). The appropriate scope of review is normally limited to “the administrative record in existence at the time of the [agency] decision and [not some new] record that is made initially in the reviewing court.” Lands Council v. Powell, 395 F.3d 1019, 1030 (9th Cir.2005) (quoting Southwest Center for Biological Diversity v. United States Forest Service, 100 F.3d 1443, 1450 (9th Cir.1996)).
Nevertheless, the administrative record prepared by the agency may be supplemented by extra-record materials in an APA case under four narrow exceptions:
(1) when it needs to determine whether the agency has considered all relevant factors and has explained its decision;
(2) when the agency has relied upon documents or materials not included in the record;3
(3) when it is necessary to explain technical terms or complex matters; and
(4) when a plaintiff makes a showing of agency bad faith.
Southwest Center, 100 F.3d at 1450.
“These limited exceptions operate to identify and plug holes in the administrative record.” Lands Council, 395 F.3d at 1030. Yet, “[t]he scope of these exceptions permitted by [Ninth Circuit] precedent is constrained, so that the exception does not undermine the general rule.” Id.
Were the federal courts routinely or liberally to admit new evidence when reviewing agency decisions, it would be obvious that the federal courts would be proceeding, in effect, de novo rather [1232]*1232than with the proper deference to agency processes, expertise, and decision-making.
Id. This is because agency action, including designation and certification of an administrative record, is entitled to a “presumption of regularity.” See McCrary v. Gutierrez, 495 F.Supp.2d 1038, 1041 (N.D.Cal. 2007) (citing Bar MK Ranches v. Yuetter, 994 F.2d 735, 739-40 (10th Cir.1993) (while the agency “may not unilaterally determine what constitutes the administrative record” the courts “assume[] the agency properly designated the [AR] absent clear evidence to the contrary”)); see also Ctr. for Native Ecosystems v. Salazar, 711 F.Supp.2d 1267, 1274 (D.Colo.2010). The party seeking supplementation bears the burden of overcoming this presumption by “clear evidence.” See Bar MK Ranches, 994 F.2d at 740; Glasser v. NMFS, 2008 WL 114913, *1 (W.D.Wash. Jan. 10 2008); In re Delta Smelt Consolidated Cases, 2010 WL 2520946, *2 (E.D.Cal. June 21, 2010).
B. “Undisputed” Documents.
Plaintiff seeks to supplement the RAR with a total of twenty-six (26) documents. Of these, Defendants agree that one document should be added to RAR and point out that three additional documents are already included in the RAR.
a. Document 8.
Document 8 is a highly redacted email thread dated June 28, 2007 regarding Dragon Skin. Defendant does not object to inclusion of Document 8 in the RAR and has attached an unredacted version to its opposition brief. The motion to supplement the record with the unredacted version of Document 8 is GRANTED.
b. Documents 9, 10, and 11.
Versions of Documents 9, 10, and 11 are already in the record. See RAR at 363, 364, 368, 427, 428. Plaintiff nevertheless requests that the record be supplemented to include their versions of these Documents because they “have in some cases been redacted differently.” Doc. 80 at 1. This does appear to be the case with respect to Document 10, which is redacted in a significantly different way from the copy in the RAR at 368.4 The request to supplement is GRANTED as to Document 10.
The request to supplement is DENIED as to Document 9. Although the redactions are different, the version in the RAR at page 363 only redacts a single sentence indicating that the author has been in contact with the Office of General Counsel regarding the subject matter of the communication. This cannot possibly be relevant to the dispute in this case and the redaction was clearly made in accordance with the attorney client privilege.
The request to supplement is also DENIED as to Document 11, which appears to be identical to the version already in the record at RAR 427-30.
C. Disputed Documents.
Plaintiff offers several alternative grounds for supplementation as to most of the remaining 22 Documents:
• Document 1, e-mail dated May 11, 2006 from Murray Neal to Alex Sundstrom (aka Richard Sundstrom) for NIJ.
• Document 2, Build Sheets for: SOV2000.1 Mil Series Vest with certifications under 05 Interim Require[1233]*1233ments; and SOV2000.1/MIL3AF01 Series Vest with certifications under 05 Interim Requirements.
• Document 3, Build Sheets for: SOV2000 Mil Series Vest (pre-compliance testing); and SOV2000.1/MIL3AF01 Series Vest (pre-compliance testing).
• Document 4, Build Sheets for: Survivor 2000 Series SO Tactical. Vest (all Aramid); Survivor 3000 Series SO Tactical Vest (all Polyethylene); Hybrid Mil Series SO Tactical Vest (Military Hybrid).
• Document 5, e-mail thread dated June 20, 2007 re Pinnacle Letter Regarding Labeling Discrepancies and Removal of SOV2000.1/MIL3AF01 from NIJ Compliance List.
• Document 6, Letter dated July 10, 2006 from Lance Miller, Deputy Director of the National Law Enforcement and Corrections Technology Center enclosing draft test protocols for flexible rifle defeating body armor under NIJ standard 0101.04 and NIJ 2005 interim requirements.
• Document 7, Final test protocols for flexible rifle defeating body armor dated December 19, 2006 under NIJ standard 0101.04 and NIJ 2005 interim requirements.
• Document 12, Letter dated October 11, 2007 from Raj Koria of Morgan Advanced Ceramics Ltd.
• Document 13, Chart of details on tested vests.
• Document 14, Graphical chart of V50 warranty test data prepared 2007 and generated from reports from United States Test Labs included in administrative record.
• Document 15, Pinnacle Armor Level 3 High Heat/Cold Temperature Cycling Testing dated December 4, 2007.
• Document 16, Level 3 adhesive temperature properties and peel adhesion testing data.
• Document 17, Excerpts from Flexible Body Armor SBIR Phase I Report (June 2003).
• Document 18, Projectile information ' excerpted from Pinnacle Proprietary report (undated).
• Document 19, P. Dole, J. Chauchard, Thermal Aging of Polyethylene-co-methyl Acrylate Elastomer Activation Energies of Physical and Physicochemical Mechanisms, 65 J. Applied Polymer Sci. 2507-2515 (1997).
• Document 20, Energy Chart delivered by projectiles based on Knudsen and Sorenson study.
• Document 21, Department of Defense Test Method Standard, V50 Ballistic Test for Armor (MIL-STD-662F) (December 18,1997).
• Document 22, Article re Pinnacle Dragon Skin SOV-2000 Test by SWAT, Coast Guard, & Navy: December 7, 2006.
• Document 23, Press Release from DOJ, Department of Justice Announces Body Armor Challenge (September 14, 2012), Digital Journal, www.digitaljournal.com/pr/ 884733.
• Document 24, declaration of Prof. Edward Rosenberg, dated November 22, 2012.
• Document 25, declaration of Nevin Rupert, dated November 21, 2012.
• Document 26, declaration of Murray Neal, dated November 23, 2012.
1. Necessary to Determine Whether Agency has Considered All Relevant Factors.
Plaintiff maintains that Documents 1-7, 12-16, and 22-23 should be [1234]*1234considered under the Southwest Center “relevant factors” exception. Plaintiff explains in its opening brief why each document is arguably relevant to NIJ’s .assessment of whether its Dragon Skin would perform over the duration of its declared warranty. But to satisfy the “relevant factors” exception, a plaintiff must establish more than just that the document is relevant. In fact, the document in question must do more than raise “nuaneed points” about a particular issue; it must point out an “entirely new” general subject matter that the defendant agency failed to consider. In re Delta Smelt Consolidated Cases, 2010 WL 2520946 at *5. For example, in the Delta Smelt Consolidated Cases, the district court refused to supplement the record with a scientific article that pointed out how a certain species of predatory fish was prevalent in marsh edge habitat and the impact those predators have on juvenile endangered and threatened salmon migrating through that habitat. Id. The agency already considered the fact that this habitat supports predators; the offered scientific article merely rose a “nuanced point” about predation. Id.
The “relevant factors” exception only applies when Federal Defendants fail to consider a general subject matter that is demonstrably relevant to the outcome of the agency’s decision, not when specific hypotheses and/or conclusions are omitted from consideration. To hold otherwise would allow Plaintiffs to drive a truck through what is supposed to be a narrow exception to the record review rule.
Id.
5
Documents 1-4.
One of the concerns raised by NIJ about Dragon Skin was the lack of detailed information on construction and manufacturing. Plaintiff argues, therefore, that Documents 1-4 should be considered under the relevant factors exception because they contain information about the manufacturing and construction of several variants of Pinnacle Armor using the Dragon Skin technology. But this is not an argument about whether NIJ neglected to consider the general subject matter of “manufacturing and construction.” Rather, Plaintiff simply argues that NIJ should have considered this particular information about manufacturing and construction. This does qualify these documents for consideration under the “relevant factors” exception.
Document 5.
Document 5 is an email thread dated June 20, 2007 bearing the subject line “Pinnacle Letter Regarding Labeling Dis[1235]*1235crepancies and Removal ' -■ of SOV2000.1/MIL3AF01 from NIJ Compliance List.” Plaintiff claims that Document 5 reveals that as of June 20, 2007, “NIJ ha[d] already reached the decision that it was going to evoke Pinnacle Armor’s body armor compliance even though the NIJ did not send the first letter to Pinnacle Armor requesting warranty related data until June 22, 2007.” Doc. 71 at 6. Nowhere does Plaintiff explain why Document 5 should be admitted under the relevant factors exception.
Documents 6 & 7.
Documents 6 and 7 are “the draft testing protocols and the final testing protocol developed by the NIJ for flexible rifle defeating body armor made by Pinnacle Armor pursuant to NIJ standard 0101.04 and the 2005 interim requirements.” Doc. 71 at 7. Plaintiff asserts generally that “the NIJ clearly considered these protocols either directly or indirectly and such knowledge of the protocols appear[s] throughout the pre-decisional documents and other documents found within the revised administrative record.” Id. Again, this argues for consideration of particular information related to a subject matter that was considered by the agency; it does not point out a factor that the agency entirely neglected to consider. The relevant factors exception does not warrant supplementation of the RAR with these two Documents.
Documents 12-16, and 22.
According to Plaintiffs own descriptions: Document 12 is a communication that demonstrates Dragon Skin performed well under temperature and flexing demonstrations; Document 13- provides detailed information on' the various body armor vests being subjected to testing in 2007; Document 14 consists of two charts prepared in September 2007, based upon data in, the RAR, presenting in graphical form the . performance results . of aged vests; Document 15 details temperature and flexing tests conducted in December 2007 on an eight-year old vest; Document 16 is a lab report which tested Dragon Skin technology and documented the results relative to temperature, including information on “peel adhesion”; finally, Document 22 is an article detailing test results from December 2006. Plaintiff argues that the information in these Documents “goes directly to the performance characteristics- of concern to law enforcement, as opposed to the military.” Doc. 71 at 10. Yet, agáin, this does nothing to trigger the “relevant factors” exception, which permits supplementation only to demonstrate that the agency entirely failed to consider a general subject matter relevant to its decision-making.
Documents 17-21.
According to Plaintiff, Document 17 is an excerpt from a report authored by Pinnacle’s CEO Murray Neal in 2003 which provides additional information and analysis on several issues raised in NIJ’s “predecisional” reports6; Document 18 is' an excerpt from Pinnacle Armor’s proprietary information providing specific and detailed data on projectiles which are referenced in the NIJ’s pre-decisional reports; Document 19 is specifically cited in'the predecisional reports discussing the Arrhenius equation; • and Document 20 is a chart developed from a study referenced in the NIJ’s pre-decisional reports.
[1236]*1236As to Documents 17-20, Plaintiff’s own descriptions preclude application of the “relevant factors” exception, as they admit that the subject matters discussed therein were addressed in NIJ’s own pre-decisional reports. Therefore, they do not raise “factors” the agency failed to consider.
Document 21 is a Department of Defense document setting forth the military standards for V60 ballistic testing. According to Plaintiff, the agency discusses ballistic performance issues at length in the RAR. Doc. 71 at 12. However, Plaintiff maintains that NIJ’s discussion of that issue is “ultimately irrelevant because the degrees of the ballistic yaw of the projectiles referenced do[ ] not exceed the rejection criteria utilized by the NIJ in its 0101.06 standard, a draft of which had been circulating as early as 2006. The rejection criteria used is no more than 5° which has been the military standard as set forth in Document 21 since 1997.” Id. Presumably, Document 21 supports this assertion because it presents the military’s own rejection criteria. The NIJ’s 0101.06 standard was published in 2008, Pinnacle Armor, 648 F.3d at 714, which post-dates the closure of the record, so cannot be used to point out the military’s rejection criteria. It is unclear, however, why Plaintiffs have not offered a draft version of the NIJ’s 0101.06 standard, if it truly was available as of 2006. Regardless, Document 21 was in existence at the time of the decision(s) in this case and does arguably raise a factor (namely, the military’s rejection criteria) that may not have been considered by the agency.7 The request to supplement the RAR with Document 21 is GRANTED.
Document 23.
Document 23 is a press release Dated September 14, 2012, which announces a “Challenge for scientists,” inviting them to come up with tools to help determine when an officer’s body armor needs to be replaced. Among.other things, the release states that “[bjeeause the safety of officers is the ultimate priority, the service life of a vest should be determined based on its actual performance, not on general warranty.” Plaintiff entirely fails to explain why this document, which is obviously post-decisional, should be admitted under the relevant factors exception.8
2. Documents Plaintiff Claims Should Have Been Included in the Record.
Plaintiff contends that many of documents it offers should be included as part of the “whole record” because they were “generated by the NIJ or submitted to the NIJ at or about the time of its decision.” Doc. 71 at 3. In addition, in a separate section of its opening brief, Plaintiff specifically invokes the second Southwest Center exception, which permits supplementation of an administrative record “when the agency has relied upon documents or materials not included in the record.” Doc 71 at 13. These two sets of arguments will be treated together, as they invoke essentially the same standards.
a. General Legal Standard for Inclusion in the “Whole Record.”
The administrative record is “not necessarily those documents that the agency has compiled and submitted as ‘the’ [1237]*1237administrative record.” Thompson v. U.S. Dept. of Labor, 885 F.2d 551, 555 (9th Cir.1989). Rather, “[t]he ‘whole’ administrative record ... consists of all documents and materials directly or indirectly considered by agency decision-makers and includes evidence contrary to the agency’s position.” Id. (emphasis added).9
An incomplete record must be viewed as a fictional account of the actual decision-making process. When it appears the agency has relied on documents or materials not included in the record, supplementation is appropriate.
Portland Audubon, 984 F.2d at 1548 (internal quotations and citations omitted). However, the record does not include “every scrap of paper that could or might have been created” on a subject. TOMAC v. Norton, 193 F.Supp.2d 182, 195 (D.D.C. 2002).
A broad application of the phrase “before the agency” would undermine the value of judicial review: Interpreting the word “before” so broadly as to encompass any potentially relevant document existing within the agency or in the hands of a third party would render judicial review meaningless. Thus, to ensure fair review of an agency decision, a reviewing court should have before it neither more nor less information than did the agency when it made its decision.
Pac. Shores Subdivision v. U.S. Army Corps of Eng’rs, 448 F.Supp.2d 1, 5 (D.D.C.2006) (internal citations and quotations omitted).
b. Documents Submitted to NIJ After September 28, 2007 (Documents 12, 13, 14, 15, and 16).
The parties dispute the relevant time period that should represent the temporal scope of the RAR. The following is a timeline, based in part upon material presented Pinnacle’s opening brief, reflecting public communications regarding the revocation of Dragon Skin’s certification:
June 22, 2007 Letter from John Morgan (NIJ) to Pinnacle requesting data or other objective evidence that supports Pinnacle’s belief that Dragon Skin will maintain its ballistic performance over its declared warranty period of six years. The letter sets a deadline of June 28, 2007 for _submission of materials to NIJ’s Alex Sundstrom. RAR 336._
June 27, 2007 Pinnacle sends its first submission of data to Sundstrom and notes _additional information to follow. RAR 338._
August 3, 2007 NIJ notifies Pinnacle that the data submitted on June 27, 2007 is insufficient to satisfy NIJ that the model will maintain its ballistic performance over its declared warranty period. NIJ issues press _release regarding same on this date. RAR 415._
August 9, 2007 Letter from Pinnacle to NIJ’s Sundstrom providing additional information regarding warranty performance (the “second _submission”). RAR 422. _
August 15, 2007 Office of Law Enforcement Standards sends “pre-decisional” report to _NIJ analyzing the second submission. RAR 435.'_
August 21, 2007 E-mail from Pinnacle to Sundstrom providing additional information _regarding warranty performance (the “third submission”). RAR 450.
[1238]*1238August 31, 2007 Office of Law Enforcement Standards sends “pre-decisional” report to _NIJ analyzing the third submission. RAR 479,_
September 18, 2007 E-mail from Pinnacle to Sundstrom providing additional information _regarding warranty performance (the “fourth submission”). RAR 513.
September 28, 2007' Letter from NIJ’s John Morgan to Pinnacle advising that based on analysis of second and third submissions (and not the fourth submission), the data provided is insufficient to satisfy the NIJ. After providing reasons, the letter invites submittal of additional information _which the NIJ will then review. RAR 543._
December 2007 Pinnacle armor sends data to Sundstrom regarding high heat cold __temperature mechanical testing data performed in 2007 and in 2002.
• The last document; chronologically, contained in 'the RAR is the September 28, 2007 letter from John Morgan at NIJ, which analyzes at least some of the data submitted by Plaintiff up to that point and refuses to reinstate Dragon Skin on the compliance list, but invites further submission of data. Plaintiff contends that, in response to Morgan’s letter, they submitted additional -data in December 2007, namely Documents 12, 13, 14, 15, 16, and 22. Accordingly, Plaintiff maintains that these documents should be included in the RAR.
An administrative record need not include documents that became available after the agency made, its decision (“postdecisional” documents). See Vermont Yankee Nuclear Power Corp. v. NRDC, 435 U.S. 519, 555, 98 S.Ct. 1197, 55 L.Ed.2d 460 (1978) (judicial review is “limited [] by the time at which the decision was madfe....”); see also Airport Communities Coal. v. Graves, 280 F.Supp.2d 1207, 1213 (W.D.Wash.2003) (court may not consider information created during the litigation that was not available at the time the agency made its decision) (citations omitted).
Defendant determined that Dragon Skin would be removed from its compliance list on August 3, 2007 and so informed Plaintiff by letter. RAR 415. This was the decision date. Plaintiff submitted additional data regarding the warranty issue. RAR 422. There followed a back and forth whereby NIJ considered additional data in an effort to determine whether a different decision should be made. NIJ’s September 28, 2007 letter informed Plaintiff NIJ was willing to review additional data if Pinnacle decided to continue to seek reinstatement and identified the data that would be required. RAR 543.
The decision to revoke compliance occurred on August 3, 2007. NIJ decided it would be appropriate to include in the RAR data submitted in August and September, as well as its own responses thereto. However, Plaintiff has not cited any authority suggesting the record regarding an agency decision must remain open indefinitely whenever an agency invites a party to submit additional information on the same subject. Instead, Plaintiff relies on the-well-established rule that when a court considers a claim that an agency has failed to act in violation of a legal obligation, “review is not limited to the record as it existed at any single point in time, because there is no final agency action to demarcate the limits of the record.” Doc. 80 at 4-5 (citing Friends of the Clearwater v. Dombeck, 222 F.3d 552, 560 (9th Cir. 2000)). However, there is fundamental flaw in Plaintiffs reasoning: they have not alleged a “failure to act” claim in this case. Neither the original complaint, Doc. 2, filed November 16, 2007, before any of the documents in question were purportedly sent to NIJ, nor the FAC, Doc. 6, filed December 7, 2007, right around the time at which any documents were purportedly [1239]*1239sent to NIJ, contain any allegations that could even arguably encompass a failure to act (e.g., a failure to re-instate) based upon information submitted in December 2007.
Moreover, Plaintiff has failed to establish that relevant decision- makers at the agency ever received the additional data contained in Documents 12, 13, 14, 15, 16, and 22.
In order to overcome the presumption that the record was [] properly designated, plaintiff must put forth concrete evidence. Plaintiff cannot merely assert that other relevant documents were before the [agency] but were not adequately considered. Instead, plaintiff must identify reasonable, non-speculative grounds for its belief that the documents were considered by the agency and not included in the record. Finally, plaintiff must do more than imply that the documents at issue were in the [agency’s] possession. Rather, plaintiff must prove that the documents were before the actual decision makers involved in the determination.
Sara Lee Corp. v. Am. Bakers Ass’n, 252 F.R.D. 31, 34 (D.D.C.2008) (internal citations and quotations omitted); see also Ctr. for Native Ecosystems v. Salazar, 711 F.Supp.2d 1267, 1275 (D.Colo.2010) (“To overcome the presumption of regularity and meet the burden of proving that the record designated by the agency is incomplete, Petitioners must clearly set forth in their motion: (1) when the documents were presented to the agency; (2) to whom; (3) and under what context.”). The agency claims to have no record of receiving any additional data in December 2007. The only evidence cited in support of Plaintiffs contention that it did submit the data to the agency is the most recent Declaration of Murray Neal, in which Neal indicates the documents were either “provided to NIJ in 2007” or in some cases “provided to the NIJ in or about December 2007.” Document 26, ¶¶ 11-15.10 Plaintiff provides no proof of mailing, cover letter,, sent email, or other evidence establishing that the data was ever sent, let alone received by anyone at NIJ..
Plaintiff has failed to meet its burden to establish that Documents 12, 13, 14, 15, 16 and 22 were before the relevant decision makers at the time of any decision challenged in the FAC.
c. Other Documents Submitted to or Created by NIJ (Documents 1-7 and 19-21).
Plaintiff also seeks to supplement the ,RAR with additional documents it claims were submitted to or created by NlJ and were “directly or indirectly considered by the NIJ” in connection with the agency decision challenged here. Doc. 71 at 6-7,13.
Defendant does not appear to dispute that these -documents were in the agency’s possession. But, that is not enough. “Plaintiff must do more than imply that the documents at issue were in the [agency’s] possession.” Sara Lee Corp. v. Am. Bakers Ass’n, 252 F.R.D. 31, 34 (D.D.C.2008). “Plaintiff must identify reasonable, non-speculative grounds for its belief that the documents were considered” by the decision makers involved in the determination. Id.; see also Dunn v. FDIC, 2012 WL 1986042, *5 (C.D.Cal. May 31, 2012) (“the crux of the analysis is whether the documents or materials ... were actually considered, directly or indirectly, but the agency decision makers”) (quoting Pac. Coast Fed’n of Fishermen’s [1240]*1240Ass’n/Inst. for Fisheries Res. v. Gutierrez, 2007 WL- 1752287 (E.D.Cal. June 15, 2007)). Plaintiff has offered no “non-speculative” grounds for such a finding as to any of these documents.
Documents 1, 2, 3, and 4 are technical documents pertaining to the manufacturing and construction details of several variants of Dragon Skin armor. Although these documents may very well have been in NIJ’s possession, Plaintiff has not met its burden to establish that these documents were considered, either directly or indirectly, by the relevant decision makers.
Documents 6 and 7 are “draft testing protocols and the final, testing protocol developed by the NIJ for flexible rifle defeating body armor made by Pinnacle Armor pursuant to NIJ standard 0101.04 and the 2005 interim requirements.” While both the 0101,04 standard and the 2005 interim requirements were included in the RAR, the flexible rifle defeating body armor protocols were not.” Doc. 71 at 7. Plaintiff asserts generally that “the NIJ clearly considered these protocols either directly or indirectly and such knowledge of the protocols appear throughout the pre-deeisional documents and other documents found within the revised administrative record.” Id. But this is a far cry from identifying non-speculative grounds for its belief that the documents were considered, either directly or indirectly, by the decision makers.
Document 19 is a scientific paper on thermal aging of a certain type of elastomer material which was “specifically cited by the NIJ in its pre-decisional report.” Doc. 71 at 13. Defendants cite Marcum v. Salazar, 751 F.Supp.2d 74, 80 (D.D.C.2010), for the proposition that “references to documents in the administrative record do not prove that that [those] documents were ‘before’ the deciding agency.” But, in so holding, Marcum relied upon WildEarth Guardians v. Salazar, 670 F.Supp.2d 1, 6 (D.D.C.2009), which specifically distinguished between “citations” to extra-record documents in the decision document, which “indicate consideration of the contents of the [extra-record] document” by decision makers, and “references” to such documents, which do not. Marcum also relied wpon Cape Halteras Access Preservation Alliance v. U.S. Department of Interior, 667 F.Supp.2d 111, 114 (D.D.C.2009). Cape Hatteras discussed “references” to a biological opinion (“BiOp”) in the record supporting issuance of an Interim Strategy for protection of certain species, holding those references did “not prove that [the BiOp] was before the agency when it made its decision”:
Despite that both parties agree that the BiOp was heavily relied upon in preparation of the Interim Strategy, that fact alone does not mean it was actually before the agency when it made its current decision. And despite the references to the BiOp in many documents that were contained in the administrative record, this too does not prove that it was before the agency when it made its decision. It may well have been that the Interim Strategy obviated the need to consider the BiOp independently, as much of the relevant information from the BiOp was included in the Interim Strategy. Additionally, the BiOp’s acknowledgment of the proposed critical habitat designations, and the effect that would have on the Interim Strategy, does not prove that it was considered by the agency when making the critical habitat designation. Finally, the fact that some comments received during the critical habitat designation process mentioned the BiOp, does not mean that the BiOp itself was considered by FWS. Because none of these references alone are enough to overcome the strong presumption that FWS properly designated the administrative record, and the plain[1241]*1241tiffs have not introduced any concrete evidence that the BiOp was before the agency, the plaintiffs’ motion to supplement the record shall be denied.
Id. Together, these cases stand for the limited proposition that an extra-record document that is cited in the agency’s actual decision document indicates “consideration of the contents of the [extra-record] document” by the decision-maker. Otherwise, a mere reference in the administrative record is insufficient. Here, Plaintiff merely indicates that Document 19 is “cited in the NIJ’s pre-decisional reports.” Plaintiff does not cite to the portions of the record in which Document 19 is referenced and does not otherwise demonstrate that it is cited in any document that predates or was drafted on the August 3, 2007 decision date in this case. Plaintiff must provide “non-speculative grounds for their belief that the [agency] actually considered the [ ] materials.” Marcum, 751 F.Supp.2d at 81. It has not done so with respect to Document 19.11
Document 20 is a chart developed by Murray Neal based upon a study referenced in NIJ’s pre-decisional reports. Document 21 is a Department of Defense document setting forth the military standards for V60 ballistic testing. Plaintiff completely fails to explain why Documents 20 and 21 should be admitted under the “considered but excluded” exception.
The final document Plaintiff lists under its request for supplementation pursuant to the “considered but excluded” exception is Document 5, a June 20, 2007 email thread concerning coordination of NIJ’s decertification decision with a press release. Plaintiff does not even appear to attempt to suggest that this document was considered, either directly or indirectly, by the relevant decision makers. Rather, Plaintiff suggests it is admissible under the bad faith exception. This will be examined separately below.
d. ArmyIDepartment of Defense Documents.
Plaintiff, in its opening brief, “notes that throughout the [RAR], comments are made that the raising of the perceived warranty concerns by the NIJ was in fact based upon information received from the Army and/or Air Force testing.” Doc. 71 at 10. Yet, Pinnacle maintains, “[t]here does not appear to be any data or information from the Army to support that there was a legitimate concern .... ” Id. Pinnacle argues “[s]uch information should have been included by the NIJ in the [RAR].” Id.12
Defendant responds that these documents are contained in the RAR, at pages 229-98. Doc. 78 at 11-12, n. 7. This does not appear to be correct, as those pages consist of a transcript of testimony at a hearing of the House Armed Serviced Committee as well as a prepared statement by NIJ’s John Morgan. These referenced pages of the RAR do not appear to contain any information or data from either the Army or the Department of Defense (“DOD”). Yet, Defendant suggests that “objective evidence” to support its initial concerns was made publicly available as part of this hearing. Id.
This raises non-speculative grounds to believe that NIJ considered additional data from the Army or DOD that has not [1242]*1242been included in the RAR. The problem is that Plaintiff appears not to be in possession of any such documents so therefore cannot offer to supplement the record with them. The Court will permit Plaintiff to file a very narrow motion for discovery to uncover whether NIJ considered additional data from the Army or DOD.
3. Bad Faith Exception (Document 5).
Plaintiff seeks supplementation of the RAR with Document 5, an email thread dated June 20, 2007. The first communication in this thread was sent at 1:33 pm on June 20, from Marc Caplan, Chief of the “Operational Technologies Division,” to John Morgan, with carbon copies to several other persons, bearing the subject line “Pinnacle Letter Regarding Labeling Discrepancies and Removal of SOV2000.1/MIL3AF01 from NIJ Compliance List.” Mr. Caplan informed Mr. Morgan:
John,
I have left a red folder in your inbox with the draft letter to Pinnacle regarding the aforementioned subject. I am also providing an electronic copy of the letter for your review and comment. Once we have incorporated your comments, we will prepare the final letter for mailing. Due to the sensitivity of the subject, I am also copying OGC (John/Ingrid) to see if they have any last minute comments before the letter is mailed.
Debra, [REDACTED] and I met with Kim Lowry and Adam Spector yesterday to provide them with background information to assist them with preparing a press release to coincide with our removal of the Pinnacle Level III armor from our compliance list. Kim also suggested a letter to the “stakeholder” community.
OCOM is waiting for us to make a final decision regarding the removal of the Pinnacle Level III armor before they prepare the release. [REDACTED] Debra and I are planning to discuss this with you during our standing body armor meeting tomorrow at 10:00 am.
Document 5 at 1-2. At 1:52 pm, Mr. Morgan then forwarded this email on to others, including Adam Spector, asking “Adam, Can you get the specific information from the committee exhibits for the second paragraph of the letter?” Id. at 1. At 1:56 pm, Mr. Spector forwarded the email on to Kim Lowry asking: “Kim, Would [REDACTED] be the right contact for this information?” Id. Finally, at 3:51, Ms. Lowry responded to Mr. Spector: Wes.” Id.13
Plaintiff claims that Document 5 reveals that as of June 20, 2007, “NIJ ha[d] already reached the decision that it was going to evoke Pinnacle Armor’s body compliance even though the NIJ did not send the first letter to Pinnacle Armor requesting warranty related data until June 22, 2007.” Doc. 71 at 6. Thus, Pinnacle maintains this document is “necessary for the court’s review of ... possible bad faith for pre-determining the outcome before reviewing data.” Id.
For the bad faith exception to apply, “[n]ormally there must be a strong showing of bad faith or improper behavior before the court may inquire into the [1243]*1243thought processes of administrative decision makers.” Animal Def. Council v. Hodel, 840 F.2d 1432, 1437 (9th Cir.1988), amended, 867 F.2d 1244 (9th Cir.1989). Here, although Mr. Caplan discusses the preparation of a press release to “coincide with [NIJs] removal of the Pinnacle Level III armor from [the] compliance list,” he also indicates that others are “waiting for [NIJ] to make a final decision regarding the removal of the Pinnacle Level III armor before they prepare the release.” Applying the presumption that government agencies act properly and according to law, see FCC v. Schreiber, 381 U.S. 279, 296, 85 S.Ct. 1459, 14 L.Ed.2d 383 (1965) (describing presumption that government agencies “act properly and according to law.”), this email appears just to be preparing for an anticipated, but not final, removal of Dragon Skin from the compliance list. This does not amount to a “strong showing” of bad faith. Plaintiffs request to supplement the RAR with Document 5 is DENIED.
4. “Necessary to Explain Technical . Terms or Complex Matters.”
Finally, Plaintiff asserts that “each and every document offered by plaintiff’ is admissible under the third Southwest Center exception, permitting consideration of documents when necessary to explain .technical terms or complex matters. “However, a party may not circumvent the general rule, that judicial review is limited to the administrative record, by simply labeling a declaration as ‘assisting the court.’” Alsea Valley Alliance v. Evans, 143 F.Supp.2d 1214, 1216 (D.Or.2001). Each document must be considered carefully to “determine whether it truly assists the court in understanding technical or complex matters or merely attempts to argue the sufficiency of the record.” Id. “Those portions of the [document] that perform the latter function, naturally fall outside the Record and should be stricken.” Id.
The distinction between an affidavit that “explains” complex and technical material as compared to an affidavit that characterizes the record with new expert opinion is critical. The former is allowed where the -latter is not. This conclusion is formed by the guiding principle that “the function of the district court is to determine whether or not as a matter of law the evidence in the administrative record permitted the agency to make the decision it did.” Occidental Eng’g Co. v. INS, 753 F.2d 766, 769 (9th Cir.1985) (emphasis added). In other words, the administrative record stands or falls on its own. Where an “explanation” of complex material naturally assists the court in its understanding of the record as it stands, expert opinion regarding the sufficiency of the record takes on independent evidentiary value of its own and naturally falls outside the record because the expert opinion was not before the agency prior to its final determination.
Id. at 1217.
Plaintiffs blanket assertion that all of the Documents offered are admissible under this exception is wholly insufficient. Plaintiffs insistence that Defendant has failed to explain why these documents “would not be helpful” entirely misses the point that, as with all of the exceptions, it is Plaintiffs burden to establish that the extra-record Documents qualify for consideration.
Plaintiff focuses on the admissibility of Documents 24, 25, and 26 under this exception.14 Document 24 is the two-page [1244]*1244Declaration of Prof. Edward Rosenberg a member of the Chemistry Department at the University of Montana. Professor Rosenberg reviewed the August 15, 2007 pre-decisional report, RAR 442-43, and presents three reasons why he believes “the conclusions reached [therein] based on the stated data are completely specious.” Document 24. The Court’s review of this Declaration reveals that it contains only impermissible characterizations of the record, without any information that explains the complex and technical material in the record in a manner that would aid the Court’s understanding.
Document 25 is the four-page Declaration of Nevin Rupert, a ballistics expert with particular familiarity with Dragon Skin technology. He reviewed the August 17, 2007 Review of Dragon Skin Body Armor Data Package (Second Submission), with particular attention to the section entitled “Critical Review of Shot Locations of Non-homogeneity Considerations,” RAR 437-39, as well as the August 31, 2007 Review of Dragon Skin Body Armor Data Package (Third Submission), with specific attention to the section entitled “Suitability of Test Methods Used to Address Concerns Raised by NIJ,” RAR 497-98. In paragraph 4 of his declaration, Mr. Rupert asserts that these sections of the RAR “demonstrate a lack of historical perspective in their evaluation of ceramic armors [like Dragon Skin].” Rather than discussing and/or explaining the existing record, he discusses test results from the 1940s and 50s, apparently as examples of information that is superior to that considered in the record. This is inadmissible expert opinion regarding the sufficiency of the record. Paragraph 5 discusses historic studies of ceramic armor systems, but does not explain anything in the existing record. Paragraph 6 discusses testing of “Ball Armor” system and suggests that the results of that testing indicate that “the specification of hit points relative to the ceramic geometric construction is unnecessary when V50 results are to be used as the bases of comparison,” while “Geometric construction consideration may become important in V0 testing due to the larger zone-of-mixed results as indicated by the V50 standard deviation.” This certainly fails to explain any complex or technical information, as it is totally incomprehensible to a lay audience. Nothing in the remainder of this Declaration fares any better. It contains only impermissible characterizations of the record, without any information that explains the complex and technical material in the record in a manner that would aid the Court’s understanding.
Document 26 is a lengthy Declaration submitted by Murray Neal, the Founder, President, and CEO of Pinnacle Armor. The first 22 paragraphs of this Declaration consist of a list of the Documents for which supplementation is requested in this motion. Then, Mr. Murray begins review of the several “pre-decisional” analyses contained in the RAR. He takes on NIJ’s statements in these analyses paragraph-by-paragraph, sometimes indicating that the statement is “incorrect,” “speculative” or otherwise unfounded and/or unsupported. The Declaration is filled with technical material and jargon, with little or no effort to explain that material to the lay audience (i.e. the Court). The thrust of the Declaration is therefore an attempt to characterize the record, rather than explain complex or technical material. A few paragraphs of the Declaration do appear to attempt to explain basic concepts relevant to the subject matters at issue in this [1245]*1245case, e.g. Document 26 ¶¶ 60-62, 64, but Plaintiff has totally failed to explain how these explanations are “necessary to explain technical terms or complex matters” already in the record.
Plaintiff is reminded that application of this exception requires, as a threshold matter, a showing that the existing record is so inadequate as to frustrate judicial review. Bair v. California State Dept. of Transp., 867 F.Supp.2d 1058, 1067 (N.D.Cal.2012) (citing Animal Defense Council v. Hodel, 840 F.2d 1432, 1437 (9th Cir.1988)). Plaintiff has not even attempted to explain why the existing record is inadequate to permit judicial review of NIJ’s actions, let alone how the Documents offered would cure any inadequacy. Plaintiff has failed to meet its burden to furnish these explanations.
Plaintiffs request to supplement the record with all 22 disputed Documents under the “necessary to explain technical terms or complex matters” is DENIED.
III. CONCLUSION
For the reasons set forth above, Plaintiffs motion to supplement the RAR is DENIED with the exceptions that the RAR shall be supplemented with:
• Document 8, pursuant to the consent of Defendant;
• Document 10, a differently redacted version of a document already in the record; and
• Document 21 under the relevant factors exception.
In addition, the Court will permit Plaintiff to file a very narrow motion for discovery to uncover whether NIJ considered additional data from the Army or DOD. Any such motion for discovery, which shall not exceed five pages in length, shall be filed on or before March 1, 2013, with any response, subject to the same page limit, due on or before March 8, 2013. The motion will be decided on the papers without oral argument. The Parties are admonished to attempt to stipulate as to the nature of any discovery and/or to supplementation of the RAR with appropriate documents. Any such stipulation must be filed on or before March 1, 2013 in lieu of a motion for discovery.
Once any such discovery is completed or the matter is resolved by stipulation, the Parties are instructed to attempt to stipulate to a schedule for resolution of this case on the merits pursuant to the supplemented RAR. Normally, APA cases are resolved on cross-motions for summary judgment, frequently with the defendant filing an opening brief, the plaintiff filing a single brief encompassing its opposition and cross-motion, followed by a reply from the defendant, and concluding with a reply from the plaintiff. If the Parties believe it would be more efficient for Defendant to withdraw its pending motion and re-brief the merits issues according to such a coordinated schedule, they are encouraged to do so. If the Parties cannot agree to a schedule, they shall contact the magistrate judge to set up a scheduling conference. IT IS SO ORDERED.