Garner v. Amazon.com Inc

• Court: District Court, W.D. Washington
• Decided: May 19, 2023
• Docket: 2:21-cv-00750
• Status: Unknown

Opinion

5 UNITED STATES DISTRICT COURT 6 FOR THE WESTERN DISTRICT OF WASHINGTON AT SEATTLE 7

8 KAELI GARNER, et al., CASE NO. 2:21-cv-00750-RSL 9 Plaintiffs, v. 10 ORDER REGARDING PLAINTIFFS’ MOTION TO PROHIBIT USE OF 11 AMAZON.COM, INC., et al., TECHNOLOGY-ASSISTED REVIEW

PROCEDURES 12 Defendant.

14 This matter comes before the Court on “Plaintiffs’ Motion to Compel Defendants to 15 Comply with the Court’s Prior Discovery Orders.” Dkt. # 143. In October 2022, the Court 16 granted plaintiffs’ motion to compel searches of the electronically-stored information 17 18 (“ESI”) of 36 Amazon employees using 38 search strings, as proposed by plaintiffs. Dkt. 19 # 136. Using the approved search terms, defendants identified approximately 2 million 20 potentially responsive documents.1 Within two weeks of the Court’s order, defendants 21 notified plaintiffs that Amazon intended to “use technology-assisted review [“TAR”] tools 22 23 to prepare its production,” promising that they would “discuss these tools with Plaintiffs 24 25

26 1 In the context of the motion to compel, defendants had estimated that the search terms would return between 3.1 and 4.4 million documents. ORDER REGARDING PLAINTIFFS’ MOTION TO 1 before beginning review.” Dkt. # 144-4 at 2.2 Two days later, plaintiffs objected, asserting 2 that “[a]pplying TAR at this stage – after the parties have agreed to search terms as a basis 3 for culling documents, and after the Court issued an order on search terms and custodians, 4 5 is improper and impermissible.” Dkt. # 144-2 at 3. Plaintiffs argued that it was too late to 6 alter the protocol for the production of ESI in this case.3 Defendants disagreed, insisting 7 that the large number of retrieved documents necessitated the use of TAR to identify 8 responsive, non-privileged documents. Dkt. # 144-1. Plaintiff again objected to any effort 9 10 “to use TAR to further review/filter the documents retrieved by the Court-ordered search 11 terms.” Dkt. # 144-3 at 3. 12 Having reviewed the memoranda, declarations, and exhibits submitted by the 13 parties and having heard the arguments of counsel, the Court finds as follows: 14 15 Plaintiffs make clear in reply that they are not trying to prevent Amazon from 16 conducting a responsiveness and privilege review of the universe of documents the search 17 terms retrieved. Rather, they seek to prevent the unilateral application of TAR, apparently 18 preferring that Amazon conduct its review by hand. In this district, it is presumed that the 19 use of technology-assisted review is a reasonable option for locating or filtering ESI. 20 21 https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.wawd.uscourt 22 23

24 2 TAR is also known as computer-assisted review or predictive coding. Reviewers typically code a set of documents as responsive or unresponsive to “train” the TAR software, which then uses the coded documents to generate algorithms for reviewing all other documents in the selected universe for responsiveness. Progressive Cas. Ins. Co. v. 25 Delaney, No. 2:11-cv-00678-LRH, 2014 WL 3563467, at *10 (D. Nev. July 18, 2014); https://legal.thomsonreuters.com/en/insights/articles/myths-and-facts-about-technology-assisted-review. 26 3 At approximately the same time, the parties negotiated an extension of the discovery deadline to August 16, 2023. ORDER REGARDING PLAINTIFFS’ MOTION TO 1 s.gov%2Fsites%2Fwawd%2Ffiles%2FModelESIAgreement_CLEAN_2.1.23.docx&wdOri 2 gin=BROWSELINK at 3. Litigants are instructed to confer and attempt to reach agreement 3 on “appropriate computer- or technology-aided methodologies[] before any such effort is 4 5 undertaken.” Id. at 3-4. That requirement is incorporated into the April 2022 ESI 6 Agreement and Order entered in this matter, which also recognized “that searching for 7 relevant ESI is an iterative process.” Dkt. # 89 at 2-4. Case law discussing the use of 8 computer-assisted methodologies to review documents for production emphasizes the need 9 10 for cooperation and transparency. In Youngevity Int’l, Corp. v. Smith, No. No. 16-cv- 11 00704-BTM (JLB), 2019 WL 1542300, at *12 (S.D. Cal. Apr. 9, 2019), for example, the 12 court noted that: 13 Technology-assisted review of ESI does require an ‘unprecedented degree of 14 transparency and cooperation among counsel’ in the review and production 15 of ESI responsive to discovery requests. Progressive Cas. Ins. Co., 2014 WL 3563467, at *10. In this regard, courts typically ‘have required the producing 16 party to provide the requesting party with full disclosure about the 17 technology used, the process, and the methodology, including the documents 18 used to ‘train’ the computer.” Id.

19 While defendants’ discovery conduct has not been beyond criticism, the nub of this 20 particular discovery dispute is that when Amazon raised the issue of using TAR, plaintiffs 21 refused to discuss it. This was not a valid option given the circumstances. The parties had 22 23 already agreed that an extension of the discovery deadline was needed: there was therefore 24 time to exchange proposals regarding the use of predictive coding to improve the accuracy, 25 speed, and efficiency of the document review Amazon was then undertaking. Courts 26 ORDER REGARDING PLAINTIFFS’ MOTION TO 1 generally agree that the producing party is best situated to evaluate the various options for 2 reviewing and producing its own ESI and places the burden on that party to make an 3 initial, detailed proposal about the technology and methodologies it intends to use. Id. at 4 5 11; Hyles v. New York City, No. 10-cv-3119(AT)(AJP), 2016 WL 4077114, at *2 6 (S.D.N.Y. Aug. 1, 2016); Progressive Cas. Ins. Co., 2014 WL 3563467, at *10. The 7 requesting party then has an opportunity to consult with its own ESI discovery experts so 8 that it can intelligently respond and the parties can move toward an agreed protocol for the 9 10 use of TAR. By refusing to discuss the use of TAR at all, plaintiffs improperly short- 11 circuited this process.4 12 Plaintiffs suggest that the use of TAR on a pool of documents already retrieved 13 through search terms is somehow improper because it “will only reduce the document pool 14 15 further and will certainly not reveal documents that the application of search terms has 16 precluded.” Dkt. # 143 at 10 (quoting In re Allergan Biocell Textured Breast Implant Prod. 17 Liab. Litig., No. 2:19-md-2921(BRM)(ESK), 2022 WL 16630821, at *3 (D.N.J. Oct. 25, 18 2022)). But, as discussed above, the Model ESI Agreement in this district clearly 19 contemplates using TAR to filter, not just locate, documents, and the ESI Agreement 20 21 4 The Court acknowledges the similarities between the facts of this case and those of Progressive Cas. Ins. Co. v. 22 Delaney. In both cases, search terms had been used to identify potentially responsive ESI before the producing party indicated that TAR should be used to further refine the production. In contrast with this case, however, the stipulated 23 ESI protocol in Progressive gave the producing party two options once the universe of potentially responsive documents was identified: to produce all of the “hit” documents or to manually review the retrieved documents and 24 produce only those that were, in fact, responsive. 2014 WL 3563467, at *11. Progressive chose a third, unauthorized, path by conducting a TAR review. It also withheld documents that had already been found to be responsive through 25 an aborted manual review process and it refused to provide details regarding its TAR proposal to the requesting party. In those circumstances, the court declined to adopt Progressive’s predictive coding proposal because it would only 26 result in more discovery disputes and delay. The court instead ordered the production of all non-privileged documents retrieved using the search terms. ORDER REGARDING PLAINTIFFS’ MOTION TO 1 entered in April 2022 simply directs the parties “to confer to attempt to reach agreement on 2 . . . appropriate computer- or technology-aided methodologies[] before any such effort is 3 undertaken.” The Court finds that the use of search terms is not, standing alone, a bar to 4 5 using technology to further refine the production. 6 It is worth repeating at this juncture that plaintiffs do not dispute that Amazon has 7 the right to conduct a responsiveness and privilege review of the documents retrieved 8 through application of the search terms to the custodial records, they just want the review 9 10 done by hand rather than through predictive coding because (a) November 2022 was “too 11 late” to raise the issue and (b) the TAR process has identified only 2,564 responsive 12 documents out of an initial universe of 2,036,172. For the reasons discussed above, the 13 Court finds that raising the possibly of using TAR once the search terms were run and the 14 15 universe of hits was known was not unreasonable and did not excuse plaintiffs from 16 engaging in a discussion regarding the methodology and protocols to be used. Plaintiffs’ 17 timeliness objection is overruled. 18 Plaintiffs’ concern regarding the incredibly low production rate (less than 0.13% of 19 the documents returned using the search terms) may be valid, but it does not appear to be 20 21 related to Amazon’s use of TAR. Amazon’s initial proposal regarding how their TAR 22 would work utilized an unacceptably low recall target of 75%.5 But as the process 23 24

25 5 Mr. Johnson states that recall requirements between 70-80% are standard because “[a]ttempting to achieve higher recall rates, for example 85%, can result in dramatically higher costs for the review, and likely eliminate any potential 26 costs savings and efficiencies that parties would hope to realize using TAR.” Dkt. # 149 at ¶ 17. The goal of discovery is the production of all responsive, relevant, non-privileged documents. If TAR is able to assist in reaching that goal ORDER REGARDING PLAINTIFFS’ MOTION TO 1 unfolded, Amazon continued its review after that rate was achieved, ultimately conducting 2 a human review of over 1.8 million documents. When a sample of the remaining pool of 3 unreviewed documents was analyzed, the human reviewers found no responsive 4 5 documents. Thus, humans reviewed over 89% of the documents identified using the 6 agreed-upon search terms and found that only 2,564 of them were actually relevant, 7 responsive, and non-privileged. The 224,924 unreviewed documents are, according to the 8 TAR methodology, the least likely to be responsive and, in fact, a human review of 1,527 9 10 of them revealed no responsive materials. Because humans reviewed the vast majority of 11 the universe of documents and the statistical estimate of responsive documents remaining 12 in the unreviewed documents is 0%, the estimated recall rate approaches 100% There is no 13 reason to suspect that the low percentage of production is, as plaintiffs argue, the result of 14 15 Amazon’s use of TAR versus human review. 16 17 For all of the foregoing reasons, plaintiffs’ motion to compel is DENIED. 18 Defendants shall, if they have not already done so, immediately produce the 2,564 19 documents tagged as responsive through the human review process. 20 21 Dated this 19th day of May, 2023. 22

23 Robert S. Lasnik 24 United States District Judge 25

26 while saving money, it is a useful tool. But if TAR is cost-effective only if 25% of responsive, relevant, non- privileged documents remain hidden, it may costs less, but it is no longer effective. ORDER REGARDING PLAINTIFFS’ MOTION TO