IN THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF KANSAS
LARRY A. LAWSON, ) ) Plaintiff, ) ) v. ) Case No. 18-1100-EFM-ADM ) SPIRIT AEROSYSTEMS, INC., ) ) Defendant. )
MEMORANDUM AND ORDER
This matter comes before the court on plaintiff Larry A. Lawson’s (“Lawson”) Motion to Compel the Production of Responsive Documents. (ECF No. 226.) In April 2019, the court first directed Lawson and defendant Spirit AeroSystems, Inc. (“Spirit”) to work together on an electronically stored information (“ESI”) search protocol. After the parties spent months trying to engage in what proved to be an unworkable, costly, and largely fruitless effort to search Spirit’s ESI, in September 2019 Spirit agreed to conduct a technology-assisted review (“TAR”) using Lawson’s proposed search terms and custodians. Spirit ceased producing responsive documents through the TAR process after reaching an 85% recall rate, meaning that the TAR algorithm had correctly identified 85% of the responsive documents in the data set. At that point, approximately 1,850 potentially responsive documents remained in the TAR set (the “residual TAR documents”). Spirit’s first-level review team had identified these documents as potentially responsive but, because the TAR process reached an 85% recall rate, they were not produced.1 Lawson now moves to compel Spirit to produce these residual TAR documents.
1 Spirit clarified in its response brief that only 800 documents were identified as responsive, and the remainder are associated family-member documents that are non-responsive. (ECF No. 240, at 5.) Lawson is no longer seeking these non-responsive documents. (See ECF No. 247, at 2 For the reasons discussed below, the court is persuaded that Spirit conducted a reasonable and diligent search for documents responsive to Lawson’s ESI demands. The court is satisfied that Lawson has had a full and fair opportunity to obtain ESI that is relevant and proportional to the needs of the case via the TAR process, as well as through other targeted document productions outside of that process. The court will therefore deny Lawson’s motion to compel Spirit to produce
the residual TAR documents. I. BACKGROUND The background of this lawsuit is more thoroughly set forth in this court’s prior orders, familiarity with which is presumed. Highly summarized, Lawson is Spirit’s former chief executive officer who retired on July 31, 2016. His Retirement Agreement contained non-compete obligations for two years, until July 31, 2018. In early 2017, Lawson engaged in business dealings with non-party investment firms Elliott Associates, L.P. and Elliott International, L.P. (collectively, “Elliott”) to provide consulting services in connection with a proxy contest Elliott launched to replace five board members of Arconic, Inc. (“Arconic”). When Spirit learned about
this, Spirit notified Lawson that his involvement with Arconic constituted a breach of his non- compete. Spirit stopped paying Lawson and demanded that he repay what the company had already paid him under the Retirement Agreement. Lawson disputes that he breached the non-compete. He filed this lawsuit seeking to recover what he believes Spirit owes him. A. The Parties’ Initial Discussions Regarding Spirit’s ESI This case was reassigned to the undersigned on March 26, 2019. By that time, Lawson had already filed a motion to compel Spirit to produce ESI directed to the issue of whether Spirit and
(stating that Lawson would agree to Spirit producing slipsheets for these non-responsive documents).) Arconic are in the same “Business,” as that term is defined in the non-compete in Lawson’s Retirement Agreement. (See ECF No. 57, at 23-24.) Lawson’s motion to compel was based on its Requests for Production (“RFPs”) seeking documents related to Spirit’s relationship with Arconic and the overlap between their businesses. (See id. at 12.) Those RFPs largely sought “all” such documents and communications. (ECF No. 58-2, at 9-14, RFP Nos. 19, 25-30, 34-38,
40.) Likewise, Lawson’s motion to compel sought “all” such documents. (ECF No. 57, at 5, 21- 24.) The parties could not agree on ESI custodians or search terms, and had difficulty meeting and conferring productively. (Id. at 7-8, 11, 13-15; ECF No. 72, at 3-4.) So Lawson filed a motion to compel Spirit to produce documents responsive to Lawson’s list of search terms and custodians. (ECF No. 57, at 24-30.) Spirit responded, arguing that Lawson’s ESI demands were “nothing short of a fishing expedition,” disproportionate to the needs of the case, and “abusive,” and that Lawson was “using discovery for the sake of creating obvious burden.” (ECF No. 72, at 2-3.) Lawson’s motion asked the court to compel Spirit to search nearly 70 custodians’ ESI using about 90 search terms.2 (Id.
at 9.) Spirit argued that Lawson had made “no effort . . . to identify and remove duplicative or tangential custodians.” (Id. at 24.) Spirt also contended that Lawson’s proposed search terms were “equally overbroad and unworkable.” (Id. at 25.) Many of those terms were common terms in the aviation industry, such as “fuselage,” “bulkhead,” and “pylon,” without any limiting modifiers. (See id.; ECF No. 61-1, at 3-4.) After receiving Lawson’s list of proposed custodians and search terms, Spirit identified eight individuals it believed would be appropriate custodians (seven of whom were included in
2 Lawson initially demanded that Spirit search each custodian’s assistant’s ESI also, but his motion did not seek to compel these searches. (ECF No. 72, at 9.) Lawson’s list). (ECF No. 72, at 25; ECF No. 72-10.) Spirit ran searches on four of those custodians’ ESI using Lawson’s proposed search terms. (ECF No. 72-13 ¶ 11, at 2-3.) After running the searches, Spirit informed Lawson that they returned more than 320,000 documents. (ECF No. 72, at 26.) Spirit reviewed approximately 400 of these documents and determined that 85% were irrelevant. (ECF No. 72-13 ¶ 13, at 3.) Because Spirit viewed Lawson’s proposed
search terms as ineffective, Spirit stated that it would craft its own search terms. (ECF No. 58-13, at 4.) Spirit also proposed limiting ESI searches to ten custodians it believed most likely to have information relevant to the parties’ dispute. (See id. at 5, 7.) B. The Initial ESI Protocol On April 23, 2019, the court convened a hearing on Lawson’s motion to compel. (ECF Nos. 76, 81.) After consultation with the parties, the court granted Lawson’s motion in part and denied it in part. In relevant part, the court ordered Spirit to produce documents responsive to the Arconic-related requests to the extent that they would be captured by the ESI protocol set forth in the court’s Memorandum and Order. See Lawson v. Spirit AeroSystems, Inc., No. 18-1100-EFM-
ADM, 2019 WL 1877159, at *2 (D. Kan. Apr. 26, 2019). That ESI protocol directed the parties to proceed as follows: Lawson would first identify up to seven categories of documents for which he was seeking ESI; For each category, Spirit would then serve a list of the top three custodians most likely to have relevant ESI, from the most likely to the least likely, along with a brief explanation as to why Spirit believed the custodian would have relevant information; Lawson would then serve an initial list of five custodians with proposed search terms for each, and a second list of five custodians with proposed search terms for each a week later; and Spirit would conduct searches of the custodians’ ESI using Lawson’s proposed search terms, engage in a sampling exercise to determine responsiveness rates, and suggest modified search terms if the terms proposed by Lawson “produced an unreasonably large number of non-responsive or irrelevant results.” Id. at *2-*3. Ultimately, the court envisioned an iterative process in which the parties would work together to formulate more effective search terms. The court ordered the parties to “meet and confer about search terms and try to achieve an estimated responsive hit rate of at least 85%.” Id. at *3. The parties initially proceeded according to this protocol. Lawson identified seven categories of documents for which he sought ESI and, for each category, Spirit identified the top three custodians. (See ECF No. 136-4.) But when Lawson chose ten custodians for Spirit to search, he chose only three of the custodians that Spirit had identified as most likely to have relevant information. Lawson provided Spirit with 803 search terms (counting terms with “OR”
as multiples) and asked Spirit to run those terms on all ten custodians’ ESI. (ECF No. 136-5; ECF No. 136-6; ECF No. 135 ¶ 7, at 2.) Spirit searched the first five custodians’ ESI with Lawson’s 803 terms, which returned nearly 196,000 documents. (ECF No. 152-1, at 27.) After reviewing a 384-document sample, Spirit determined that only 16.7% were responsive to Lawson’s discovery requests. (Id.) Spirit provided this information to Lawson, along with hit reports for Lawson’s search terms. (See id. at 29-42.) In an effort to achieve the 85% hit rate goal set by the court, Spirit also proposed its own set of search terms with corresponding hit reports for the first five custodians. (See id. at 43-56.) Spirit also searched the second five custodians’ ESI with Lawson’s 803 terms. (See ECF
No. 136-7.) In total, the searches across all ten custodians returned approximately 304,000 documents. (Id.; ECF No. 135 ¶¶ 7-8, at 2.) Spirit reviewed a 384-document sample of these results and determined that only 7.8% were responsive. (ECF No. 136-7; ECF No. 135 ¶ 9, at 2.) C. The Modified ESI Protocol On June 6, 2019, the court convened a discovery conference at the parties’ request to discuss their progress on ESI. (ECF Nos. 84, 87.) After consultation with the parties, the court limited Lawson to 25 search terms for each individual custodian. (ECF No. 88, at 1.) The court again told the parties to work together “in a back-and-forth, iterative process to agree on search
terms that will achieve an estimated responsive hit rate of at least 85%.” (Id.) On June 28, Lawson sent Spirit revised proposed search terms in accordance with the modified ESI protocol. (See ECF No. 136-8.) Many of Lawson’s revised terms were again common aviation-related terms, such as “nacelle” or “nut,” as well as verbs commonly used in many industries, such as “fasten*” and “procure*.” (See id. at 4.) Spirit conducted new searches of the ten custodians’ ESI with Lawson’s revised terms in July 2019, which returned approximately 322,000 documents. (ECF No. 135 ¶ 12, at 3.) After conducting a sampling exercise, Spirit told Lawson the responsiveness rates for each custodian. (See ECF No. 136-9.) Those rates ranged from 0.5% to 13.5%, with an average across all custodians of 5.1%. (Id.; ECF No. 135 ¶ 13, at 3.)
The parties met and conferred about search terms on August 9. (See ECF No. 147-2, at 133.) At that time, Spirit stated that it believed discussing individual search terms and custodians would not be productive. (Id.) Spirit agreed to produce to Lawson responsive documents from its sampling exercises, as well as some non-responsive documents, to assist in the parties’ efforts to refine search terms. (Id.) Lawson agreed to provide additional information on what documents Lawson was expecting to see from particular custodians. (Id.) D. Technology Assisted Review By September 2019, the parties had abandoned all efforts to work together on refining search terms to meet the 85% hit-rate goal. They instead discussed conducting a TAR of the 322,000-document set identified in July 2019. (See ECF No. 227-1, at 5.) Spirit’s ESI vendor Legility offers a TAR tool called “Predict.” On September 12, Spirit provided Lawson with information on this tool. (See 147-2 ¶ 15, at 3.) After an initial set of documents is coded for responsiveness, the Predict tool—which utilizes “continuous active learning”—uses those standards to code additional documents. (See id. at 155.) Predict ranks coded documents from the
most likely responsive to the least; top-ranked documents are then reviewed by humans. (See id.) When Predict determines that the pool of responsive documents is depleted such that the effort of continued review is disproportionately outweighed by the possibility of additional gain, review ceases. (See id. at 156.) Legility then conducts a statistical validation of the TAR’s results. (See id.) The parties met and conferred regarding TAR logistics on September 26. (ECF No. 227- 1, at 5.) They agreed that the initial review of TAR documents would be conducted by Legility’s contract attorneys, and a second-level review would be conducted by attorneys at Arcadi Jackson, LLP, Spirit’s law firm. (Id.) The parties did not at that time agree to a target recall rate, i.e., the
percentage of responsive documents out of the entire set they aimed to locate and produce through the TAR process. On November 8, the court convened a discovery conference to discuss the current status of discovery, including the completion of document production. (ECF No. 169.) Spirit stated that it was producing documents identified through TAR on a rolling basis and planned to end the TAR process after achieving a 65% recall rate. (Id. at 2.) Because the parties had not previously discussed their respective expectations regarding a recall rate, the court ordered them to meet and confer in an attempt to reach an agreement. (Id.) The parties conferred through November and December 2019. Spirit believed that a 65% recall rate was proportional to the needs of the case, taking into the account the number of responsive documents located and the cost and time of review. (ECF No. 227-1, at 15.) Lawson, however, was concerned that 65% would be inadequate, and that a 75-85% recall rate was more typical. (Id.) Spirit explained to Lawson that the expected additional cost to reach an 85% recall rate could be up to an additional $35,000, on top of the thousands of dollars already spent on TAR and non-TAR productions. (Id. at 11.) Ultimately,
however, Spirit agreed to expand its review to reach an 80% recall rate in an effort to avoid further motion practice, and in light of the depositions that were, at that time, set to begin in January 2020.3 (Id.) E. Lawson’s Motion On January 10, 2020, the court convened a discovery conference during which the parties explained a dispute they were having over the residual TAR documents. (Id. at 37; ECF No. 128.) Spirit stated that it would be producing the additional TAR documents it had agreed to review to reach the 80% recall rate within the next few days. (ECF No. 227-1, at 67.) Lawson stated that the residual TAR documents had already been identified as responsive and should be produced.
(See id. at 58-60.) Spirit explained that these documents had been through the first-level of review, but Spirit believed that engaging in the second level of review and preparing the residual TAR documents for production was not proportional to the needs of the case. (Id. at 63-64.) Spirit cited the relatively small number of responsive documents found in the 322,000-document set and described the costs incurred in producing those documents. (See id. at 62-65.) On top of the approximately $380,000 Spirit had already spent in costs relating to the TAR process, engaging in
3 The court later allowed the parties to postpone the timeline for conducting depositions. (ECF No. 221, at 1.) a second-level review of the residual TAR documents and producing them would cost an estimated $40,000. (Id. at 62-63.) With the hope of efficiently resolving this dispute, the court asked if Lawson would be willing to bear the costs of the second-level review and production of the residual TAR documents. (Id. at 65.) The Agreed Order Establishing Protocol for ESI and Paper Documents in this case,
which the parties agreed to and submitted to the court, provides that the identification and production of responsive ESI is “subject to . . . the development of reasonable and appropriate cost allocation agreements.” (ECF No. 40, at 8.) Lawson immediately refused to bear these costs. (ECF No. 227-1, at 65.) After considering the parties’ arguments, the court explained that it would not be inclined to order Spirit to proceed with a second-level review at that time, but, because the parties remained at an impasse, the court granted Lawson leave to file the instant motion to compel. (ECF No. 227-1, at 66; ECF No. 221, at 2-3.) Lawson’s motion now argues that the residual TAR documents are “critical to [his] ability to rebut Spirit’s unsupported assertion that it and Arconic are in the same Business” and their
production is proportional to the needs of the case. (ECF No. 227, at 5-6, 8.) Lawson contends that Spirit is obligated to produce documents it has identified as responsive, even though Spirit’s production already reached its target recall rate; allowing otherwise “would open the door to discovery gamesmanship.” (Id. at 6-7.) Lawson further argues that a second-level review is unnecessary, and Spirit’s claims regarding costs are inflated. (Id. at 6-8.) Spirit argues that second-level review and production of the residual TAR documents is not proportional to the needs of the case. Spirit states that it has already produced (or withheld as privileged) 85% of the responsive documents in the TAR set, which has cost approximately $400,000 in fees and costs to Legility and $200,000 in fees incurred by Spirit’s law firms. (ECF No. 240, at 3 n.7, 4.) Spirit contends that the residual TAR documents are unlikely to be important in resolving the issues in this case, as there are only about 800 that have been preliminarily marked responsive, along with approximately 1000 associated documents (i.e., family members that are non-responsive and irrelevant. (Id. at 4-5.) Spirit estimates that review and production of all of these documents will cost over $30,000.4 (Id. at 4.) Spirit also argues that its cost estimates are
not inflated, and a second-level review by outside counsel is necessary to confirm responsiveness and to review documents for confidentiality, privilege, and compliance with International Traffic in Arms Regulations. (Id. at 6.) II. LEGAL STANDARD “Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party’s claim or defense and proportional to the needs of the case.” FED. R. CIV. P. 26(b)(1). In other words, considerations of both relevance and proportionality now expressly govern the scope of discovery. FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment. When evaluating proportionality, the court considers “the importance of the issues at stake in the action,
the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit.” FED. R. CIV. P. 26(b)(1). The party resisting discovery on proportionality grounds still bears the burden to support its objections. FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment (“Restoring the proportionality calculation to Rule 26(b)(1) does not change the existing responsibilities of the court and the parties . . . .”); Nat’l R.R. Passenger Corp. v. Cimarron Crossing Feeders, LLC, No.
4 As noted above, Lawson has since clarified that he does not seek production of the non- responsive, irrelevant family members. (See ECF No. 247, at 2.) 16-CV-1094-JTM-TJJ, 2017 WL 4770702, at *4 (D. Kan. Oct. 19, 2017) (“The party resisting discovery bears the burden to support its objections based upon proportionality[.]”). The practical effect of the rule is that both parties must typically provide information pertinent to the proportionality analysis. See In re Bard IVC Filters Prod. Liab. Litig., 317 F.R.D. 562, 565 (D. Ariz. 2016); FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015
amendment (“The parties and the court have a collective responsibility to consider the proportionality of all discovery and consider it in resolving discovery disputes.”). This is because [a] party claiming undue burden or expense ordinarily has far better information — perhaps the only information — with respect to that part of the determination. A party claiming that a request is important to resolve the issues should be able to explain the ways in which the underlying information bears on the issues as that party understands them. The court’s responsibility, using all the information provided by the parties, is to consider these and all the other factors in reaching a case-specific determination of the appropriate scope of discovery. FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment. “No single factor is designed to outweigh the other factors in determining whether the discovery sought is proportional, and all proportionality determinations must be made on a case-by-case basis.” Oxbow Carbon & Minerals LLC v. Union Pac. R.R. Co., 322 F.R.D. 1, 6 (D.D.C. 2017). The court retains an independent ongoing obligation to assess proportionality. See FED. R. CIV. P. 26(b)(2)(C) (providing that, on a motion or on its own, the court must limit the frequency and extent of disproportional discovery); see also FED. R. CIV. P. 26 advisory committee’s notes to the 2015 amendment (stating the court has a “responsibility to consider the proportionality of all discovery and consider it in resolving discovery disputes”). III. ANALYSIS It is well settled that TAR “is an acceptable way to search for relevant ESI in appropriate cases.” Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 183 (S.D.N.Y. 2012); see also, e.g., Youngevity Int’l, Corp. v. Smith, No. 16CV00704BTMJLB, 2019 WL 1542300, at *11 (S.D. Cal. Apr. 9, 2019) (“Predictive coding or TAR has emerged as a far more accurate means of producing responsive ESI in discovery than manual human review of keyword searches.”); Entrata, Inc. v. Yardi Sys., Inc., No. 2:15-CV-00102, 2018 WL 5470454, at *7 (D. Utah Oct. 29, 2018) (stating that “it is ‘black letter law’ that courts will permit a producing party to utilize TAR”); Aurora
Coop. Elevator Co. v. Aventine Renewable Energy-Aurora W., LLC, No. 4:12CV230, 2015 WL 10550240, at *2 (D. Neb. Jan. 6, 2015) (encouraging the parties to “work cooperatively . . . in developing and implement[ing] computer-assisted review”). TAR is defined as “[a] process for prioritizing or coding a collection of [ESI] using a computerized system that harnesses human judgments of subject matter expert(s) on a smaller set of documents and then extrapolates those judgments to the remaining documents in the collection.” The Sedona Conference Glossary: E- Discovery & Digital Information Management (Fourth Edition), 15 SEDONA CONF. J. 305, 357 (2014); see also Maura R. Grossman & Gordon V. Cormack, The Grossman-Cormack Glossary of Technology-Assisted Review, 7 FED. CTS. L. REV. 1, 32 (2013). In other words, human reviewers
“code a ‘seed set’ of documents. The computer [then] identifies properties of those documents that it uses to code other documents.” Da Silva Moore, 287 F.R.D. at 184. With TAR processes like the Predict tool used in this case, which use continuous active learning (sometimes referred to as “TAR 2.0”), the software continuously analyzes the entire document collection and ranks the population based on relevancy. Human coding decisions are submitted to the software, the software re-ranks the documents, and then presents back to the human additional documents for review that it predicts as most likely relevant. This process continues until the TAR team determines that the predictive model is reasonably accurate in identifying relevant and nonrelevant documents, and that the team has identified a reasonable number of relevant documents for production. BOLCH JUDICIAL INST. & DUKE LAW, TECHNOLOGY ASSISTED REVIEW (TAR) GUIDELINES 4 (Jan. 2019) [hereinafter TAR GUIDELINES], available at https://judicialstudies.duke.edu/wp- content/uploads/2019/02/TAR-Guidelines-Final-1.pdf. Measures to determine whether TAR is effective include “recall” and “precision.” “Recall is the fraction of relevant documents identified during a review; precision is the fraction of
identified documents that are relevant. Thus, recall is a measure of completeness, while precision is a measure of accuracy or correctness.” Da Silva Moore, 287 F.R.D. at 189-90. When a recall rate increases, a search is generally less precise. See The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 15 SEDONA CONF. J. 217, 238 (2014). “Effectively, one can cast either a narrow net and retrieve fewer relevant documents, along with fewer irrelevant documents, or cast a broader net and retrieve more relevant documents, but at the expense of retrieving more irrelevant documents.” Id. The proportionality standard in Rule 26(b)(1) applies to ESI discovery. Indeed, in cases involving a considerable amount of ESI, “proportionality considerations [are] particularly
significant.” In re EpiPen (Epinephrine Injection, USP) Mktg., Sales Practices & Antitrust Litig., No. 17-MD-2785-DDC-TJJ, 2018 WL 1440923, at *1 (D. Kan. Mar. 15, 2018) (citing The Sedona Principles, Third Edition: Best Practices, Recommendations & Principles for Addressing Electronic Document Production, 19 SEDONA CONF. J. 1, Principle 2, 51 (2018)). The goal of TAR is “to result in higher recall and higher precision than another review method, at a cost proportionate to the ‘value’ of the case.” Da Silva Moore, 287 F.R.D. at 190. The court has not found, and Lawson tellingly has not identified, any instance in which a court has required a party engaging in TAR to reach a 100% recall rate. “[W]hile parties must impose a reasonable construction on discovery requests and conduct a reasonable search when responding to the requests, the Federal Rules do not demand perfection.” Reinsdorf v. Skechers U.S.A., Inc., 296 F.R.D. 604, 615 (C.D. Cal. 2013); see also Da Silva Moore, 287 F.R.D. at 191 (“[C]omputer-assisted review is not perfect, [but] the Federal Rules of Civil Procedure do not require perfection.”); Treppel v. Biovail Corp., 233 F.R.D. 363, 374 (S.D.N.Y. 2006) (“[T]here is no obligation on the part of a responding party to examine every scrap of paper in its potentially
voluminous files in order to comply with its discovery obligations.”); Radiologix, Inc. v. Radiology & Nuclear Med., LLC, No. 15-4927-DDC-KGS, 2019 WL 354972, at *11 (D. Kan. Jan. 29, 2019) (explaining that courts do not require perfection, rather a party must conduct a reasonable search for responsive information pursuant to a reasonably comprehensive search strategy). “[I]t is inappropriate to hold TAR to a higher standard than keywords or manual review.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 129 (S.D.N.Y. 2015); see also Winfield v. City of New York, No. 15CV05236LTSKHP, 2017 WL 5664852, at *9 (S.D.N.Y. Nov. 27, 2017) (“[P]erfection in ESI discovery is not required . . . .”). In contrast, authority supports the reasonableness of Spirit’s recall rate. Courts have found
TAR processes achieving a 75% recall rate to be appropriate. See The Sedona Conference Tar Case Law Primer, 18 SEDONA CONF. J. 1, 37 (2017). Parties have mutually agreed on similar recall rates to include in court-approved ESI protocols.5 See, e.g., In re Bair Hugger Forced Air Warming Prod. Liab. Litig., No. MDL 15-2666(JNE/FLN), 2016 WL 3702959, at *2 (D. Minn. July 8, 2016) (setting an 80% target recall rate). A court should take into account the facts and circumstances of each case to determine whether a particular recall rate is reasonable, but recent guidance suggests that rates of 75-85% are appropriate in many cases. TAR GUIDELINES, supra,
5 The court notes that the parties’ Agreed Order Establishing Protocol for ESI and Paper Documents (ECF No. 40) does not address TAR or recall rates. at 23; see also TIMOTHY T. LAU & EMERY G. LEE III, FED. JUDICIAL CTR., TECHNOLOGY-ASSISTED REVIEW FOR DISCOVERY REQUEST: A POCKET GUIDE FOR JUDGES 12 (Mar. 28, 2017), available at https://www.fjc.gov/content/321579/technology-assisted-review-discovery-requests (“[A] recall or precision of 80% may be appropriate for one particular review, this does not mean that 80% is a benchmark for all other reviews.”).
In this case, the court finds that Spirit’s TAR was reasonable and production of the residual TAR documents is not proportional to the needs of the case. As explained above, the parties have a long and tortured history with ESI discovery. After the court’s ESI Protocol proved unworkable because Lawson demanded searches (both of custodians and search terms) that yielded such exceptionally low responsiveness rates, the parties resorted to Spirit conducting a TAR of the 322,000-document set identified in July 2019 (using Lawson’s revised proposed search terms and his selected custodians). The parties agreed to TAR parameters in September 2019 that included two levels of review. As Spirit points out, Lawson had no objection to a second-level review at that time, and even requested that Spirit’s second-level reviewers look at representative samples
of documents marked non-responsive to ensure that first-level reviewers were not undercoding responsive documents. (ECF No. 227-1, at 5.) The parties agreed to this second-level review previously, for sound reasons. As Spirit explains, this second-level review is necessary for Spirit’s law firm to review the documents for confidentiality, privilege, and compliance with TAR. Courts have found a second-level manual review following a TAR to be reasonable. See TAR Case Law Primer, supra, at 41. The court will not now allow Lawson to renege on this agreed second-level review. The parties never reached a mutual agreement as to the appropriate recall rate. Spirit believed a 65% recall rate was adequate. During the parties’ meet and confer in November 2019, Lawson informed Spirit that a 75-85% recall rate was typical. (ECF No. 227-1, at 15.) Spirit eventually agreed to reach a recall rate of 80% and “produced (or withheld as privileged) 23,951 documents, consisting of 170,083 pages (of these, 9,128 are responsive documents, and the remainder are non-responsive, irrelevant family members).” (ECF No. 240, at 3.) Spirit states that the 322,000 document set had a 3.3% responsive rate and, ultimately, it produced 85% of
those responsive documents, i.e. an 85% recall rate. (Id.) The court finds this rate reasonable under the circumstances of this case, and falls within the range that commentators—and Lawson himself—indicate is typical for most cases. Requiring Spirit to engage in a second-level review and produce the residual TAR documents is not proportional to the needs of the case. Under the proportionality analysis, the court must consider the importance of the discovery in conjunction with whether its burden or expense outweighs its likely benefit. Lawson argues generally that the residual TAR documents are “critical to [his] ability to rebut Spirit’s unsupported assertion that it and Arconic are in the same Business.” (ECF No. 227, at 8.) But Lawson admits that Spirit has already produced
documents bearing on this issue through the TAR process. (See id. at 7.) Spirit has also produced documents relating to overlap outside of the TAR process. (See ECF No. 240, at 2 (discussing Spirit’s proposal to continue reviewing and producing documents “regarding the ‘Business’ and overlap based on Spirit’s identification of custodians most likely to have relevant and responsive information”).) Spirit has spent approximately $600,000 in vendor fees and attorneys’ fees relating to the TAR process, all to produce less than 10,000 responsive documents. The court will not require Spirit to spend thousands more to review and ready for production documents that would appear to have marginal or duplicative benefit to the parties, if any benefit at all. Perhaps most telling to the court is the fact that—although Lawson refers to the residual TAR documents as “critical,” he does not apparently believe them to be critical enough to cover the costs of their production. And, there is no guarantee that the 800 documents that have been marked preliminarily responsive by first-level reviewers are actually responsive. As Spirit notes, second-level reviewers have made coding changes previously. (ECF No. 240, at 6.) Further, this is not a case where Lawson requires
an elusive “smoking gun” document to establish his claims. Spirit and Arconic either provided, marketed, or sold overlapping products or services, or they did not. Lawson has already likely received far more documents bearing on the issue than he would have had Spirit used any other form of review. See Da Silva Moore, 287 F.R.D. at 190 (“[T]he myth that exhaustive manual review is the most effective—and therefore the most defensible—approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort.” (alteration in original).) The court has also considered the remaining proportionality factors, including the amount in controversy and the parties’ resources. Lawson cursorily points out that approximately $50
million is at stake in this case, and that Spirit is a company worth $8 billion in annual revenues. (ECF No. 227, at 5-6.) As the court has recognized before, this is relevant to the analysis, but not determinative. If these factors were determinative, “they would eradicate proportionality considerations in every case against high-profile litigation targets with substantial resources.” Lawson v. Spirit AeroSystems, Inc., No. 18-1100-EFM-ADM, 2020 WL 1285359, at *8 (D. Kan. Mar. 18, 2020). In this case, the court is particularly troubled by Lawson’s apparent disincentive to meaningfully tailor his ESI demands to further the “just, speedy, and inexpensive” determination of this action. FED. R. CIV. P. 1. Lawson’s original motion to compel demanded that Spirit search more than 70 custodians using about 90 search terms. After the court imposed an ESI Protocol, Spirit identified the custodians most likely to have responsive ESI, and Lawson was allowed to select ten ESI custodians. Lawson apparently largely ignored Spirit’s list, selecting only three ESI custodians from Spirit’s list and selecting seven other ESI custodians who had not been listed as custodians most likely to have the categories of ESI Lawson was seeking. Compounding this
unwieldy list of custodians, Lawson provided Spirit with 803 search terms to run on those custodians’ ESI, with no effort to tailor those searches to particular ESI custodians. Given Lawson’s scattershot approach, it is no wonder that the ESI process yielded such exceptionally low responsive rates. Meanwhile, Spirit has incurred hundreds of thousands of dollars trying to appease Lawson. And, now, Lawson refuses to pay the approximate $40,000 for the second-level review and production of the residual TAR documents that he claims are “critical” but yet Lawson has not cited a single case or other authority to support what he effectively demands, which is a 100% target recall rate. This lack of reasonableness throughout the pendency of this case borders on the abusive.
Ultimately, the court finds that the second-level review and production of the residual TAR documents is not proportional to the needs of the case under Rule 26(b)(1). The court therefore denies Lawson’s motion. IT IS THEREFORE ORDERED that Larry A. Lawson’s Motion to Compel the Production of Responsive Documents (ECF No. 226) is denied. IT IS SO ORDERED. Dated April 9, 2020, at Topeka, Kansas. s/ Angel D. Mitchell Angel D. Mitchell U.S. Magistrate Judge