ORDER
THOMAS B. SMITH, United States Magistrate Judge
This matter comes before the Court on Defendants1 Sparton Corporation and Spar-ton Electronics Florida, Inc.’s Objection (Doc. 134) to the Report and Recommendation of the Special Master (Doc. 130). Plaintiff L-3 Communications Corporation has filed a response (Doc. 141), and on February 5, 2015, the Court held oral argument on the matter. The Court will SUSTAIN in part [664]*664and OVERRULE in part Defendants’ objections to the Special Master’s Report and Recommendation.
Background
Plaintiff, manufacturer of a “ruggedized” modem used by the United States military, brings this lawsuit against Defendants, who for a time manufactured two of the modem’s five major assemblies — the “Ref Gen CC” and the “Rx CCA.” Plaintiff alleges that the Ref Gen CC’s and Rx CCA’s it purchased from Defendants were defective, resulting in damages to Plaintiff (See Doc. 1). Defendants deny Plaintiffs allegations of liability and damages (Doc. 42).
The parties have executed an Electronic Discovery Protocol and other Discovery Agreements (“ESI Agreement”) (Doc. 137, pp. 2-10). In the ESI agreement, the parties agree to conduct electronic discovery by running mutually agreed search terms against databases and other electronic document repositories reasonably believed to contain relevant information (Id.).
On July 3, 2014, Plaintiff propounded a list of 153 search terms to Defendants (Doc. 134-4). The vast majority (113 of 153) of these terms are structured as follows: X and “Linkabit” or “L3” or “L-3” or “Ruggedized” or “Receiver” or “Rx” or “Ref Gen” or “Reference Generator,” where X is either a single string or series of strings connected by the Boolean operator “or.”2 Before Defendants had run any search terms, Plaintiff agreed to remove “Receiver” from each of these search terms; so as modified, these 113 terms read: X and “Linkabit” or “L3” or “L-3” or “Ruggedized” or “Rx” or “Ref Gen” or “Reference Generator” (Doc. 134, p. 7 n. 6). While the remaining search terms vary in structure, they include: 53. “Linkabit” or “L3” or “L~ 3”; 96. “Ref Gen” or “Reference Generator”; 109. “Ruggedized”; and 110. “Rx” or “Receiver CCA” or “Receiver Circuit Card Assembly.”
In response to Plaintiffs request, Defendants ran searches using the terms: “L-3,” “Rx,” “Linkabit,” “l-3com.com,” “28718,” “28731,” “33183,” “33176,” “36700,” “36035,” “Receiver CCA,” “23787,” “23811,” “36707,” “34426,” “36971,” and “Receiver Circuit Card Assembly” (Doc. 134, p. 16; Doc. 136-5). Unsatisfied with Defendants’ production, Plaintiff submitted a motion to compel to the Special Master, requesting that Defendants be required to run all of Plaintiffs search terms (Doc. 137-11).
The Special Master submitted his Report and Recommendation on Plaintiffs motion on November 24, 2014 (Doc. 130). The Special Master noted a passage from the Court’s previous order stating that no party could “veto” another party’s search terms, and reasoned that because “[t]he parties have not agreed between themselves on the limiting the search terms and the court has not revised, amended nor revoked its previous order regarding search terms[,] ... Defendants] shall run all search terms propounded by Plaintiff subject to further order of the court” (Doc. 130, p. 2-3).
Defendants objected to the R & R, and Plaintiff filed its response to the objections (Docs.134, 141). The parties’ disagreement focuses largely on the following search terms:
13. “Clearance” and “Component”
14. “Co planar” or “Co-planar” or “Coplanarity”
17. “Corrective Action Request” or “SCAR”
25. “Design of Experiments” or “Designs of Experiments” or “DOE”
26. “Det Norske Veritas”
30. Emails to, from, or including the email domain “emerson.com”
31. “Emerson”
39. “Flush
[665]*66545. “High Temp” or “High Temp” or “High Temperature” and “Solder”
47. “Indium”
52. “Lessons Learned”
53. “Linkabit” or “L3” or “L-3” (only “L3” disputed)
54. “Low-Temp or “Low Temp” or “Low Temperature” and “Solder”
61. “No Clean Flux” or “No-Clean Flux”
84. “Product Development Capabilities”
91. “Quality Management System”
94. “Quality Systems Manual”
109. “Ruggedized”
128. “Tin Connectors” or “Tinning Connectors”
130. “Trace Laboratories”
142. “Z Axis” or “Z-Aris”
(Doc. 134, p. 10; Doc. 141, p. 13-14). At the hearing, the parties advised the Court that they have resolved their dispute concerning the term “Ruggedized.”
Legal Standards
Standard of Review of Special Master’s Report and Recommendation
Pursuant to the Court’s order appointing the Special Master, the Court reviews objections to the Special Master’s findings of facts and conclusions of law de novo (Doc. 118, ¶ 8).3
Standards Governing Discovery
The Federal Rules of Civil Procedure adopt a liberal approach toward discovery, with the aim of ensuring that “civil trials in the federal courts [are] no longer ... earned on in the dark.” Hickman v. Taylor, 329 U.S. 495, 501, 67 S.Ct. 385, 91 L.Ed. 451 (1947); see also Tiedman v. American Pigment Corp., 253 F.2d 803, 808 (4th Cir.1958) (“[A] trial is not a sporting event, and discovery is founded upon the policy that the search for the truth should be aided.”). To this end, parties in federal litigation “may obtain discovery regarding any nonprivileged matter that is relevant to any party’s claim or defense.” Fed. R. Crv. P. 26(b)(1). Information need not be admissible at trial to be discoverable, so long as “the discovery appears reasonably calculated to lead to the discovery of admissible evidence.” Id.
Discovery of documents and electronically stored information is governed by Rule 34. Under the rule, parties may serve on each other requests “to produce and permit the requesting party ... to inspect, copy, test, or sample ... designated documents or electronically stored information — including writings, drawings, graphs, sound recordings, images, and other data or data compilations”— that are in the other party’s possession. Fed. R. Civ. P. 34(a)(1)(A). A party subject to a request for production “must respond in writing within 30 days after being served.” Fed. R. Civ. P. 34(b)(2)(A).
When a party subject to a request for production under Rule 34 fails to respond to the request or improperly refuses to produce the requested information, the requesting party may file a motion to compel. Fed. R. Civ. P. 37(a)(3)(B). If the court grants the motion, or if the discovery is provided after the motion is filed, the moving party is entitled to recover its reasonable expenses in making the motion, including attorney’s fees, unless “(i) the movant filed the motion before attempting in good faith to obtain the disclosure or discovery without court action; (ii) the opposing party’s nondisclosure, response, or objection was substantially justified; or (in) other circumstances make an award of expenses unjust.” Fed. R. Civ, P. 37(a)(5)(A). If it denies the motion, the court may enter a protective order under Rule 26(c) “and must, after giving an opportunity to be heard, re[666]*666quire the movant, the attorney filing the motion, or both to pay the [opposing party’s] reasonable expenses incurred in opposing the motion,” unless “the motion was substantially justified or other circumstances make an award of expenses unjust.” Fed. R. Crv. P. 37(a)(5)(B). If the court grants in part and denies in part a motion to compel, it may enter a protective order and shift expenses between parties as the court deems appropriate. Fed. R. Civ. P. 37(a)(5)(C).
Although the courts accord the discovery rules “a broad and liberal treatment,” Hickman, 329 U.S. at 507, 67 S.Ct. 385, there are limits to what a party may discover. A court “must limit the frequency or extent of discovery” upon determining that what is sought is unreasonably cumulative or duplicative or more reasonably obtainable from another source; that the requesting party has had ample opportunity to obtain information; or that the burden of the proposed discovery outweighs the likely benefits, “considering the needs of the case, the amount in controversy, the parties’ resources, the importance of the issues at stake in the action, and the importance of the discovery in resolving the issues.” Fed. R. Civ. P. 26(b)(2)(C).
Discussion
The Use of Search Terms in Electronic Discovery
The last quarter century has seen a revolution in the way information is stored and maintained. The cost of storing information electronically has plummeted by roughly six orders of magnitude, and with this decline in cost, individuals and organizations are storing more and more data electronically. See The Sedona Conference Best Practices Commentary on the Use of Search & Information Retreival Methods in E-Discovery, 15 Sedona Conf. J. 217, 220 (2014) (hereinafter "Best Practices”). A 250 gigabyte laptop hard drive can store the information equivalent of tens of millions of pages of text. Id. When such large quantities of information become subject to discovery in litigation, “linear” review becomes impossible. Instead, parties and their lawyers must rely on information retrieval tools to identify relevant and responsive documents.
The simplest and most common of these information retrieval tools is the “keyword search” — a “set-based search[ ] using simple word or word combinations, with or without Boolean and related operators”. Best Practices at 231. At a minimum, keyword searching permits a party to efficiently identify documents containing a specific word or combination of words. Windy City Innovations, LLC v. America Online, Inc., 2006 WL 2224057, at *3 (N.D.Ill. July 31, 2006); see also In re Lorazepam & Clorazepate, 300 F.Supp.2d 43, 46 (D.D.C.2004) (“The glory of electronic information is not merely that it saves space but that it permits the computer to search for words or ‘strings’ of text in seconds.”). “ ‘[B]asic keyword searching techniques have been widely accepted by both courts and parties as sufficient to define the scope of their obligation to perform a search for responsive documents — ’ ” Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251, 261 (D.Md.2008) (quoting The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 194-95, 201-02 (2008)).
While the utility of keyword searching is widely accepted, keyword searches are not a perfect tool for distinguishing relevant documents from irrelevant ones. The effectiveness of keyword searches can be measured by several metrics, the most basic of which are “precision” and “recall.” Best Practices at 237. A search method’s precision is defined as the percentage of documents retrieved by the methods that are relevant. Id. at 238. The higher a search’s precision, the fewer “false positives” there are. A search method’s recall is defined as the percentage of all relevant documents in the search universe that are retrieved by that search method. Id. at 237. The higher the recall, the fewer “false negatives” (i.e., relevant but unretrieved documents) there are. Often, there is a trade-off between precision and recall — a broad search that misses few relevant documents will usually capture a lot of irrelevant documents, while a narrower [667]*667search that minimizes “false positives” will be more likely to miss some relevant documents.
Crafting appropriate search terms requires “careful thought, quality control, testing, and cooperation with opposing counsel,” and is not always an easy task. William A. Gross Constr. Assocs., Inc. v. American Manufacturers Mut. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y.2009); see also In re Seroquel Products Liability Litig., 244 F.R.D. 650, 652 (M.D.Fla.2007) (“[W]hile key word searching is a recognized method to winnow relevant documents from large repositories, use of this technique must be a cooperative and informed process”). Attorneys must recognize that proper use of information retrieval tools, including keyword searching, requires expertise that a competent or even a highly skilled litigator may not necessarily possess. Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 338 (D.D.C.2008); United States v. O’Keefe, 537 F.Supp.2d 14, 24 (D.D.C.2008). “Whether search terms or “keywords” will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.” O’Keefe, 537 F.Supp.2d at 24 (citing George L. Paul & Jason R. Baron, Information Inflation: Can the Legal System Adapt?, 13 Rich. J.L. & Tech. 10 (2007)). Selection of search terms therefore “requires careful advance planning by persons qualified to design effective search methodology. The implementation of the methodology should be tested for quality assurance; and the party selecting the methodology must be prepared to explain the rationale for the method chosen to the court, demonstrate that it is appropriate for the task, and show that it was properly implemented.” Victor Stanley, 250 F.R.D. at 262; see also Seroquel, 244 F.R.D. at 652 (“Common sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness.”).
Lawyers involved in crafting search terms must also have a thorough understanding of how keyword searching works. Best Practices at 241, 246. From their experience with commercial databases like Westlaw and Lexis, lawyers should already be familiar with the theory and implementation of Boolean searches using operators “AND,” “OR,” and “NOT,” along with the use of parentheses to specify the order in which Boolean operations should be performed. Lawyers should also be familiar with the use of phrase and wildcard searches, as well as proximity searches.4 Lawyers must also understand the functioning and capabilities of any software used to implement keyword searching. Best Practices at 246-47. And, they must be able to explain the methods and tools they use to the court, opposing parties, and their clients. Id. at 246.
Finally, and perhaps most importantly, for keyword searching to be an effective information retrieval tool, the process of crafting keywords must be a cooperative one. The party responsible for production will usually be most familiar with its own records, and therefore will be “best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” Sedona Conference, Sedona Principles at 38 (2d ed.2007). Efficiently crafting search terms requires “input from the ESI’s custodians as to the words and abbreviations they use.” William A. Gross, 256 F.R.D. at 136; see also id. at 135 (complaining about lawyers “designing keyword searches in the dark, by the seat of the pants, without adequate (indeed, here, apparently without any) discussion with those who wrote the emails”).
Plaintiffs Search Terms: Search terms of the form: X and “Linkabit” or “L3” or “L-3” or “Ruggedized” or “Rx” or “Ref Gen” or “Reference Generator”
The search terms in this category are problematic on two levels. First, because Plaintiff didn’t bother to include parentheses to indicate which operations should be evaluated first, the search terms are ambiguous. Indeed, any expression containing an “and” and an “or” will be ambiguous without parentheses, as the diagrams below illustrate:
[668]*668[[Image here]]
The Court assumes Plaintiff intended the “or” operators to be evaluated first and the “and” operators to be evaluated last.
Second, after Plaintiff -withdrew “Receiver” from these 114 search terms they became duplicative of four other search terms in Plaintiff’s list: 53. “Linkabit” or “L3” or “L-3”; 96. “Ref Gen” or “Reference Generator”; 109. “Ruggedized”; and 110. “Rx” or “Receiver CCA” or “Receiver Circuit Card Assembly.” Any document that hits on one of these 114 search terms must contain “Linkabit,” “L3,” “L-3,” “Ruggedized,” “Rx,” “Ref Gen,” or “Reference Generator.” But any document containing one of these terms will also be a hit for search term 53, 96, 109, or 110. Since they do not retrieve any documents that the remainder of Plaintiffs search terms miss, they have no independent significance and the Court can safely ignore them.
Plaintiff’s Search Terms: “LS”
Plaintiff argues that Defendants should be required to run the search term “L3.” (Doc. 141, p. 17). Defendants point out that they have already run the term “L-3,” and argue that requiring them to now run “L3” will “in effect require [Defendants] to repeat the entire process again at great expense with the most likely result a duplicate production of a lot of pages.”5 (Doc. 134, p. 21). Defendants also fault Plaintiff for not raising Defendants’ failure to run the term “L3” before October 16, when it filed its motion with the Special Master. (Id.). Plaintiff responds that running “L3” in addition to “L-3” is necessary because the parties use “L3” and “L-3” interchangeably to refer to Plaintiff (Doc. 141, p. 17).
Defendants do not dispute that the search term “L3” is aimed at documents relevant to the case, or that it is unlikely to uncover many irrelevant documents. In fact, they represent to the Court that “L3” was among the terms they agreed to search. (See Doc. 134-4). Defendants’ only objection is that a search for “L3” will be expensive and, for the most part, only uncover documents that they have already produced (Doc. 134, p. 21). Defendants presented no evidence of how expensive such a search might be, and given that “L3” is particularly likely among the disputed terms to uncover relevant evidence, the Court rejects Defendants’ objection that a search for “L3” would be overly burdensome.
As for duplication, the Court suspects that this problem could be eliminated by constructing an appropriate search term designed to include documents with the term “L3” but exclude documents with terms already searched for. At the hearing, the Court asked if Defendants could search their [669]*669database using the Boolean “not” operator in conjunction with “and,” for example, by searching “Solder” and not “L-3” to uncover documents that contained the term “Solder” but not “L-3.” Defendants did not know whether or not they could conduct such a search. If they can, then by running a search for “L3” and not “L-3,” they can eliminate from the search results any documents uncovered when they searched for “L-3.”6
Finally, it is possible that the search for “L-3” automatically included all documents that included the term “L3.” The hit counts Defendants have provided for the disputed terms suggest that their e-discovery software does not distinguish between hyphens and spaces between words. “Z Axis” and “Z-Axis” return the same number of hits (20,-052), as do “No Clean Flux” and “No-Clean Flux” (5,291) and “Co planar” and “Co-planar” (457). It is possible that “Z Axis” with a space and “Z-Axis” with a hyphen both happen to appear 20,052 times in Defendants’ database, but it is more likely that the two searches are looking for the same thing and uncovering the same documents. If a search for “L-3” finds both “L-3” and “L 3,” then perhaps it also finds “L3.”7 At the hearing, the parties could not give a firm answer to this question. They should try to find out, because if the answer is yes, then their dispute over the search term “L3” is moot. If the answer is no, then Defendants will be required to run the search term “L3,” although Defendants may narrow the search term by using “and not” to exclude documents that were hits for searches they have already performed.
Plaintiffs Search Terms: Remaining search terms
The remaining disputed search terms reflects a more basic disagreement about the appropriate scope of discovery in this case. Defendants argue that Plaintiff should be limited, for the most part, to discovery concerning the Ref Gen CC and Rx CCA, (Doc. 134, pp. 18, 20). They object to the disputed search terms on the grounds that they are not limited to the Ref Gen CC or Rx CCA, or even to a limited number of other specific products made by Defendants, but are aimed at Defendants’ products generally.
Plaintiff argues that it should be entitled to discover information relating to other products Defendants designed and manufactured for other customers (Doc. 141, pp. 11,15). It says that its search terms are “calculated to hit on documents critical to L-3’s claims in this case” and that “the documents that should result are critical for L-3 to determine, for example, how Sparton utilized its ‘standard Quality Management System’ under the Statement of Work, or interpreted ‘high professional standards’ under L-3’s Purchase Orders.” (Id., p. 13).
The Court agrees with Plaintiff that Defendants’ application of manufacturing standards in “other, similar contexts” and complaints by Defendants customers about products “subject to the same or similar processes” fall within the scope of discoverable information under Fed. R. Civ. P. 26(b)(1). But, this does not mean Plaintiff is entitled to unfettered discovery regarding Defendants’ other products, even those that could be regarded as “similar” to the Ref Gen CC and Rx CCA. The benefits of Plaintiffs forays into discovery about unrelated products are likely to be relatively limited. Most information about unrelated products is likely to be completely irrelevant, and what is relevant is unlikely to have great probative value, relative to information about the Ref Gen CC and Rx CCA itself. Because the expected benefit of the discovery Plaintiff seeks is limited, the Court will not compel Defendants to respond to Plaintiffs requests in this area where responding would expose Defendants to disproportionate expense. See Fed. R. Civ. P. 26(b)(2)(C).
[670]*670The search terms Plaintiff has chosen to do this job simply cast too wide a net. A search for “Corrective Action Request” or “SCAR,” for example, will return every single customer complaint in Defendants’ database. It is likely that most of these complaints are completely irrelevant to this case. Likewise, the search terms aimed at high temperature and low temperature solder will hit on communications that lack even arguable relevance (e.g., an email asking if someone can order some high-temperature solder). And, Defendants’ analogy between searching an electronics design and manufacturing firm for “DOE” (short for “design of experiment”) and “Quality Management System” to searching a law firm for “pleading” on a law firm’s servers seems an apt one. Absent significant narrowing, the expansive search terms proposed by Plaintiff will not do.
Resolving disputes over search terms
Today’s Order compels Defendants to run the search term “L3” (unless Defendants can show that the previous search for “L-3” uncovered all documents containing “L3”), and relieves Defendants of the burden of running any other search terms. While it does not prevent Plaintiff from pursuing discovery into matters like customer complaints and quality management systems as they relate to products other than the Ref Gen CC and Rx CCA, it makes clear that discovery requests and proposed search terms aimed at unrelated products must be carefully tailored so as to minimize the burden on Defendants.
If confronted with a search term they believe is overly burdensome, Defendants have an obligation not just to assert them objection, but explain the grounds for the objection and to point to evidence supporting it. If asked, Defendants must offer specific suggestions for narrowing the offending search terms in a way that addresses their concerns while still retrieving as many of the relevant documents targeted by the disputed search terms as possible. See The Case for Cooperation, 10 Sedona Conf. J. 339, 354 (2009) (“[C]ounsel may not use his superior information as to the location or nature of responsive documents to thwart good faith discovery requests by refusing to engage cooperatively to identify ... the search terms likely to produce responsive documents.”). This obligation arises from the counsel’s duty under Rule 37(a)(1) and Local Rule 3.01(g), to confer in good faith with opposing counsel in an effort to resolve or at least narrow the scope of a discovery dispute before bringing the dispute before the Court.
Conclusion
Defendants’ objections to the Special Master’s Report and Recommendation are SUSTAINED in part and OVERRULED in part. Defendants are ORDERED run the search term “L3,” unless Defendants can show that the previous search for “L-3” located all documents containing “L3.” Defendants may modify the search query to exclude documents located by searches it has already conducted. Defendants need not run the remainder of the disputed terms from Plaintiffs first set of search terms.
DONE and ORDERED in Orlando, Florida on February 12, 2015.
REPORT AND RECOMMENDATION
MANUEL FARCH, Special Master
SPECIAL MASTER MANUEL FAR-ACH, upon the arguments and filings presented, enters the following proposed Report and Recommendation.
Background Facts
1. Plaintiff and Defendant are involved in complex litigation regarding a “ruggedized” modem capable of military use.
2. Discovery disputes have arisen with regard to discovery propounded between Plaintiff and Defendant and upon each third party.
3. Specifically, the Defendant objects to discovery propounded upon third party testing and cleanliness laboratories employed by Defendant known as Trace Laboratories and Foresite, Inc. (the “Testing Laboratories”).
4. The issue in this case is whether the soldering or other connections within the modems were defective to the point not being in [671]*671compliance with the contact and other legal requirements.
5. There may be confidential trade information with regard to answering this discovery, but the risk of the releasing private and confidential information can be minimized (if not eliminated) by an appropriate confidentiality order of the form typically employed in the Middle District of Florida and in camera inspection when appropriate.
6. A dispute has also arisen with regard to search terms to be used by the Defendant in searching its ESI.
7. The search terms issue was the subject of a previous court hearing, and the court directed that no party could “veto” another party’s search terms.
8. Defendant has, however, objected to several of Plaintiffs search terms apparently arguing the search terms produce an apparently large number of documents whose relevance is questionable.
9. Plaintiff has apparently complied with the search propounded upon it by Defendant.
UPON THESE BACKGROUND FACTS, the Special Master makes the following
Recommendation
1. The discovery propounded on the Testing Laboratories should be limited to production by the Testing Laboratories of information that products manufactured by or delivered for testing by Defendant were tested for cleanliness or connectivity (or lack thereof), and the results of those tests. The complete results of those tests should, subject to confidentiality and in camera concerns, be produced.
2. Discovery upon the Testing Laboratories sought beyond that set forth above may be relevant in the future, but its relevance has not been shown at this stage of the proceedings.
3. The parties have not agreed between themselves on the limiting of search terms and the court has not revised, amended nor revoked its previous order regarding search terms so the Defendant shall run all search terms propounded by Plaintiff subject to further order of the court.
4. In light of the complex issues presented in this case and vigorous advocacy, it is recommended that weekly or bi-monthly telephonic hearings be held by the Special Master in order to resolve any pending issues.
DONE AND ORDERED in West Palm Beach, Florida this 24th day of November, 2014.