San Francisco Division NANTWORKS, LLC, et al., Case No. 20-cv-06262-LB
Plaintiffs, ORDER GRANTING NIANTIC SUMMARY JUDGMENT v. Re: ECF No. 245 Defendant. NantWorks sued Niantic for infringing NantWorks’ patents in Niantic’s augmented-reality (AR) game apps Pokémon Go and Harry Potter: Wizards Unite. The games use the camera and GPS system on a mobile device and an AR platform to superimpose AR objects onto digital representations of a device’s actual surroundings. For example, Pokémon Go users go on scavenger hunts to collect virtual objects (such as Pokémon characters) that are represented on the device as if they are in the player’s real-world location. U.S. Patent No. 10,4034,051 (the ’051 patent) is directed to incorporating virtual objects into a digital representation of an actual scene surrounding a device (such as a mobile phone). Niantic moved for summary judgment in part on the ground that the ’051 claims are directed to an abstract idea (receiving information about a location and displaying materials based on that information), lack an inventive concept, and thus are not patent-eligible The ’051 patent (titled “Interference Based Augmented Reality Hosting Platforms”) was filed in 2018, issued in 2019, and claims priority to April 8, 2011.1 It is directed to incorporating virtual objects seamlessly and more realistically into a digital representation of an actual scene surrounding a device (such as a mobile phone).2 In one prior AR system, for example, users had access to different AR levels and had to manually select them.3 Other existing AR systems — while allowing some AR content contextualization based on the environment surrounding the device — failed “to appreciate that objects within an environment or scene can interfere with each other to give rise to an augmented reality experience.”4 “[E]xisting infrastructures fail[ed] to treat [AR] objects as distinct manageable objects in an infrastructure[-agnostic] manner[.]”5 This meant that some AR objects (a gaming avatar, for example) would always appear on top of other AR objects and could not go behind or around other AR objects. The ’051 patent allegedly provides technological improvements in delivering a realistic AR experience with AR objects (within a scene) that can interfere with each other. For example, [N]etworking nodes within a networking fabric can provide augmented reality objects . . . to edge AR-capable devices [mobile phones, tablets, and computers, for example]. . . . As the edge devices . . . interact with the networking fabric by exchanging data, the fabric can determine which augmented reality objects are most relevant or even which augmented reality itself is most relevant for the device based on context derived from observed real-world elements. Augmented reality context can now be used to determine how elements in a scene, a location relevant to an individual, can interfere with each other to give rise to relevant augmented reality experiences.6
1 U.S. Patent No. 10,403,051 (filed Nov. 9, 2018) – ECF No. 114-2. Citations refer to material in the Electronic Case File (ECF); pinpoint citations are to the ECF-generated page numbers at the top of documents. 2 Id. at 2 (at [57]), 11 (col. 1 ll. 63–67), 12 (col. 3 ll. 58–67, col. 4 ll. 1–31). 3 Id. at 11 (col. 1 ll. 53–61). 4 Id. (col. 2 ll. 3–29, 37–41). 5 Id. at 12 (col. 3 ll. 8–11). The alleged improvements allow incorporation of the virtual object more smoothly and realistically into a digital representation of a device by selecting virtual objects (and representing them on the device) based on the elements surrounding the device and enhancing or suppressing the virtual object in the presentation, again based on the context and factors surrounding the device.7 NantWorks’ remaining asserted claims are 7, 22, 23, and 25, which all depend from claim 1.8 Claim 1 recites the following: 1. An augmented reality (AR) platform system comprising: an AR object repository storing available AR objects in a first non-transitory computer readable memory; and an AR server coupled with the AR object repository and, upon execution of software instructions stored in a second non-transitory computer readable memory by a processor, is configured to: obtain digital data representative of an environment of an AR capable mobile device, the digital data including a device location of the AR capable device and a virtual element attribute; determine at least one context related to the AR capable device and pertinent to the environment based at least on the device location; identify relevant AR objects from the AR repository representing available AR objects corresponding to the at least one context; determine whether to alter presence of a relevant AR object based on at least the device location and the virtual element attribute; and cause the AR capable device to render the relevant AR object according to its altered presence.9 The court’s claim construction was as follows:10 Claim Term Construction “virtual element attribute” plain and ordinary meaning “augmented reality”/“AR” “the presentation of virtual objects in a scene alongside of real-world elements” “determine at least one context related to the AR “determine at least one context related to the AR capable device and pertinent to the environment capable device and related to the environment based at least on the device location” based at least on the device location”
7 Id. at 19 (col. 17 ll. 63–66, col. 18 ll. 18–21, 42–45). 8 Mot. – ECF No. 246 at 7; Opp’n – ECF No. 264 at 10. 9 ’051 Patent – ECF No. 114-2 at 21 (col. 21 ll. 46–67, col. 22 ll. 1–2). “AR object” “the virtual object that is to be presented to the user” “identify relevant AR objects from the AR subject to the construction of “AR” and “AR repository representing available AR objects object,” plain and ordinary meaning corresponding to the at least one context” “alter presence”/“altered presence” “alter the presentation” The following are asserted claims 7, 22, 23, and 25, which depend from claim 1. Remaining Asserted Claims 7. The system of claim 1, wherein the relevant AR objects is caused to be rendered based on an orientation of the AR capable device relative to the environment. 22. The system of claim 1, wherein the determination of whether to alter presence of the relevant AR depends on a time. 23. The system of claim 22, wherein the presence alteration of the relevant AR changes with time. 25. The system of claim 24, wherein the commercial transaction includes an exchange of virtual currency.11 The court must grant summary judgment where there is no genuine dispute as to any material fact and the moving party is entitled to judgment as a matter of law. Fed. R. Civ. P. 56(a); Anderson v. Liberty Lobby, Inc., 477 U.S. 242, 247–48 (1986). Material facts are those that may affect the outcome of the case. Anderson, 477 U.S. at 248. A dispute about a material fact is genuine if there is sufficient evidence for a reasonable jury to return a verdict for the nonmoving party. Id. at 248–49. The party moving for summary judgment has the initial burden of informing the court of the basis for the motion and identifying portions of the pleadings, depositions, answers to interrogatories, admissions, or affidavits that demonstrate the absence of a triable issue of material fact. Celotex Corp. v. Catrett, 477 U.S. 317, 322–23 (1986). To meet its burden, “the moving party must either produce evidence negating an essential element of the nonmoving party’s claim or defense or show that the nonmoving party does not have enough evidence of an essential 11 ’051 Patent – ECF No. 114-2 at 21 (claim 7: col. 22 ll. 17–19; claims 22–23: col. 22 ll. 56–60; claim element to carry its ultimate burden of persuasion at trial.” Nissan Fire & Marine Ins. Co. v. Fritz Cos., 210 F.3d 1099, 1102 (9th Cir. 2000); see Devereaux v. Abbey, 263 F.3d 1070, 1076 (9th Cir. 2001) (“When the nonmoving party has the burden of proof at trial, the moving party need only point out ‘that there is an absence of evidence to support the nonmoving party’s case.’”) (quoting Celotex, 477 U.S. at 325). “Where the moving party will have the burden of proof on an issue at trial, the movant must affirmatively demonstrate that no reasonable trier of fact could find other than for the moving party.” Soremekun v. Thrifty Payless, Inc., 509 F.3d 978, 984 (9th Cir. 2007). If the moving party meets its initial burden, then the burden shifts to the nonmoving party to produce evidence supporting its claims or defenses. Nissan, 210 F.3d at 1103. (“Once the moving party carries its initial burden, the adverse party may not rest upon the mere allegations or denials of the adverse party’s pleading, but must provide affidavits or other sources of evidence that set forth specific facts showing that there is a genuine issue for trial.”) Devereaux, 263 F.3d at 1076 (cleaned up). If the non-moving party does not produce evidence to show a genuine issue of material fact, the moving party is entitled to summary judgment. Celotex, 477 U.S. at 322–23. In ruling on a motion for summary judgment, the court does not make credibility determinations or weigh conflicting evidence. Instead, it views the evidence in the light most favorable to the non-moving party and draws all factual inferences in the non-moving party’s favor. E.g., Matsushita Elec. Indus. Co., v. Zenith Radio Corp., 475 U.S. 574, 587–88 (1986); Ting v. United States, 927 F.2d 1504, 1509 (9th Cir. 1991). Under 35 U.S.C. § 101, “[w]hoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor.” The “laws of nature, physical phenomena, and abstract ideas” are “specific exceptions to § 101’s broad patent-eligibility principles.” Bilski v. Kappos, 561 U.S. 593, 601 (2010) (citation omitted). The Supreme Court’s two-step test governs the § 101 inquiry. Alice Corp. Pty. v. CLS Bank directed to one of [the] patent-ineligible concepts,” such as abstract ideas. Id. If the claims are directed to an abstract idea, then at step two, the court “considers the elements of each claim individually and as an ordered combination to determine whether the additional elements ‘transform the nature of the claim’ into a patent eligible application.” Id. Step two is a “search for an ‘inventive concept,’” meaning, “an element or combination of elements that is sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the ineligible concept itself.” Id., at 217–218. (cleaned up). 1. Step One: Abstract Idea Niantic contends that the claim 1 is representative of the asserted claims and is directed to the abstract idea of receiving information about a location and displaying materials based on that information.12 NantWorks counters that the claims represent specific, concrete improvements in computer technology for presenting AR in a more realistic manner.13 On this record, the asserted claims are directed to the abstract idea of receiving information about a location and displaying materials based on that information. “The Supreme Court has not established a definitive rule to determine what constitutes an ‘abstract idea’ sufficient to satisfy the first step of the Mayo/Alice inquiry.” Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1334 (Fed. Cir. 2016). Courts “have approached the Step 1 ‘directed to’ inquiry by asking what the patent asserts to be the focus of the claimed advance over the prior art,” focusing “on the language of the [a]sserted [c]laims themselves . . . considered in light of the specification.” TecSec, Inc. v. Adobe Inc., 978 F.3d 1278, 1292–93 (Fed. Cir. 2020) (cautioned against “overgeneralizing claims in the § 101 analysis” because “characterizing the claims at a high level of abstraction all but ensures that the exceptions to § 101 swallow the rule”) (cleaned up). “In cases involving software innovations, th[e] inquiry often turns on whether the claims focus on specific asserted improvements in computer capabilities or instead on a process or system that
12 Mot. – ECF No. 246 at 9. NantWorks did not dispute that claim 1 is representative. qualifies [as] an abstract idea for which computers are invoked merely as a tool.” TecSec, 978 F.3d at 1293. For example, if a process can be performed in other ways — mentally, or by putting pen to paper — it is an abstract idea. Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 1147 (Fed Cir. 2016). Conversely, if the claims provide a specific technical solution — such as, in Data Engine Technologies, a specific method for navigating three-dimensional spreadsheets and the resulting improvement in functionality from prior art — then the claims are not abstract ideas and are patent eligible. Data Engine Techs. LLC v. Google LLC, 906 F.3d 999, 1002, 1007–08 (Fed. Cir. 2018). Niantic contends that claim 1, which is representative of the asserted claims, recites three abstract steps: (1) receiving location information; (2) matching material to that location information; and (3) displaying the material (altering presence) based on that information.14 It illustrates this by mapping the limitations of representative claim 1 to these three abstract steps. Content Extraction & Transmission LLC v. Wells Fargo Bank, Nat’l Ass’n, 776 F.3d 1343, 1349 (Fed. Cir. 2014) (it is enough to analyze a representative claim under § 101 where the claims are substantially similar and linked to the same abstract idea).
Claim 1 Abstract Step an augmented reality (AR) platform system comprising:
an AR object repository storing available AR objects in a first Generic computer hardware non-transitory computer readable memory; and (with material to be displayed) an AR server coupled with the AR object repository and, upon execution of software instructions stored in a second non- transitory computer readable memory by a processor, is configured to: obtain digital data representative of an AR capable mobile Receiving location device, the digital data including a device location of the AR information capable device and a virtual element attribute;
determine at least one context related to the AR capable device
and pertinent to the environment based on at least the device Matching material to that location; location information identify relevant AR objects from the AR repository representing available AR objects corresponding to the at least one context; determine whether to alter presence of a relevant AR object based on at least the device location and the virtual element Displaying material based attribute; and on that information15 cause the AR capable device to render the relevant AR object according to its altered presence. Niantic points to the specification to confirm the high-level nature of claim 1. For the “receiving location information limitations . . . [t]he environment data can include a broad spectrum of data reflecting the real-world environment.”16 For the “matching material to that location information” limitation, “[c]ontexts can take on many different forms and be defined as desired,” and relevant AR objects are “those objects pertaining to” the context.17 In other words, Niantic contends, it’s “just information.”18 Also, the patent states that “[e]nhanced presence and suppressed presence can take many different forms depending on the nature of relevant AR objects,” including “[a]t a most basic level, presence could simply mean relevant AR objects are present (enhanced) or not present (suppressed).”19 This, Niantic says, means that “alter[ing] presence” can be just showing or not showing information.20 NantWorks counters that the claims represent specific, concrete improvements in computer technology for presenting AR in a more realistic manner by considering the context of a situation and the way scene elements interfere with the virtual elements presented in that scene.21 “The platform system then obtains digital data representative of an environment of an AR capable device,” “including a device location of the AR capable device” and “a virtual element attribute.”22 Then, “at least one context” is determined “related to the AR capable device and
15 Id. at 10. 16 Mot. – ECF No. 246 at 10 (quoting ’051 Patent – ECF No. 114-2 at 14 (col. 7 ll. 7–8)). 17 Id. (quoting ’051 Patent – ECF No. 114-2 at 14 (col. 8 ll. 23–24), 16 (col 12 ll. 34–36)). 18 Id. 19 Id. at 11–12 (quoting ’051 Patent – ECF No. 114-2 at 19 (col. 18 ll. 18–23)). 20 Id. at 11. 21 Opp’n – ECF No. 264 at 10 (citing Turk Rebuttal Rep. – ECF No. 245-8 at 28 (¶ 101), 11–13 (¶¶ 312, 315). pertinent to the environment based on at least the device location.”23 Then, the system “identif[ies] relevant AR objects from the AR object repository representing available AR objects corresponding to the at least one context.”24 The system “determine[s] whether to alter the presence of a relevant AR object based on at least the device location and the virtual element attribute.”25 Finally, the “relevant AR object” is rendered on “the AR capable device . . . according to its altered presence.”26 “When operated in this manner, virtual objects are selected and displayed on the device based on the device surroundings with presentation of the virtual object enhanced or suppressed based on the particular situation and the device location.”27 In an earlier order, the court invalidated the ’518 patent because providing information based on a location on a map is an abstract idea.28 That analysis supports the conclusion that the ’051 patent is abstract because it is directed to filtering or picking information or materials relevant to a location or context, which is a human problem, not any specific improvement to a computing technology. TecSec, 978 F.3d at 1293. Recent cases support the conclusion that claims like these are abstract. For example, in Sanderling Mgmt. Ltd. v. Snap Inc., the Federal Circuit affirmed the district court’s holding of § 101 invalidity for the claims of three patents directed to a method using distribution rules to load digital branding functions when certain conditions were met. 65 F.4th 698, 701 (Fed. Cir. 2023). The Federal Circuit considered claim 1 of the Sanderling ’412 patent to be representative. Id. at 701. Niantic identifies the similarities between this patent and the Sanderling patent: Figure 4 of the ’051 patent gives the example of displaying material (a racing pennant) based on information about a location (such as GPS or weather), and the Sanderling Figure 7 presents material (a racecar driver) based on information about location.29 Niantic also maps the limitations
23 Id. (quoting ’051 Patent – ECF No. 114-2 at 21 (col. 21 ll. 59–61)). 24 Id. (quoting ’051 Patent – ECF No. 114-2 at 21 (col. 21 ll. 62–64)). 25 Id. (quoting ’051 Patent – ECF No. 114-2 at 21 (col. 21 ll. 65–67)). 26 Id. (quoting ’051 Patent – ECF No. 114-2 at 21 (col. 22 ll. 1–2)). 27 Id. (citations to patent omitted). 28 Order – ECF No. 149 at 8–12 (collecting and analyzing cases). The court does not repeat the analysis and instead incorporates it herein by this reference. and steps of claim 1 of the ’051 patent against claim 1 of the ’412 patent.30 It is hard to conclude in the face of this comparison — and given the extensive analysis that the court has undertaken in its previous orders — that the patents are directed to improving the prior AR technology, not merely displaying material based on information about location. The dependent claims do not change this analysis. Claim 7 adds the use of orientation to display information, which again is the abstract idea of using information about a device’s location.31 Claims 22 and 23 are directed to the use of time in displaying information.32 Intellectual Ventures I LLC v. Cap. One Bank (USA), 792 F.3d 1363, 1370 (Fed. Cir. 2015). Claim 25 adds the concept of using money, which is abstract.33 cxLoyalty, Inc. v. Maritz Holdings Inc., 986 F.3d 1367, 1377 (Fed. Cir. 2021) (affirming invalidity of claims directed to automated system for transactions using company- loyalty points because “[h]umans have long intermediated these very transactions by collecting and relaying the very same information”). As to NantWorks’s assertion that the ’051 patent improves computer functionality, there are efficiencies (such as speed) that attend automation, but those do not amount to a technological improvement or confer patent eligibility on an otherwise abstract idea. Intellectual Ventures I, 792 F.3d at 1367 (“claiming the improved speed or efficiency inherent with applying the abstract idea on a computer [does not] provide a sufficient inventive concept”); Ericsson Inc. v. TCL Commc’n Tech. Holdings, 955 F.3d 1317, 1330 (Fed. Cir. 2020). The following points also inform the analysis here. First, NantWorks strays from the claims in its arguments. For example, it talks about the seamless linking of previously separate augmented realities.34 Assuming this is a technological improvement, it is not in the asserted claims. Similarly, as Niantic points out, NantWorks contends
30 Id. at 15–16. 31 Id. at 17–18 (collecting and analyzing cases on this point). 32 Id. at 18. 33 Id. that the ’051 claims improved technology by encompassing the concept of interference in between the elements, which causes enhancement or suppression.35 Again, this is not in the claims. Second, this patent — unlike the other invalidated patents — is being decided at summary judgment. Niantic summarizes sealed testimony by the inventor (Dr. Patrick Soon-Shiong) that supports the conclusion that the claims are directed only to abstract ideas about information.36 Third, the generic benefit associated with displaying content based on a device’s location is about the abstract concept of content selection.37 Interval Licensing v. AOL, 896 F.3d 1335, 1345 (Fed. Cir. 2018) (“the collection, organization, and display of two sets of information on a generic display is abstract”). Fourth, NantWorks cites cases that do not change the outcome. McRO v. Bandai Namco Games Am. involved a specified, automated, rules-based process for facial animation that was different from manual approaches performed by animators. 837 F.3d 1299, 1314–16 (Fed. Cir. 2016). By contrast, this case involves generic steps. Sanderling, 65 F.4th at 703; Interval Licensing, 896 F.3d at 1346 (distinguishing McRO). Core Wireless, for example, improved the functioning of the computer, which is not the case here. Core Wireless Licensing S.A.R.L. v. LG Elecs., 880 F.3d 1356, 1363 (Fed. Cir. 2018). The court previously distinguished Blackbird Tech. LLC v. Niantic, Inc. No. 17-cv-1810-RGA, 2018 WL 5630452, at *2 (D. Del. Oct. 31, 2018).38 The patent claims there were directed to technical steps of integrating images of real spaces into video images of virtual spaces. Id. It also preceded other cases, including Sanderling. In sum, the asserted claims do not focus on asserted improvements in computer technology and instead are directed to an abstract idea. 35 Reply – ECF No. 274 at 10 (citing Opp’n – ECF No. 264 at 9–10). 36 Mot. – ECF No. 246 at 11–12 (citing sealed deposition testimony). 37 Reply – ECF No. 274 at 10 (making this point). ] 2. Step Two: Inventive Concept 2 “[A]n inventive concept must be evident in the claims.” RecogniCorp, LLC v. Nintendo Co., 3 855 F.3d 1322, 1327 (Fed. Cir. 2017) (citing Alice, 573 U.S. at 221). The claims here describe 4 only generic computer components applied to the abstract idea of providing information based on 5 receiving information about a location and displaying materials based on that information. This is 6 not an inventive concept. 7 For a software patent, the step-two analysis of whether the asserted claims are “more than 8 well-understood, routine, or conventional” overlaps with the step-one analysis of whether the 9 claims focus on a specific asserted technical advance or improvement. BSG Tech. v. BuySeasons, 10 899 F.3d 1281, 1290 (Fed. Cir. 2018); Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1353 11 (Fed. Cir. 2016) (Federal Circuit decisions “make clear that the two [Alice] stages involve 12 overlapping scrutiny of the content of the claims”); Enfish, 822 F.3d at 1339 (the “analysis of 13 whether there are arguably concrete improvements in the recited computer technology could take 14 place under” step one in some cases and step two in others). 15 NantWorks again claims improvement of the AR technology. As discussed in the previous a 16 || section, there are no specific improvements in computer technology. And the abstract ideas alone 17 || are not an inventive concept. BSG Tech., 899 F.3d at 1290 (“It has been clear since Alice that a Z 18 claimed invention’s use of the ineligible concept to which it is directed cannot supply the inventive 19 concept that renders the invention ‘significantly more’ than that ineligible concept.”). NantWorks 20 cites its expert, but there is nothing in the claims that is an inventive concept under Alice. 21 The court does not rely on Niantic’s expert (Polish) in reaching these conclusions. 22 24 The court grants Niantic’s summary judgment. In light of this holding, the court does not reach 25 the other pending motions. 27 Dated: July 9, 2024 LAUREL BEELER 28 United States Magistrate Judge