In Re PUBLICOVER

• Court: Court of Appeals for the Federal Circuit
• Decided: May 15, 2020
• Docket: 19-1883
• Status: Unpublished

Opinion

Case: 19-1883 Document: 46 Page: 1 Filed: 05/15/2020

NOTE: This disposition is nonprecedential.

United States Court of Appeals for the Federal Circuit ______________________

IN RE: NELSON GEORGE PUBLICOVER, LEWIS JAMES MARGGRAFF, ELIOT FRANCIS DRAKE, SPENCER JAMES CONNAUGHTON, Appellants ______________________

2019-1883 ______________________

Appeal from the United States Patent and Trademark Office, Patent Trial and Appeal Board in No. 15/131,273. ______________________

Decided: May 15, 2020 ______________________

THERESA H. NGUYEN, Perkins Coie, LLP, Seattle, WA, argued for appellants. Also represented by DAN L. BAGATELL, Hanover, NH.

MAUREEN DONOVAN QUELER, Office of the Solicitor, United States Patent and Trademark Office, Alexandria, VA, argued for appellee Andrei Iancu. Also represented by THOMAS W. KRAUSE, BENJAMIN T. HICKMAN, MEREDITH HOPE SCHOENFELD. ______________________

Before O’MALLEY, BRYSON, and CHEN, Circuit Judges. CHEN, Circuit Judge. Case: 19-1883 Document: 46 Page: 2 Filed: 05/15/2020

2 IN RE: PUBLICOVER

Appellants Nelson George Publicover, Lewis James Marggraff, Eliot Francis Drake, and Spencer James Con- naughton (collectively, Publicover) appeal the determina- tion of the Patent Trial and Appeal Board that claims 43– 51 of their patent application, No. 15/131,273 (the ’273 Ap- plication), would have been obvious under 35 U.S.C. § 103(a). See Ex Parte Publicover et al., No. 2018–005362, 2019 WL 1453980 (P.T.A.B. Mar. 8, 2019). For the reasons explained below, we affirm the Board’s decision. BACKGROUND I. The ’273 Application The ’273 Application relates generally to methods of tracking eye movements in a graphical user interface envi- ronment such as wearable virtual- or augmented-reality display devices, using those movements to determine the user’s intent, and performing an action based on that in- tent. ’273 Application ¶¶ 17–21. The eye-tracking technol- ogy thereby enables the user to interact with and control the device with his or her eye movements, rather than us- ing traditional tools such as a computer mouse, joystick, or touch-sensitive displays. Id. ¶¶ 4–6. According to Pub- licover, the purported invention seeks to solve certain unique challenges associated with discerning user intent based on eye movements. Appellant’s Br. at 4 (citing ’273 Application ¶ 6). One such challenge, and stated goal of the purported invention, is to accurately discern user intent by distin- guishing the user’s involuntary movements from voluntary movements intended to interact with, and control, the de- vice. ’273 Application ¶¶ 6, 87. Two types of “[v]oluntary movements that may [be used to] convey purposeful intent” are saccadic eye movements and voluntary head move- ments that produce vestibulo-ocular eye movements. Id. ¶¶ 89, 115. Case: 19-1883 Document: 46 Page: 3 Filed: 05/15/2020

IN RE: PUBLICOVER 3

The appealed claims rely on identifying vestibulo-ocu- lar eye movements to discern user intent. 1 Id. ¶¶ 89, 98, 179. Vestibulo-ocular eye movements are “generally ac- cepted” as one of “four fundamental types of eye move- ments.” Id. ¶ 115. In order for a person’s eyes to remain focused on a particular target while his or her head is mov- ing, vestibulo-ocular eye movements “compensate for head movements by” rapidly and reflexively “moving the eye[s] through the same angle as a head rotation, but in the op- posite direction.” Id. ¶¶ 116, 222. The eye and head move- ments “correspond” “approximate[ly] . . . in movement magnitudes.” Id. ¶ 224, Fig. 4. “This has the effect of” keeping the eyes on a fixation point and “stabilizing an im- age of the external world.” Id. ¶ 116. Although the head movements that produce the vestibulo-ocular movement are generally voluntary, “the vestibulo-ocular [eye] move- ment itself is involuntary.” Id. ¶ 182. Vestibulo-ocular movements can occur in conjunction with another fundamental type of eye movement, saccadic eye movements. “Saccadic eye movements are rapid move- ments that abruptly change the fixation point of the eye[s].” Id. ¶ 119; see also id. ¶¶ 217–18. After a person performs a saccadic eye movement, he or she may naturally rotate the head to align it with the new fixation point of the eyes. As the specification explains, “any shift in gaze greater than about 20° . . . is usually associated with a head movement.” Id. ¶ 179. If the eyes were to remain still as the head moves, the head would carry the eyes off target. The involuntary vestibulo-ocular reflex of the eyes keeps the eyes on target by moving the eyes in the opposite direc- tion of the head rotation. Id. ¶ 116.

1 The Board affirmed the examiner’s rejection of all pending claims, claims 31–39, 41–56, and 58–61, but Pub- licover’s appeal is limited to claims 43–51. Case: 19-1883 Document: 46 Page: 4 Filed: 05/15/2020

4 IN RE: PUBLICOVER

II. Procedural History The examiner rejected claims 43 and 46–51 of the ’273 Application as obvious over the combination of U.S. 2014/0380230 A1 (Venable) and U.S. 2012/0081666 A1 (Ki- derman). J.A. 1147–50. Additionally, the examiner re- jected claims 44–45 as obvious over the combination of Venable, Kiderman, and U.S. 6,932,090 (Reschke). J.A. 1161–63. Publicover appealed to the Board, which af- firmed the examiner’s rejections. On appeal before us, Publicover directs its argument solely to independent claim 43, from which claims 44–51 depend. Claim 43 recites a method for using a graphical user interface to determine a user’s intent by detecting when there has been a vestibulo-ocular eye movement as- sociated with a head movement, and then performing an action related to a viewed object or its location: 43. A method for providing a graphical user inter- face to determine intent of a user based at least in part on movement of the user’s head and one or both of the user’s eyes using a head movement de- tector and an eye movement detector, comprising: identifying, with the head movement detec- tor, when the user’s head moves at a head velocity; identifying, with the eye movement detec- tor, when the one or both of the user’s eyes move at an eye velocity; identifying, based at least in part on the head velocity and the eye velocity occurring concurrently, a vestibulo-ocular movement of the one or both of the user’s eyes; confirming that the one or both of the user’s eyes are directed at a viewed object at a viewed object location based at least in part Case: 19-1883 Document: 46 Page: 5 Filed: 05/15/2020

IN RE: PUBLICOVER 5

on the determined vestibulo-ocular move- ment of the one or both of the user’s eyes; and performing, in response to the confirming, an action related to one or more of the viewed object and the viewed object loca- tion. ’273 Application at claim 43 (emphases added). We have jurisdiction pursuant to 28 U.S.C. § 1295(a)(4). DISCUSSION Obviousness is a question of law based on underlying factual findings. Randall Mfg. v. Rea, 733 F.3d 1355, 1362 (Fed. Cir. 2013). The factual findings include, inter alia, “the scope and content of the prior art” and “differences be- tween the prior art and the claims at issue.” Id. (cit- ing KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 406 (2007)). This court reviews the Board’s legal determinations de novo, and the Board’s factual findings underlying those de- terminations for substantial evidence. Belden Inc. v. Berk- Tek LLC, 805 F.3d 1064, 1073 (Fed. Cir. 2015). A finding is supported by substantial evidence if a reasonable mind might accept the evidence to support the finding. Consol. Edison Co. v. NLRB, 305 U.S. 197, 229 (1938). Publicover argues that the Board erred in finding that the prior art teaches the claim limitation of “identifying . . . a vestibulo-ocular movement.” Publicover contends that neither Kiderman alone nor the combination of Venable and Kiderman teach this limitation. Appellant’s Br. at 17. We disagree because substantial evidence supports the Board’s finding that the combination of Venable and Kider- man would result in performing the “identifying . . . a ves- tibulo-ocular movement” limitation. Case: 19-1883 Document: 46 Page: 6 Filed: 05/15/2020

6 IN RE: PUBLICOVER

As the Board explained, Venable “teaches selecting user interface elements via position signals derived from sequential eye gestures that may include saccades.” Pub- licover, 2019 WL 1453980, at *5 (citing Venable ¶ 26). Ve- nable is directed to “an eye tracking system . . . configured to produce a periodically updated position signal based upon a determined location of a user’s gaze on a . . . user interface” element. Venable ¶ 19. Venable discloses that some embodiments rely on identifying saccadic eye move- ments to determine the user’s intent to select the user in- terface element. Id. ¶ 26. Once Venable’s system “determin[es] that the user has selected the U[ser ]I[nter- face] element,” it will perform an action such as, for exam- ple, “launching a program represented by the element.” Id. ¶ 37. As shown in Venable’s figures, the eye tracking system is integrated with “a head-mounted display . . . configured to display augmented reality images.” Id. ¶ 15; see also id. Figs. 1A–C, 4A–C, 5A–D, 6. Venable states that its system may comprise not only an eye tracking system but also “may include . . . motion sensors . . . and/or . . . image sen- sors” that may be “incorporated into [the] head-mounted display.” Id. ¶¶ 19–20. Venable contemplates using sev- eral different types of components alone or together, in- cluding “a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition.” Id. ¶ 51. Based on Venable’s disclosure of the “head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition,” the Board and examiner reasonably found that Venable teaches concur- rently tracking eye movements and head movements, and is not limited “to head-fixed saccades” for discerning user intent. J.A. 1501; Publicover, 2019 WL 1453980, at *8. We disagree with Publicover’s arguments to the contrary. Kiderman is directed to a head mounted goggle based video oculography system for recording and evaluating sac- cadic eye movements “for assisting in the diagnosis of” Case: 19-1883 Document: 46 Page: 7 Filed: 05/15/2020

IN RE: PUBLICOVER 7

certain brain injuries and disorders “that cause highly sac- cadic results.” Kiderman ¶¶ 3, 5, 18. Importantly, with regard to vestibulo-ocular movements, Kiderman discloses that they occur in conjunction with saccadic eye move- ments. Id. ¶ 12. In the background of the invention, Ki- derman explains that “[d]uring . . . saccades” with “shift[s] in gaze larger than about 20°,” “first the eye produces a saccade to get gaze on target, whereas the head follows more slowly and the vestibulo-ocular reflex causes the eyes to roll back in the head to keep gaze on the target.” Id. Kiderman further explains that “[t]he vestibular system stabilizes vision with head movement . . . , sensing the di- rection and speed of head acceleration and moving the eyes accordingly.” Id. ¶ 7. The examiner reasoned that the basis to combine Ki- derman’s disclosure of vestibulo-ocular movement with Ve- nable’s system for tracking saccadic eye movements to “identify[]. . . a vestibulo-ocular movement” was to “im- prove[] on the accuracy of” Venable’s system “by consider- ing the effect of head movement on the saccade.” J.A. 1501–02; see also J.A. 1141. This is particularly so given Kiderman’s teaching that vestibulo-ocular eye move- ments are commonly associated with and naturally follow from saccadic eye movements and Venable’s disclosure of embodiments that already track both eye and head move- ments. Kiderman ¶ 12; Venable ¶ 51. Before the Board, Publicover argued that Kiderman taken individually does not teach the “identifying . . . a ves- tibulo-ocular movement” limitation. J.A. 1477–78; J.A. 1588. The Board rejected that argument, noting that “Kiderman only is relied upon to teach that vestibulo-ocu- lar movement, such as that recited in claim 43, is one known type of head movement.” Publicover, 2019 WL 1453980, at *9 (citing Kiderman ¶ 12). The Board found that “the eye tracking system of Venable may be combined with . . . Kiderman[’s]” disclosure of vestibulo-ocular move- ment. Id. Like the examiner, whose Answer the Board Case: 19-1883 Document: 46 Page: 8 Filed: 05/15/2020

8 IN RE: PUBLICOVER

adopted as its own, the Board found that the accuracy of Venable’s system which tracks both eye and head move- ments would be improved by identifying vestibulo-ocular movements in addition to saccadic eye movements. Pub- licover, 2019 WL 1453980, at *3, *9. Substantial evidence supports this finding, despite Publicover’s arguments to the contrary. Before this court, Publicover presents additional argu- ments that were not previously presented to the Board. For example, Publicover contends that neither reference, taken individually or together, disclose “identifying” that “the head velocity and eye velocity occur[] concurrently” during “a vestibulo-ocular movement,” as the claims re- quire. ’273 Application at claim 43 (emphasis added); Ap- pellant’s Reply Br. at 5–6. Because Publicover did not raise these arguments to the Board, we conclude that they are waived. See In re Watts, 354 F.3d 1362, 1368 (Fed. Cir. 2004) (“declin[ing] to consider the appellant’s new argu- ment regarding the scope of [the prior art] raised for the first time on appeal”). Moreover, contrary to Publicover’s argument, Kiderman teaches that the head velocity and eye velocity occur concurrently during vestibulo-ocular movement, as the claims require. Kiderman discloses that when the head moves, “the vestibulo-ocular reflex causes the eyes to” move “to keep the gaze on the target.” Kider- man ¶ 12; see also id. ¶ 7 (“The vestibular system stabilizes vision with head movement,” as opposed to allowing vision to “shift” “during head movement,” by “sensing the direc- tion and speed of head acceleration and moving the eyes accordingly.”). In order “to keep the gaze on the target,” as Kiderman discloses, the head and eyes must be moving concurrently in opposite directions with roughly the same movement magnitudes. 2

2 Kiderman “incorporated . . . by reference” certain related patent application publications “in their entirety”: Case: 19-1883 Document: 46 Page: 9 Filed: 05/15/2020

IN RE: PUBLICOVER 9

We also note the examiner’s finding, which was adopted by the Board, that Publicover’s written description with respect to the “identifying . . . a vestibulo-ocular move- ment” limitation is rather thin in detail as to how the ap- plicant’s system identifies vestibulo-ocular movement. J.A. 1501; Publicover, 2019 WL 1453980, at *3, *8. The exam- iner explained that the limitation “is merely supported” by “paragraph [224]” of Publicover’s specification, which pro- vides the “well-known” definition of vestibulo-ocular move- ment. J.A. 1501. Paragraph 224 of Publicover’s specification states: If the right and left eye move in the same direction . . . and this direction is opposite the direction of the head . . . with an approximate correspondence in movement magnitudes, then the eye movement is classified as vestibulo-ocular . . . . ’273 Application ¶ 224; see also id. ¶ 116 (providing a simi- lar description of how vestibulo-ocular movement occurs). Publicover argues that Venable and Kiderman’s disclo- sures are too sparse to adequately explain to a skilled arti- san how to modify Venable’s system to identify vestibulo- ocular movement. But as the examiner and Board correctly found, Publicover’s specification is just as sparse on how a system would identify this type of eye movement. Under the circumstances, we find this attorney argument as to the capabilities of a skilled artisan unpersuasive. See Uber Techn., Inc. v. X One, Inc., No. 19-1164, 2020 WL 2123399, at *5 (Fed. Cir. May 5, 2020) (“The specification of the [pa- tent-at-issue] is entirely silent on how to transmit user

U.S. 2005/0099601, U.S. 2007/0132841, U.S. 2008/0049186, U.S. 2008/0049187. Kiderman ¶ 22. The fact that these publications are directed to systems that concurrently track head movements and eye movements lends additional support to this reading of Kiderman. Case: 19-1883 Document: 46 Page: 10 Filed: 05/15/2020

10 IN RE: PUBLICOVER

locations and maps from a server to a user’s mobile device, suggesting that a person of ordinary skill in the art was more than capable of selecting between the known methods of accomplishing this.”); In re Epstein, 32 F.3d 1559, 1568 (Fed. Cir. 1994) (“[T]he Board’s observation that appellant did not provide the type of detail in his specification that he now argues is necessary in prior art references supports the Board’s finding that one skilled in the art would have known how to implement the features of the references and would have concluded that the reference disclosures would have been enabling.”). Accordingly, the Board did not err in determining that the combination of Venable and Kider- man would result in the “identifying . . . a vestibulo-ocular movement” limitation. CONCLUSION We have considered Publicover’s remaining arguments and find them unpersuasive. For the foregoing reasons, we affirm the decision of the Board. AFFIRMED COSTS No costs.