The Augmented World — Legal and Privacy Perspectives on Augmented Reality (AR)

Posted on by

You’ve likely heard that Augmented Reality (AR) is the next technology that will transform our lives. You may not realize that AR has been here for years. You’ve seen it on NFL broadcasts when the first down line and down/yardage appear on the screen under players’ feet. You’ve seen it in the Haunted Mansion ride in Disneyland when ghosts seem to appear in the mirror riding with you in your cart. You’ve seen it in cars and fighter jets when speed and other data is superimposed onto the windshield through a heads-up display. You’re seeing it in the explosion of Pokémon Go around the world. AR will affect all sectors, much as the World Wide Web did in the mid-1990s. Any new technology such as AR brings with it questions on how it fits under the umbrella of existing legal and privacy laws, where it pushes the boundaries and requires adjustments to the size and shape of the legal and regulatory umbrella, and when a new technology leads to a fundamental shift in certain areas of law. This article will define augmented reality and the augmented world, and analyze its impact on the legal and privacy landscape.

What is “augmented reality” and the “augmented world?”

One of the hallmarks of an emerging technology is that it is not easily defined. Similar to the “Internet of Things,” AR means different things to different people, can exist as a group of related technologies instead of a single technology, and is still developing. However, there are certain common elements among existing AR technologies from which a basic definition can be distilled.

I would define “augmented reality” as “a process, technology, or device that presents a user with real-world information, commonly but not limited to audiovisual imagery,Âaugmented with additional contextual data elements layered on top of the real-world information, by (1) collecting real-world audiovisual imagery, properties, and other data; (2) processing the real-world data via remote servers to identify elements, such as real-world objects, to augment with supplemental contextual data; and (3) presenting in real time supplemental contextual data overlaid on the real-world data.” The real world as augmented through various AR systems and platforms can be referred to as the “augmented world.” AR and the augmented world differs from “virtual reality” (VR) systems and platforms, such as the Oculus Rift and Google Cardboard, in that VR replaces the user’s view of the real world with a wholly digitally-created virtual world, where AR augments the user’s view of the real world with additional digital data.

“Passive” AR (what I call “first-generation AR”) is a fixed system — you receive augmented information but do not do so interactively, such as going through the Haunted Mansion ride or watching your television set. The next generation of AR is “active,” meaning that AR will be delivered in a changing environment, and the augmented world will be viewed, through a device you carry or wear. Google Glass and the forthcoming Microsoft HoloLens are examples of “active AR” systems with dedicated hardware; when worn, the world is augmented with digital data superimposed on the real-time view of the world. However, AR has found ways to use existing hardware — your smartphone. HP’s Aurasma platform is an early example of an active AR system that uses your smartphone’s camera and screen to create digital content superimposed on the real world. What AR has needed to go fully mainstream was a killer app that found a way for AR to appeal to the masses, and it now has one — Pokémon Go. Within days of its launch in early July, TechCrunch reported that Pokémon Go had an average daily user base of over 20 million users. Some declared it the biggest “stealth health” app of all time as it was getting users out and walking.

Active AR has the capacity to change how people interact with the world, and with each other. It is an immersive and engaging user experience. It has the capacity to change the worlds of shopping, education and training, law enforcement, maintenance, healthcare, and gaming, and others. Consider an AR system that shows reviews, product data, and comparative prices while looking at a shelf display; identifies an object or person approaching you and makes it glow, flash, or otherwise stand out to give you more time to avoid a collision; gives you information on an artist, or the ability to hear or see commentary, while looking at a painting or sculpture; identifies to a police officer in real time whether a weapon brandished by a suspect is real or fake; or shows you in real time how to repair a household item (or how to administer emergency aid) through images placed on that item or on a stricken individual. For some, the augmented world will be life-altering, such as a headset as assistive technology which reads road signs aloud to a blind person or announces that a vehicle is coming (and how far away it is) when the user looks in the vehicle’s direction. For others, the ability to collect, process and augment real-world data in real time could be viewed as a further invasion of privacy, or worse, technology that could be used for illegal or immoral purposes.

As with any new technology, there will be challenges from a legal and digital perspective. A well-known example of this is the Internet when the World Wide Web became mainstream in the mid-1990s. In some cases, existing laws were interpreted to apply to the online world, such as the application of libel and slander to online statements, the application of intellectual property laws to file sharing over peer-to-peer networks, and the application of contract law to online terms of use. In others, new laws such as the Digital Millennium Copyright Act were enacted to address shortcomings of the existing legal and regulatory landscape with respect to the online world. In some instances, the new technology led to a fundamental shift in a particular area of law, such as how privacy works in an online world and how to address online identity theft and breaches of personal information. AR’s collection of data, and presentation of augmented data in real time, creates similar challenges that will need to be addressed. Here are some of the legal and privacy challenges raised by AR.

  • Rethinking a “reasonable expectation of privacy.” A core privacy principle under US law is that persons have a reasonable expectation of privacy, i.e., a person can be held liable for unreasonably intruding on another’s interest in keeping his/her personal affairs private. However, what is a “reasonable expectation of privacy” in a GoPro world? CCTV/surveillance cameras, wearable cameras, and smart devices already collect more information about people than ever before. AR technology will continue this trend. As more and more information is collected, what keeping “personal affairs private” looks like will continue to evolve. If you know someone is wearing an AR device, and still do or say something you intend to keep private, do you still have a reasonable expectation of privacy?

What is a “reasonable expectation of privacy” in a GoPro world?

 

  • Existing Privacy Principles. Principles of notice, choice, and “privacy by design” apply to AR systems. Providers of AR systems must apply the same privacy principles to AR as they do to the collection of information through any other method. Users should be given notice of what information will be collected through the AR system, how long it will be kept, and how it will be used. Providers should collect only information needed for the business purpose, store and dispose of it securely, and keep it only as long as needed.

AR systems add an additional level of complexity — they are collecting information not just about the user, but also third parties. Unlike a cellphone camera, where the act of collecting information from third parties is initiated by the user, an AR system may collect information about third parties as part of its fundamental design. Privacy options for third parties should be an important consideration in, and element of, any AR system. For example, an AR system provider could ensure users have the ability to toggle the blocking of third party personal data from being collected or augmented, so personal information is only augmented when the user wants it to be. AR system providers may also consider an indicator on the outside of the device, such as an LED, to let third parties know that the AR system is actively collecting information.

Additionally, AR may create interesting issues from a free speech and recording of communications perspective. Some, but not all, court rulings have held that the freedom of speech guaranteed by the First Amendment extends to making recordings of matters of public interest. An AR system that is always collecting data will push the boundaries of this doctrine. Even if something is not in the public interest, many states require the consent of both parties to record a conversation between them. An AR system which persistently collects data, including conversations, may run afoul of these laws.

  • Children’s Privacy.ÂIt is worth a special note that AR creates an especially difficult challenge for children’s privacy, especially children under 13. The Children’s Online Privacy Protection Act (“COPPA”) requires operators of online services, including mobile apps, to obtain verifiable parental consent before collecting any personal information from children under 13. “Personal information” includes photos, videos, and audio of a child’s image or voice. As AR systems collect and process data in real time, the passive collection of a child’s image or voice (versus collection of children’s personal information provided to a company through an interface such as a web browser) is problematic under COPPA. AR operators will need to determine how to ensure they are not collecting personal information from children under 13. I expect the FTC will amend the COPPA FAQ to clarify their position on the intersection of AR and children’s privacy.
  • Intellectual Property. Aside from the inevitable patent wars that will occur over the early inventors of AR technologies, and patent holders who believe their patent claims cover certain aspects of AR technologies, AR will create some potentially interesting issues under intellectual property law. For example, an AR system that records (and stores) everything it sees will invariably capture some things that are protected by copyright or other IP laws. Will “fair use” be expanded in the augmented world, e.g., where an album cover is displayed to a user when a song from that is heard? Further, adding content to a copyrighted work in the augmented world may constitute a prohibited derivative work. From a trademark perspective, augmenting a common-law or registered trademark with additional data, or using a competitor’s name or logo to trigger an ad about your product overlaid on the competitor’s name or logo, could create issues under existing trademark law.
  • Discrimination.  AR systems make it easy to supplement real-world information by providing additional detail on a person, place or thing in real time. This supplemental data could intentionally or inadvertently be used to make real-time discriminatory decisions, e.g., using facial or name recognition to provide supplemental data about a person’s arrest history, status in a protected class, or other restricted information which is used in a hiring or rental decision. An AR system that may be used in a situation where data must be excluded from the decision-making process must include the ability to automatically exclude groups of data from the user’s augmented world.

The world of online digital marketing and advertising will expand to include digital marketing and advertising in the augmented world. Imagine a world where anything — and I mean anything — can be turned into a billboard or advertisement in real time. Contextual ads in the augmented world can be superimposed anytime a user sees a keyword. For example, if you see a house, imagine if an ad for a brand of paint appears because the paint manufacturer has bought contextual augmented ads to appear in an AR system whenever the user sees a house through the augmented world.

Existing laws will need to be applied to digital marketing and advertising in the augmented world. For example, when a marketing disclaimer appears in the online world, the user’s attention is on the ad. Will the disclaimer have the same effect in an augmented environment, or will it need to be presented in a way that calls attention to it? Could this have the unintended consequence of shifting the user’s attention away from something they are doing, such as walking, thereby increasing the risk of harm? There are also some interesting theoretical advertising applications of AR in a negative context. For example, “negative advertising” could be used to blur product or brand names and/or to make others more prominent in the augmented world.

  • The Right of Publicity.  The right of publicity — a person’s right to control the commercial use of his or her name, image, and likeness — is also likely to be challenged by digital marketing in the augmented world. Instead of actively using a person’s likeness to promote a product or service, a product or service could appear as augmented data next to a person’s name or likeness, improperly (and perhaps inadvertently) implying an endorsement or association. State laws governing the right of publicity will be reinterpreted when applied to the augmented world.
  • Negligence and Torts. AR has the capacity to both further exacerbate the problem of “distracted everything,” paying more attention to your AR device than your surroundings, as some users of Pokémon Go have discovered. Since AR augments the real world in real time, the additional information may cause a user to be distracted, or if the augmented data is erroneous could cause a user to cause harm to him/herself or to others. Many have heard the stories of a person dutifully following their GPS navigation system into a lake. Imagine an AR system identifying a mushroom as safe to eat when in fact it is highly poisonous. Just as distracted driving and distracted texting can be used as evidence of negligence, a distracted AR user can find him/herself facing a negligence claim for causing third party harm. Similarly, many tort claims that can arise through actions in the real world or online world, such as liable and slander, can occur in the augmented world. Additionally, if an AR system augments the real world in a way that makes someone think they are in danger, inflicts emotional distress, or causes something to become dangerous, the AR user, or system provider, could be legally responsible.
  • Contract liability. We will undoubtedly see providers of AR systems and platforms sued for damages suffered by their users. AR providers have and will shift liability to the user through contract terms. For example, Niantic, the company behind Pokémon Go, states in their Terms of Use that you must “be aware of your surroundings and play safely. You agree that your use of the App and play of the game is at your own risk, and it is your responsibility to maintain such health, liability, hazard, personal injury, medical, life, and other insurance policies as you deem reasonably necessary for any injuries that you may incur while using the Services.” AR providers’ success at shifting liability will likely fall primarily to tried-and-tested principles such as whether an enforceable contract exists.

None of the above challenges are likely to prove insurmountable and are not expected to slow the significant growth of AR. What will be interesting to watch is how lawmakers choose to respond to AR, and how early hiccups are seized on by politicians and reported in the press. Consider automobile autopilot technology. The recent crash of a Tesla in Autopilot mode is providing bad press for Tesla, and fodder for those who believe the technology is dangerous and must be curtailed. Every new technology brings both benefits and potential risks. If the benefits outweigh the risks on the whole, the public interest is not served when the legal, regulatory and privacy pendulum swings too far in response. Creating a legal, regulatory and privacy landscape that fosters the growth of AR, while appropriately addressing the risks AR creates and exacerbates, is critical.