What is spatial computing?

Spatial computing is an umbrella concept used to describe the interaction between digital technologies and objects/space in the real world. It includes technologies such as augmented reality (AR), mixed reality (MR), virtual reality (VR), the IoT and digital twins. Spatial computing is already being used, for example, in self-driving vehicles and autonomous machines.

Commentators predict that spatial computing will soon become part of everyday life due to advancements in artificial intelligence and spatial technology, as well as the growing accessibility of products and services.

Possible uses include overlaying virtual repair manuals onto physical products to help guide the repair process, creating spatial models of manufacturing processes that can be manipulated to optimize those processes, immersing video gamers in virtual gaming worlds, training employees in virtual environments and creating virtual real estate visits, shopping experiences and design processes.

Spatial computing often requires a device, like a phone, as required for Pokémon Go and Snapchat filters, or a headset, as used for the Microsoft Hololens and Apple Vision Pro. Increasingly, haptic technology such as gloves and bodysuits is being used to detect the wearer's sound and movements and deliver physical sensations in response.

The IP issues relating to spatial computing

The IP issues engendered by spatial computing are largely two-fold: first, does the use of spatial computing infringe any third party IP rights and, secondly, how can the unique experiences and outputs of spatial computing technologies be protected under IP laws? We consider these issues here.

Use of third party IP in spatial computing

Copyright

Spatial computing often involves the copying (reproduction) of copyrighted material. For this article, we ignore any copying that might occur in the context of the training (and outputs) of any underlying AI since this has already been written about extensively. Instead, we focus on copying in the context of the experience and outputs of the spatial technology itself (although the issues are closely similar). Copying in this context can be intentional or unintentional; temporary or permanent.

Any reproduction (copying) of a copyright-protected work without the owner's permission usually constitutes infringement unless an exception applies. It should not matter whether or not:

• The works are stored in their original forms (eg as a recognisable drawing) or in digital form (ie some form of electronic code) - copying includes storing the work in any medium by electronic means.
• Copying is direct or indirect or whether intervening acts infringe.
• The whole of a copyright work has been copied - there is also infringement where a substantial part (measured qualitatively not quantitatively) is copied.

It is easy to envisage how there might be a number of disputes in the coming years about whether copyright subsists in third party works, whether a "substantial" part has been taken and whether there has been reproduction in the sense required. Likewise, we can expect much discussion and potentially also disputes about whether there is and should be infringement where the technology 'crawls over' copyright works (or ‘hyperlinks’ to them), without actually copying them. Under existing UK and EU laws, this would not in all cases infringe the reproduction right. While we might obtain some guidance on these issues from legal cases in other areas (such as the Stable Diffusion and Stability AI copyright infringement claims around generative AI) they are unlikely to fully answer all of the questions relevant to spatial computing.

There is also likely to be debate about whether other forms of infringement apply eg whether some spatial computing experiences constitute a new form of communication to the public.

There is also likely to be debate about whether any exceptions/defences to infringement do (or should) apply. Possible exceptions of relevance include the temporary copies, incidental inclusion and fair dealing exceptions in the UK, the de minimis exception in the Netherlands, the text and data mining exception in the EU (which is wider than in the UK, extending to commercial as well as non-commercial situations) and the fair use defence in the US (although the application of that defence has recently been cut back by the US Supreme Court).

While these are largely untested in the context of spatial computing and analogous situations, they are unlikely to help in the vast majority of cases where, for example, a third party work has been copied to form an integral part of the spatial computing output or experience.

If an exception does not apply, developers/service providers will need to obtain permission from, or collaborate with, the owners of copyright-protected works. This is likely to mean negotiating new agreements or reviewing and renegotiating existing agreements (since the use of content for spatial computing might not fall within the scope of existing permissions). This is likely to be difficult.

Those supplying the works may require significant remuneration and/or part ownership of (or at least control over the use of) any new or derivative works. As a result, we are likely to see a number of interesting collaborations – as well as disputes - in the coming years.

Trade marks

A service provider might display brand names and logos as part of its spatial computing experience. This use could be intended (eg where a particular piece of branded machinery is reproduced) or unintended (eg brands in the background of a street view).

Relatively recent reforms to trade mark laws in the EU (and UK) mean that a wider array of signs can now potentially be protected as registered trade marks. This is due to the move away from requiring a "graphical representation" of a trade mark and towards an open-minded approach to ways of describing the "clear and precise subject matter of the protection". By way of example, the shape of various cars, buildings and public sculptures as well as a 22-second multi-media clip are all the subject of registered trade mark protection in the UK/EU.

Although trade mark owners have the right to control the use of their marks, use of a trade mark without permission does not automatically amount to infringement in the UK and EU. Much depends on the nature of the use in question. Furthermore, use must "in the course of trade", "in relation to goods/services" and result in one of the specified types of injury to the functions of a trade mark. There are also various defences to infringement including the use of a mark to identify goods/services as coming from a particular proprietor and various descriptive uses.

Use of a third party mark on a branded handbag in a virtual environment such as a video game is likely to infringe but other uses (if more incidental) might not. Again, there is little to no case law on this yet.

Whilst trade mark law has been largely harmonised across the EU (and UK), there remain differences of approach. For example, Benelux trade mark law also protects against use of a mark “other than in relation to goods/services” where that use is detrimental to or takes unfair advantage of the reputation of the mark. Defamatory use of a mark in a spatial computing environment could fall under this ground for infringement.

It is not just registered trade marks that are relevant. In the UK, the law protects goodwill generated through use of a brand in the form of passing off. False endorsement can also constitute passing off. In the EU, the law of unfair competition might also protect certain uses of unregistered brands.

It is therefore important that those who use or might use third party brands (whether registered or unregistered) in the context of spatial computing tread carefully and obtain specialist advice as to whether permissions from brand owners is required. This includes use of non-traditional trade marks (such as multi media marks, shapes and colours) as well as traditional marks (such as brand names and logos).

Designs

Design law protects the appearance of the whole or part of a product resulting from the features of, in particular, the lines, contours, colours, shape, texture and/or materials of the product itself and/or its ornamentation. To be valid, the design must be new, have individual character and not otherwise be excluded from protection.

The definition of a design is broad enough to cover packaging and logos, as well as the designs of 2D and 3D products. Protection is given for the design itself. This means that the design is protected for all products not just the product in which it is incorporated.

The owner of a registered design has the right to use the design and any other design which does not produce a different overall impression on the informed user. Acts such as using a product incorporating the design constitute infringement. Products are largely thought to mean real world products and computer programs are specifically excluded.

However, current proposals to amend the EU design law would see the definition of a design updated to include "the movement, transition or any other sort of animation of" the features of the appearance of the whole or part of a product. In addition, the definition of product will be expanded to include any industrial or handicraft item other than a computer program "regardless of whether it is embodied in a physical object or materialises in a digital form" (see here).

If and when these amendments are made, there will be greater scope for designs to subsist in virtual worlds and on virtual products and therefore greater scope for infringement through spatial computing technologies.

Since designs can be registered and unregistered, special care must be taken to avoid infringement.

Protecting IP rights in spatial computing

The underlying technology used in spatial computing may be protected through IP regimes such as patent and design laws. Furthermore, the experiences and outputs of the technology itself, such as the graphics, visual designs, words, sound effects and music, may also be protectable through copyright, trade mark, and design laws.

For copyright to subsist in a literary, music or artistic work, it must be original. Under the current EU test for originality (authors own intellectual creation), it is likely that many aspects of the spatial computing experience and outputs with be protected by copyright. Moreover, UK copyright law specifically recognises "computer-generated works".

The bigger issue might be who owns such works especially where they are computer generated. Service providers will need to ensure that they clarify contractually who owns copyright (if any) in spatial computing experiences and outputs.

Spatial computing experiences and outputs might also be protected under the law of designs in the EU if and when the new legislation mentioned above comes into force.

Lastly, any brands used in the spatial computing environment will be capable of protection in the usual way. This includes non-traditional signs such as motion and multimedia marks as described above, provided they are distinctive (or have acquired distinctiveness over time). A question that immediately comes up is how to describe the goods and services for such a mark, which question has also arisen regarding marks used for NFTs and in the metaverse (see here). Further guidance might be required from IP offices when it comes to registering marks in the context of spatial computing.

New forms of IP rights for "feel"

Some commentators argue that there is a need for new forms of rights to ensure that the investment in particular environments, whether real or virtual, is better protected. Huge sums of money are invested into differentiating real world environments (such as shops and buildings) and the same is true of virtual environments, too. In addition, spatial computing increasingly uses haptic technologies and relies on the user's sense of touch or movement.

Current forms of IP protection are ill-suited to protect those investments or any sort of immersive experience, other than sound. The protection of smells and tastes as trade marks is still in its infancy some 20 years after their introduction, and the overall impression of a place is still generally regarded as too indefinable and subjective to be the subject of serious legal rights.

Some might argue that the time is ripe to extend the boundaries of IP protection to cover immersive experiences. The difficulty has been how to define a "feel" sensation, smell or taste and a good way of doing so has not yet been found. This is clearly in the minds of those in the industry. The EUIPO's recently updated Tech Watch Discussion Paper states that "all the [spatial computing] technologies raise questions about the protectability of innovation and creativity related to the technologies themselves."

What does this all mean?

Even though spatial computing could be seen as just another umbrella term for already existing technologies that come with known IP challenges, there are many new questions popping up.

Registering trade marks relating to spatial computing will probably need guidance from IPO’s in the very near future. And the question of whether any motion or multimedia marks for use in spatial computing devices are already distinctive – or can only be registered when acquired distinctiveness is proven - will be an interesting point to watch out for.

We expect to see mainly copyright enforcement matters in the short and medium term given the obvious use of copyright protected content in spatial computing environments, with issues such as what infringing acts are taking place, whether there has been reproduction in the sense required, how the exceptions apply and disputes about licensing being front and centre.