While Apple hasn't publicly admitted to working on a virtual reality (VR) headset, it keeps filing patents relating to such a device. Apple CEO Tim Cook has been very vocal about how important a similar technology, augmented reality (AR), is to the future. AR goes a step beyond VR, allowing a pass-through view of the real world. When using VR or seeing the world through a busy AR viewport, there is an inherent risk of crashing into furniture or other physical objects that might be blocked by computer-generated images. A recent Apple patent addresses this problem with a solution that is different from current VR headsets.
Apple expressed an interest in AR in 2017, launching a developer's interface called ARKit. This was several years behind Google's early work in the field, but Apple immediately gained a leading position by enabling the use of this technology in a large number of its mobile devices. iPhone and iPad models dating back to 2015 have had access to this capability and, at present, Apple claims that it has hundreds of millions of AR-enabled devices. While VR has its own challenges, it is closely related and a full-featured AR headset would include VR as a subset of its capabilities.
A recent Apple patent application reveals that a future Apple VR headset might detect real-world obstacles and alert the wearer so they can avoid colliding with TV stands, sofas, or other nearby objects. There are two parts to this process, first sensing objects that are too close to the wearer and then communicating the danger within the VR world, ideally in such a way that it doesn't interfere with the use and enjoyment of the headset. Detecting the distance to objects is a simple matter for Apple given its history of depth-sensing technology found in Face ID, portrait mode photos, and LiDAR scanners. Any of these methods might be used for a VR headset to identify potential dangers to a person that has their vision obscured by the virtual world that the headset generates.
Apple VR Collision Alerts
Apple's patent application about a VR headset's collision prevention is more advanced than that of the leading VR headset, Facebook's Oculus Quest. The Quest asks the user to draw a virtual safe area upon the floor while showing a low-resolution AR view of the room and reveals this boundary as a graphical cage when the wearer approaches the edge. While this does help, a two-dimensional outline doesn't account for objects with protrusions at different heights, such as floor lamps and wall-mounted TVs. The Quest also depends on the user drawing accurately and taking into consideration arm reach. The depth-sensing accuracy of a future Apple VR headset can't be compared, but the suggestion is that it might take into account actual objects rather than relying upon a predefined outline of a safe area. Apple's solution to alerting the wearer to a potential collision is even more interesting, describing an advanced blending of virtual and real-world scenes and sounds.
Apple's cross-dissolve to the real world is described as possibly matching the perspective of the actual room and having certain graphic elements fading faster than others, such as walls, floors, and ceiling disappearing more rapidly than game characters and virtual objects. The later of which might appear behind objects within the real environment, as if elements of the two realities were blending. The patent document indicates that the audio of the virtual world might also cross-fade to real-world audio as well. Acoustic properties are also mentioned as a blending of virtual and actual environmental sounds during the transition. This would provide a more cinema-like experience as if traveling through a portal than the abrupt change that would come from a hard cut or a simple fade, a good example of the care that goes into Apple's planning. This merging of worlds is described as working in both directions as well, meaning a transition while entering as well as exiting VR and a partial shift when in danger of crashing with physical objects. While this is only a patent application, rumors and leaks suggest an Apple VR headset may be coming as soon as next year and this is a glimpse at the sort of advanced work being put into researching such a product.
Source: USPTO