Apple is reportedly taking its innovative gesture controls, originally introduced with the Vision Pro headset, to the next level by planning to implement them across its other devices, including iPhones, iPads, and Macs. The Vision Pro revolutionized interaction with apps, enabling users to perform actions like moving windows and resizing documents with intuitive hand gestures, making them feel natural and seamless. This success has seemingly inspired Apple to bring this technology to a broader audience.
A recent patent application, titled “Devices, Methods, And Graphical User Interfaces For Using A Cursor To Interact With Three-Dimensional Environments,” provides a glimpse into Apple’s ambitious vision. This filing suggests that Apple envisions a future where these gesture controls become a universal control method across its diverse product lineup. While Apple has explored similar ideas in the past, like a 2009 patent, the success of the Vision Pro gestures has seemingly breathed new life into these endeavors.
The patent includes numerous illustrative drawings and diagrams, many of which focus on the Vision Pro. These drawings provide a detailed breakdown of the headset’s components, revealing how it detects user gestures and transmits these signals to a Mac or another device. Notably, the drawings also showcase users interacting with an iPad or a traditional desktop computer, both without the need for a headset. This indicates that Apple is actively exploring ways to implement gesture controls on devices without relying on the specialized hardware of the Vision Pro.
Apple’s patent application describes these devices as providing “computer-generated experiences,” encompassing virtual reality and mixed reality via a display. This suggests that gesture controls could seamlessly integrate into existing devices equipped with Face ID sensors, which are already used for facial authentication. It’s not a far leap to imagine these sensors recognizing hand movements to control device functions.
Currently, Face ID sensors are utilized for more than just unlocking devices. In iOS 18, these sensors enable users to navigate their iPhones with just their gaze. The new patent application focuses on accurately recognizing gestures and dismissing false positives, such as accidental hand movements like scratching one’s nose. The system is designed to evaluate whether a hand movement meets specific criteria to trigger an action, like moving a cursor or adjusting a window’s position. If the criteria aren’t met, the system ignores the gesture.
While the patent primarily details the technical process of gesture recognition and its resulting actions, it hints at a broader vision for Apple’s devices. The company has traditionally resisted adding touchscreens to Macs, citing ergonomic concerns about users reaching for the screen. By introducing gesture controls, Apple could offer an alternative that prioritizes user comfort while enhancing device interaction.
This patent application, attributed to inventors including Evgenii Krivoruchko, highlights Apple’s unwavering dedication to enhancing user experiences through innovative technology. Krivoruchko has multiple patents related to utilizing user attention to control elements in 3D environments. As Apple continues to push the boundaries of device interaction, users may soon find themselves waving their hands to command their iPhones, iPads, and Macs, transforming touch-free control into a reality.
