Giving another boost to Android's accessibility tricks, Google has introduced a new feature that allows users to control their phone using facial expressions, such as smiling, raising eyebrows, and eye movement. Android already offers a ton of accessibility features like voice access for touch-less control, action blocks for quickly performing a specific task, support for hearing aids, and live transcription to name a few.
These accessibility settings are either baked in at the OS level as part of the Android Accessibility Suite or are made available in the form of dedicated apps. One of those apps is called ‘Look to Speak,’ which relies on tracking the gaze to execute tasks for users with speech and motor impairments. In addition to Android, Google has also been porting over some of these accessibility features to Chrome OS to ease the process of remote learning for students.
The latest accessibility tool from Google targets Android smartphones and relies on using facial expressions and gestures to perform tasks. The fourth Android 12 beta updated the Android Accessibility Suite and added a new Camera Switches feature. The Camera Switch presets are aimed at using an Android phone without interacting with the touch screen, or even voice commands. Once enabled and customized to the user's liking or needs, facial movements such as opening the mouth, looking left/right/up, and smiling will be used to perform a variety of tasks. To recall, Google previously released a similar health-centric feature that uses the front camera to measure the heart and respiratory rate of users.
How To Customize Facial Expressions
In order to enable the new Camera Switches feature, the Android phone user will need to make sure that they have the latest version of Android Accessibility Suite installed on their device via the Play Store, which was released following the fourth Android 12 beta update. To access the feature, open the settings menu and then tap on Accessibility followed by Switch Access. Once users reach the Switch Access page, they will need to enable the ‘Use Switch Access’ toggle to reach the gesture customization stage. After enabling it, tap on ‘Settings’ at the bottom and then on ‘Camera Switch settings’ on the next page. On the ‘Switch Access settings’ page, enable the ‘Use Camera Switches’ toggle. Once it has been activated, users can choose between six commands: Open Mouth, Smile, Raise Eyebrows, Look Left, Look Right, and Look Up.
Tapping on any of these gestures opens a new page where the user can customize the gesture strength for controlling their Android phone. Using ‘Open Mouth’ as an example, there is the option to choose between eight levels based on how wide the user wants to open their mouth for the command to be recognized and how long they must keep their mouth open for. At the bottom is an ‘Assignment’ button which opens a list of commands that will be executed when the camera detects the user has opened their mouth. Available actions include, moving forward or back from a page, touch & hold, returning to the home screen, opening the notifications section, accessing the quick settings, and so on. The rest of the commands such as smiling or looking left or right can be customized in the same was on an Android phone.
Source: Android Developers