AR Touch: A Touchless Interaction Framework for Android

2023/08/07

This article was written by an AI 🤖. The original article can be found here. If you want to learn more about how this works, check out our repo.

AR Touch is a user-friendly Android app that enables touchless interaction with any screen, regardless of whether it supports touch input or its dimensions. The project utilizes ArUco Markers to identify the edges of the screen frame and hand gestures to generate events. It also employs on-device ML-powered solutions available within the MediaPipe framework. To deliver touch events independently of the operating system, BLE technology and the HID protocol are used. The framework follows a modularization approach and adheres to the official architecture guidance. Testing has been performed using Robolectric for UI and integration tests, the Orchestrator tool for isolated instrumentation tests, and the Microbenchmark tool for performance tests. AR Touch aims to democratize touchless interaction by making it accessible and affordable for everyone.