On-skin Interaction Design for Smart Watches with Friction Sounds
Project Description
This project will design a new interaction method for smartwatches where the back of the hand can be turned into a trackpad and users can interact with the watch with touch gestures on the hand back. The rationale behind this design is the assumption that finger strokes on the back of the hand can be tracked by analyzing the friction sounds caused by finger scrolling. This process involves a series of audio signal processing and deep learning pipelines. Specifically, the audio signals will first undergo time-frequency analysis, where the start and end of the finger strokes are located and background noises are filtered. Then, the time-frequency features will be processed by DL models specially tailored for processing audio features, where the models output the predicted finger stroke trajectories.
Supervisor
XIE, Wentao
Quota
2
Course type
UROP1100
UROP2100
UROP3100
UROP4100
Applicant's Roles
- Survey the related research papers and form a general understanding of the design requirement.
- Design audio signal-processing and deep learning pipelines to facilitate finger tracking.
- Collect the finger-stroking dataset to power the deep-learning models.
- Design audio signal-processing and deep learning pipelines to facilitate finger tracking.
- Collect the finger-stroking dataset to power the deep-learning models.
Applicant's Learning Objectives
- Have a taste of the basic research methods, including literature review, research formulation, algorithm design, prototyping/implementation, solution evaluation, etc.
- Train DL/ML software skills, e.g., with scikit-learn, PyTorch, etc.
- Gain coding experience through hands-on research implementation.
- Train DL/ML software skills, e.g., with scikit-learn, PyTorch, etc.
- Gain coding experience through hands-on research implementation.
Complexity of the project
Moderate