Human Machine Interaction and robotics

Human Machine Interaction

This video shows our preliminary work on human-machine interactions. The video demos an interactive robotic arm system. The robotic arm is set to track and follow the right hand of the subject in an opposite position in an interactive manner. The system utilized RBG and IR depth-finding camera for user behavior capturing and inexpensive 5-DOF robotic arms for interaction.

Human Activity Recognition

These demos showcase a series of novel approachs on device-free/wearable sensors, real-time activity recognition It can be potentially used in a wide range of applications such as Fall Detection, Ambulatory Monitoring, assistive living and abnormal activities detection etc. (download)

Related Papers

• Xiang Zhang,Lina Yao, Chaoran Huang, Sen Wang, Mingkui Tan, Guodong Long, Can Wang, Multi-modality Sensor Data Classification with Selective Attention. The 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), July 13-19 2018, Stockholm, Sweden.
• Kaixuan Chen,Lina Yao, Xianzhi Wang, Dalin Zhang，Tao Gu, Zhiwen Yu and Zheng Yang. Interpretable Recurrent Convolutional Neural Networks with Parallel Attentions for Multi-Modality Activity Modeling.International Joint Conference on Neural Networks (IJCNN 2018), Rio de Janeiro, Brazil, July 8 - 13, 2018.
• Kaixuan Chen,Lina Yao, Tao Gu, Zhiwen Yu, Xianzhi Wang and Dalin Zhang. Fullie and Wiselie: A Dual-Stream Recurrent Convolutional Attention Model for Activity Recognition arXiv
• Lina Yao, Quan Z. Sheng, Xue Li, Tao Gu, Mingkui Tan, Xianzhi Wang, Sen Wang and Wenjie Ruan. Compressive Representation for Device-Free Activity Recognition with Passive RFID Signal Strength. IEEE Transactions on Mobile Computing (TMC) , 2017
• Lina Yao, Feiping Nie, Quan Z. Sheng, Tao Gu, Xue Li, Sen Wang, Learning from Less for Better: Semi-Supervised Activity Recognition via Shared Structure Discovery. The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2016). Heidelberg, Germany, September 12-16, 2016.
• Lina Yao, Quan Z. Sheng, Xue Li, Tao Gu, Sen Wang, Wenjie Ruan and Wan Zou.Freedom: Online Activity Recognition via Dictionary-based Sparse Representation of RFID Sensing Data. The IEEE International Conference on Data Mining (ICDM 2015), Atlantic City, NJ, USA, November 14 - 17, 2015.
• Lina Yao, Quan Z. Sheng, Wenjie Ruan, Tao Gu, Xue Li, Nickolas J.G. Falkner, and Zhi Yang.RF-Care: Device-Free Posture Recognition for Elderly People Using Passive RFID Tag Array. The 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous 2015). Coimbra, Portugal, July 22–24, 2015.

Some project demos from students who take research project in our group

• "Activity Recognition Using Embedded Sensors in Smartphones", Bachelor CS (Advanced) Research Project by Leon Chea, 2015.

  This project explores the process of developing an Android system that utilises the embedded sensors in a smartphone to recognise a number of common human actions and postures (Standing, Sitting, Walking, Lying,...). Smartphones are a widely available commercial device and using it as a basis for this project creates the possibility of future widespread usage and potential applications. The sensors used include the accelerometer, gyroscope and magnetometer, all of which are commonly found in modern smartphones.

• "Automatically Recognize Unhealthy Use of Smartphones", Bachelor CS (Advanced) Research Project by Yuchieh (Henry) Yang, 2015.

  This project explores an automated, objective and repeatable approach for assessing problematic usage via collecting a wide range of phone usage data from smartphones, identify a number of usage features that are relevant to this assessment, and build detection models automatically detecting problematic use. For example, using phones in the darkness.