This video shows our preliminary work on human-machine interactions. The video demos an interactive robotic
arm system. The robotic arm is set to track and follow the right hand of the subject in an opposite position
in an interactive manner. The system utilized RBG and IR depth-finding camera for user behavior capturing
and inexpensive 5-DOF robotic arms for interaction.
Human Activity Recognition
These demos showcase a series of novel approachs on device-free/wearable sensors, real-time activity
recognition It can be potentially used in a wide range of applications such as Fall Detection, Ambulatory
Monitoring, assistive living and abnormal activities detection etc.
Xiang Zhang,Lina Yao, Chaoran Huang, Sen Wang, Mingkui Tan, Guodong Long, Can Wang, Multi-modality
Sensor Data Classification with Selective Attention. The 27th International Joint Conference on
Artificial Intelligence (IJCAI 2018), July 13-19 2018, Stockholm, Sweden.
Kaixuan Chen,Lina Yao, Xianzhi Wang, Dalin Zhang，Tao Gu, Zhiwen Yu and Zheng Yang. Interpretable
Recurrent Convolutional Neural Networks with Parallel Attentions for Multi-Modality Activity
Modeling.International Joint Conference on Neural Networks (IJCNN 2018), Rio de Janeiro,
Brazil, July 8 - 13, 2018.
Kaixuan Chen,Lina Yao, Tao Gu, Zhiwen Yu, Xianzhi Wang and Dalin Zhang. Fullie and Wiselie: A
Dual-Stream Recurrent Convolutional Attention Model for Activity Recognition arXiv
Lina Yao, Quan Z. Sheng, Xue Li, Tao Gu, Mingkui Tan, Xianzhi Wang, Sen Wang and Wenjie Ruan. Compressive
Representation for Device-Free Activity Recognition with Passive RFID Signal Strength. IEEE
Transactions on Mobile Computing (TMC) , 2017
Lina Yao, Feiping Nie, Quan Z. Sheng, Tao Gu, Xue Li, Sen Wang, Learning from Less for Better:
Semi-Supervised Activity Recognition via Shared Structure Discovery. The 2016 ACM International
Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2016). Heidelberg, Germany, September
Lina Yao, Quan Z. Sheng, Xue Li, Tao Gu, Sen Wang, Wenjie Ruan and Wan Zou.Freedom: Online
Activity Recognition via Dictionary-based Sparse Representation of RFID Sensing Data. The IEEE
International Conference on Data Mining (ICDM 2015), Atlantic City, NJ, USA, November 14 - 17, 2015.
Lina Yao, Quan Z. Sheng, Wenjie Ruan, Tao Gu, Xue Li, Nickolas J.G. Falkner, and Zhi Yang.RF-Care:
Device-Free Posture Recognition for Elderly People Using Passive RFID Tag Array. The 12th
International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services
(MobiQuitous 2015). Coimbra, Portugal, July 22–24, 2015.
Some project demos from students who take research project in our group
"Activity Recognition Using Embedded Sensors in Smartphones", Bachelor CS (Advanced) Research
Project by Leon Chea, 2015.
This project explores the process of developing an Android system that utilises the embedded sensors
in a smartphone to recognise a number of common human actions and postures (Standing, Sitting,
Walking, Lying,...). Smartphones are a widely available commercial device and using it as a basis
for this project creates the possibility of future widespread usage and potential applications. The
sensors used include the accelerometer, gyroscope and magnetometer, all of which are commonly found
in modern smartphones.
"Automatically Recognize Unhealthy Use of Smartphones", Bachelor CS (Advanced) Research Project by
Yuchieh (Henry) Yang, 2015.
This project explores an automated, objective and repeatable approach for assessing problematic usage
via collecting a wide range of phone usage data from smartphones, identify a number of usage
features that are relevant to this assessment, and build detection models automatically detecting
problematic use. For example, using phones in the darkness.