Real Time Hand Gesture Recognition in Industry

Abstract

With the 4th industrial revolution and the increased use of cobots in the industries comes many opportunities for new generation control panels. In this article, we proposed to develop a deep learning model to recognize in real time 10 different gestures that can be used to interact with a cobot. We proposed a new dataset containing gestures that can be used in an industrial context. The videos were taken from a computer webcam and then processed to remove the noise created by the background by isolating the movement of the gray scale images. We proposed to extract the spatio-temporal features by the combination of 3D convolution and LSTM layers. We also proposed a real time method to recognize our gestures, the frames are captured continuously and fed to the model to get a prediction every 2.4 seconds. Our experimental results show for 8 out of 10 gestures, a recognition rate of more than 90%. Furthermore, an interface was created to test our method in real time and to add new classes of gestures to be recognized by our model.