Building block “Manual Task Recognition” can identify a known manual task in real-time.
This block was initially developed for the European project “HADRIAN” to improve the comfort and safety of drivers of autonomous vehicles. Its aim was to identify whether the hands of drivers were holding the steering wheel or not.
A camera films the expected hands’ location and an artificial neural network identifies in real time the executed task. The used camera can be either RGB or infrared to allow use in dark environments with infrared lighting. In order to train the artificial neural network, a big database of images, taken while performing the expected tasks, must be acquired and annotated by a third party. A GPU is required to run the artificial neural network. The size of the database may be reduced if using transfer learning from a previously pre-trained model. In inference mode, the system tracks the hands and does not keep any images in the memory. In learning mode, the camera can be placed so that faces are not visible in the recorded images. An automatic face blurring block can also be added to the recording system so that the stored images would not contain any facial information.
In the following figure, four cases of hand detection are presented.
More details on the developed system are given in the following video (NB: for EARASHI project, the part of the system using grip sensors isn’t proposed):
https://www.youtube.com/watch?v=qfYOM4sdWr4
This building block can be used to verify whether a workstation is well adapted to the operator or not. For example, if hands are not detected at the expected positions, it can allow to identify that an adaptation of the workstation must be done to help the operator keep his/her hands in the right position during hours. It also can allow to improve safety by interrupting the robot’s movement if the operator is not executing the good task at the right moment.
This building block can be used in situations where there is a will to improve physical working conditions and the safety of operators.
For more information, please contact:
- Claire Guyon-Gardeux – claire.gardeux@cea.fr