BB12: Mixed Reality for Operators [FM]


Collection of systems to provide ergonomic guidance during manufacturing or allow the operator to teach a robot in virtual reality.

This building block has been developed to provide operators with easy-to-use tools in order to achieve complex assembly tasks or robot-controlling tasks without the need for supervision or prior training. In order to do so, it leverages the power of Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) to visually help and provide the Operator with the necessary information to perform the required tasks in real-time. In addition, the system keeps track of the task completion and is able to warn the operator if a mistake has been made.

The added value of this method is to avoid dividing the workflow into a training and a practice phase, allowing to bypass the training and directly jump into hands-on practice with the help of visual cues. Moreover, it allows untrained workers to fulfil critical tasks on the field and on the spot if required as well as monitor the correct completion of the tasks.

This technology is especially useful for companies that require operators to perform skilled manual tasks (like assembly or disassembly) or require operators to program a robot to perform such tasks.

The requirements for using such technology can range from a simple Mobile device (like an iPad in the case of AR) to VR headsets or XR technologies like Microsoft’s HoloLens.

To drive VR headsets, a powerful desktop computer must be purchased.

Here are a few Projects falling under the category of Mixed Reality for Operator:

PROROB Framework

Capturing of human motion using consumer-grade VR/MR hardware to program a robot.

DWI2PE (Digital Work Instruction to Production Employee)

Application that can visualize the necessary instructions for an operator throughout each step of a discrete assembly/disassembly procedure. Can connect with advanced state estimators.

VR Training for Agricultural Machine             

VR driving training for Agricultural machines.

Mixed Reality visualization of Robot Forces

Application that showcases an overlay in AR of the force exerted on a robot.

SmartFactory AR data visualization    

Visualization of Data of a smart assembly line.

Additional resources

ROBOT AR Overlay :

Smart Factory :

For more information, please contact:



Posted on

January 31, 2023