Mixed Reality Human-Robot Interaction (2/4)

Researchers at DFKI present Mixed Reality Production 4.0, a new
form of multiple site Human-Robot Collaboration (HRC). Adaptable
and trainable lightweight robots at the exhibit stand of the Federal
Ministry of Education and Research interact with their human and
robotic colleagues at the DFKI exhibit. Supported by mixed reality
and virtual reality systems, employees at one location in combination
with robots and colleagues at another location can flexibly perform
the task of handling hazardous materials.

CeBIT visitors can experience first-hand how the actions of a
human at one site directly control a robot’s actions at another site.
The exhibit is supplemented by displays of what the first-person
view of the mixed reality looks like through HoloLens goggles.

The demonstration shows how three identical robots at the DFKI
stand are telemanipulated by an operator with HoloLens goggles
physically located at the neighboring BMBF exhibit stand. The exhibit
is expanded by the telemanipulation of two robots (Universal
Robotics UR and the logistics robot MiR 100) via live-circuit with the
HRC 4.0 Innovation Lab in DFKI-Saarbrücken.

Mixed reality systems make it possible to manipulate objects in a
three-dimensional virtual representation of the collaborative scenario
in real time. The user can interact with the robots by watching
a virtual representation of the target object and, through gestures,
trigger the required actions by the robot.

Additionally, telemanipulation is demonstrated in the immediate
vicinity. Three robots at the BMBF stand (ABB Yumi, Universal Robotics UR, and MiR 100) perform tasks together with the operator. The operator controls the activities in a mixed reality environment
(HoloLens) by means of gestures. The robots execute the commands,
grasp and move objects to a tray or table mounted on the
logistics robot. In this way, objects are safely transported between
the lightweight robots. The aim here is to achieve the standard of
error-free robot action in some remote “hazardous material” scenario.

Practical application areas in an industrial manufacturing context
are remote maintenance, telepresence, teleproduction and teleoperation as well as Human-Robot Collaboration.