Abstract
We teach a real robot to grasp real fish, by training
a virtual robot exclusively in virtual reality. Our approach
implements robot imitation learning from a human supervisor
in virtual reality. A deep 3D convolutional neural network
computes grasps from a 3D occupancy grid obtained from depth
imaging at multiple viewpoints. In virtual reality, a human
supervisor can easily and intuitively demonstrate examples of
how to grasp an object, such as a fish. From a few dozen of these
demonstrations, we use domain randomization to generate a
large synthetic training data set consisting of 100 000 example
grasps of fish. Using this data set for training purposes, the
network is able to guide a real robot and gripper to grasp
real fish with good success rates. The newly proposed domain
randomization approach constitutes the first step in how to
efficiently perform robot imitation learning from a human
supervisor in virtual reality in a way that transfers well to
the real world.
a virtual robot exclusively in virtual reality. Our approach
implements robot imitation learning from a human supervisor
in virtual reality. A deep 3D convolutional neural network
computes grasps from a 3D occupancy grid obtained from depth
imaging at multiple viewpoints. In virtual reality, a human
supervisor can easily and intuitively demonstrate examples of
how to grasp an object, such as a fish. From a few dozen of these
demonstrations, we use domain randomization to generate a
large synthetic training data set consisting of 100 000 example
grasps of fish. Using this data set for training purposes, the
network is able to guide a real robot and gripper to grasp
real fish with good success rates. The newly proposed domain
randomization approach constitutes the first step in how to
efficiently perform robot imitation learning from a human
supervisor in virtual reality in a way that transfers well to
the real world.