To main content

Teaching a robot to grasp real fish by imitation learning from a human supervisor in virtual reality

Abstract

We teach a real robot to grasp real fish, by training
a virtual robot exclusively in virtual reality. Our approach
implements robot imitation learning from a human supervisor
in virtual reality. A deep 3D convolutional neural network
computes grasps from a 3D occupancy grid obtained from depth
imaging at multiple viewpoints. In virtual reality, a human
supervisor can easily and intuitively demonstrate examples of
how to grasp an object, such as a fish. From a few dozen of these
demonstrations, we use domain randomization to generate a
large synthetic training data set consisting of 100 000 example
grasps of fish. Using this data set for training purposes, the
network is able to guide a real robot and gripper to grasp
real fish with good success rates. The newly proposed domain
randomization approach constitutes the first step in how to
efficiently perform robot imitation learning from a human
supervisor in virtual reality in a way that transfers well to
the real world.
Read the publication

Category

Academic article

Language

English

Affiliation

  • SINTEF Ocean / Fisheries and New Biomarine Industry
  • Norwegian University of Science and Technology

Year

2018

Published in

Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

ISSN

2153-0858

Page(s)

7185 - 7192

View this publication at Norwegian Research Information Repository