The idea of Neodroid is to create a reality-ready robot brain in virtual reality (VR). We specifically focus on creating a robot brain capable of humanoid visual-motor ability. Visual-motor ability is the integration between visual perception and motor skills. More specifically, it is the ability to perform constructive tasks integrating both visual perception and motor skills. The motivation behind Neodroid is to enable robots to assist humans in performing such tasks.
The primary objective of Neodroid is:
- Develop deep learning architectures for creating a reality-ready robot brain in virtual reality.
The secondary objectives are:
- Implement a virtual reality environment for training a virtual robot.
- Develop deep learning architectures for visual-motor tasks. - Implement human operation of a robot in virtual reality.
- Demonstrate robot deep learning in virtual reality.
- Demonstrate transfer-learning between virtual reality and the real world.
The essence of Neodroid is a two-stage learning process in VR, followed by introduction to reality. The VR learning stages are first a 'baby' phase with environment-assisted learning, and a 'school' phase where a human enters VR to teach more complex tasks. After the school phase, the brain is transferred from the virtual to the real robot, so learnt visual-motor skills can be applied in the real world.
The key challenges to enabling near-human level robot visual motor skill, using deep learning, are to generate a large set of training examples and to discover the best deep learning architectures. We believe that these challenges are best met by prototyping deep learning in VR, since this provides greater flexibility and a more cost-efficient and timely data acquisition and teaching process than using real robots and sensors. Once this architecture is found, it can be transferred to a real robot and refined.