Robota: Companion robot

A concept implementation of a companion robot utilizing scene recognition. Took an excelence award in a Hackathon hosted by Yahoo.

Preview

Details

  • A concept implementation of an everyday robot companion "Robota".

  • Robota will wake you up, greet you when you come back home, praise you when you work hard, scold you when you sleep too long etc.

  • A webcam is placed in the room to capture the sitution and allow for these kinds of interactions.

  • Worked as a team of two. I was in charge of the action estimation, my friend of the robot controls.

  • At first, precision of the action estimation was very low. This was due to overfitting since we only had less than a thousand images. The solution was to use a larger variety of webcam angles, in combination with more aggressive data augmentation.

Image of a robot

System overview

Real-time action estimation in a custom environment

Presentation Video from Hackathon (Japanese, 180s)

Tools

  • Trained with pytorch.

  • Backbone Model is MobilenetV2, since this had to run in real-time on a laptop.

  • Weights & Biases was used to monitor all training runs.

  • Docker was used to ensure the same environent between training (A multi-GPU server) and running for inference (a laptop)

Gallery

image-0