After implementing some more classic behavior for comprobo (described here), I began melding my machine learning interests with the Neato framework. So, for the next project I created (*drum roll*) Boobot:
“What does it do?” you may ask. Well, as the name suggests, it mimics the behavior of the Boos from Mario. If you’re not familiar, it’s a similar behavior as the weeping angels from Doctor Who. If you’re still not familiar, you’re beyond hope so just watch this cat video to get the idea. Here’s some videos of the Boobot in action:
Implementation (nerdy stuff)
This problem naturally breaks down into two subproblems: person tracking and gaze detection.
Since I didn’t particularly feel like re-implementing a difficult and partially solved problem for a 2 week project, I decided to mostly outsource the person tracking. Since the code had to be fast and I was running it on a laptop, I went with the MobileNet on the Tensorflow Zoo to find the bounding boxes for people within the frame. Using the center of these bounding boxes, I implemented a proportional controller to continually move towards a target while keeping them in the center of the frame, and it worked … like really unreasonably well. I could bounce all over the room and the Neato would effectively find me and move towards me.
Of course it would start to break down if there are multiple people in frame and any of them have a higher detection confidence then me. But, I was able to avoid this problem by giving people around my testing area angry looks until they moved.
With this tool, I set the Neato to take images a couple times a second and had it take pictures of me with both facing it and with my back turned as I wandered around the room. In the end I had about 3,000 segmented labeled images in a variety of outfits. I then trained a simple ConvNet with Dropout for about 2 minutes when the test error (on unseen outfits) reached a maximum and incorporated it into the system too produce the final result.
For my final Comprobo project, I implemented a behavior following the classical Lassie story. You can read more about it on the github page and see videos of it below, they took me longer then I’m willing to admit to edit.
I left out ~80% of the detail of the projects so let me know if you have any questions!