ML Comprobo


After implementing some more classic behavior for comprobo (described here), I began melding my machine learning interests with the Neato framework. So, for the next project I created (*drum roll*) Boobot:

*actual photo*

“What does it do?” you may ask. Well, as the name suggests, it mimics the behavior of the Boos from Mario. If you’re not familiar, it’s a similar behavior as the weeping angels from Doctor Who. If you’re still not familiar, you’re beyond hope so just watch this cat video to get the idea. Here’s some videos of the Boobot in action:

Implementation (nerdy stuff)

This problem naturally breaks down into two subproblems: person tracking and gaze detection.

Since I didn’t particularly feel like re-implementing a difficult and partially solved problem for a 2 week project, I decided to mostly outsource the person tracking. Since the code had to be fast and I was running it on a laptop, I went with the MobileNet  on the Tensorflow Zoo to find the bounding boxes for people within the frame. Using the center of these bounding boxes, I implemented a proportional controller to continually move towards a target while keeping them in the center of the frame, and it worked … like really unreasonably well. I could bounce all over the room and the Neato would effectively find me and move towards me.

Of course it would start to break down if there are multiple people in frame and any of them have a higher detection confidence then me. But, I was able to avoid this problem by giving people around my testing area angry looks until they moved.

With this tool, I set the Neato to take images a couple times a second and had it take pictures of me with both facing it and with my back turned as I wandered around the room. In the end I had about 3,000 segmented labeled images in a variety of outfits. I then trained a simple ConvNet with Dropout for about 2 minutes when the test error (on unseen outfits) reached a maximum and incorporated it into the system too produce the final result.

Github Repo


For my final Comprobo project, I implemented a behavior following the classical Lassie story. You can read more about it on the github page and see videos of it below, they took me longer then I’m willing to admit to edit.

Github Repo

I left out ~80% of the detail of the projects so let me know if you have any questions!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s