Freehand Circling for Object Selection Using a Head-mounted Camera
Baidu Eye is prototype head-mounted device with front facing camera. As a research intern in Baidu’s Institute of Deep Learning, I was responsible for developing a light algorithm that allows wearer to circle around things in his view using his pointing finger in order to select a specific object from a large scene. The selected item was then sent to Baidu’s image search API for object recognition.
This can be used for searching image of products, banners, logos, and even clipping a portion of an advertisement.
- Avoiding registration of close points to prevent hand shaking errors
- Ending selection by approaching a loop
- Getting a clean shot of object that is not covered by hand
- Counteracting background movements to fix the selection using:
- Lucas-Kanade optical flow algorithm
- Classifying camera versus other movements (hand, background noise)