BIT-VO: Visual Odometry at 300 FPS using Binary Features from the Focal Plane

Focal-plane Sensor-processor (FPSP) is a next-generation camera technology which enables every pixel on the sensor chip to perform computation in parallel, on the focal plane where the light intensity is captured. SCAMP-5 is a general-purpose FPSP used in this work and it carries out computations in the analog domain before analog to digital conversion. By extracting features from the image on the focal plane, data which is digitised and transferred is reduced. As a consequence, SCAMP-5 offers a high frame rate while maintaining low energy consumption. Here, we present BIT-VO, which is, to the best of our knowledge, the first 6 Degrees of Freedom visual odometry algorithm which utilises the FPSP. Our entire system operates at 300 FPS in a natural scene, using binary edges and corner features detected by the SCAMP-5.

BIT-VO uses only sparse features (corner coordinates and binary edges) to perform high frame-rate visual odometry. These features are computed on the SCAMP-5 FPSP in the analog-domain, before analog-to-digital conversion, minimising the necessary readout from the sensor.

Presentation at IROS 2020

Test sequences used in the paper

For the conference version of the paper, please cite as:

@inproceedings{Murai:etal:IROS:2020,
    title={BIT-VO: Visual Odometry at 300 FPS using Binary Features from the Focal Plane},
    author={Murai, Riku and Saeedi, Sajad and Kelly, Paul HJ},
    booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
    year={2020},
    organization={IEEE}                
}  
            

For the journal version of the paper, please cite as:

@inproceedings{Murai:etal:AR:2023,
    title={High-frame rate homography and visual odometry by tracking binary features from the focal plane},
    author={Murai, Riku and Saeedi, Sajad and Kelly, Paul HJ},
    journal={Autonomous Robots},
    pages={1--14},
    year={2023},
    publisher={Springer}
}   
      

We would like to thank Piotr Dudek, Stephen J. Carey, and Jianing Chen at the University of Manchester for kindly providing access to SCAMP-5. For more information about SCAMP-5, please visit here.