Robotics Docs

Perception

Cameras, depth and finding objects

How a camera turns the world into pixels, how to turn pixels back into points, and how a robot finds an object and measures its size and pose, with the techniques, models and licences compared.

2 chapters14 sections6 h 55 min of reading
0 of 14 sections read
Start reading
  1. 01

    Cameras

    A robot arm that has to pick something up first has to find it, and the usual way to find things is with a camera. This doc explains how a camera works, starting from nothing: what it records, what it loses, how a depth camera…

    4 sections · 1 h 50 min

    1. 1.1Cameras: the basics23 min
    2. 1.2Finding an object in a picture23 min
    3. 1.3Finding one box with a camera24 min
    4. 1.4Finding one box: the code38 min
  2. 02

    Object Perception

    A robot that is going to pick something up has to answer two questions about it. What is this, and which part of the picture is it on. Then, before it can close its fingers, how big is it and which way is it turned.

    10 sections · 5 h 5 min

    1. 2.1Object perception: finding things and measuring them16 min
    2. 2.2The sensors, and the software for each49 min
    3. 2.3Methods you write yourself41 min
    4. 2.4Models that find objects17 min
    5. 2.5Models that measure12 min
    6. 2.6Licences, platforms and the comparison grids12 min
    7. 2.7Making it work: measuring, timing and debugging33 min
    8. 2.8The wrist camera, end to end35 min
    9. 2.9Choosing where to look48 min
    10. 2.10Tracking and association: deciding it is the same object43 min
Next bookBook 3: Frameworks & ManipulationMuJoCo, Gazebo, MoveIt and training arms