Robotics Docs

Pushing the Glasses Apart

One manipulation problem, answered six ways and measured

The glasses stand too close together for the gripper to reach between them, so the arm has to drag them apart without knocking any of them over. This book answers that six ways, from a single fixed nudge with nothing learned at all, through a model that ranks pushes the geometry proposed, to a borrowed robot foundation model fine-tuned on this cell. All six are built and scored by one examiner, and the result is worth the reading: nothing learned beats the written rule here, the one learned method that wins is the one that learned the quantity the geometry gets wrong, and every method that learns something topples at least one glass where the geometry topples none.

10 chapters62 sections7 h 30 min of reading
0 of 62 sections read
Start reading
  1. 01

    The Problem

    The camera work is finished. Telling the glasses apart in a picture tells the arm where each glass stands. But some glasses stand too close together. The gripper cannot get round one of them without hitting its neighbour.

    6 sections · 1 h 25 min

    1. 1.1Push the glasses apart — what is asked for12 min
    2. 1.2What goes in and what comes out13 min
    3. 1.3The examiner — the same question for every answer16 min
    4. 1.4The target layout — where the glasses should end up18 min
    5. 1.5Pushing without toppling17 min
    6. 1.6Questions and answers8 min
  2. 02

    The Seven Solutions

    This book asks the arm to push crowded glasses apart. Each glass must end with room for the gripper, and no glass may fall over. What is asked for sets out the task in full. This chapter answers it in seven different ways. The…

    9 sections · 38 min

    1. 2.1The seven solutions — same table, seven ways to push11 min
    2. 2.2One fixed nudge4 min
    3. 2.3Geometry generates, a model ranks3 min
    4. 2.4Imitation from demonstrations4 min
    5. 2.5A world model, then plan with it6 min
    6. 2.6A foundation model as it downloads3 min
    7. 2.7The same model, fine-tuned here4 min
    8. 2.8A newer, larger model, fine-tuned here1 min
    9. 2.9Questions and answers2 min
  3. 03

    Solution 1: One Fixed Nudge

    This page describes the simplest way to move a crowded glass away from its neighbour. It is for a reader who wants to see the baseline before the six solutions that try to beat it. The method is one subtraction, one…

    7 sections · 48 min

    1. 3.1What it is8 min
    2. 3.2The code9 min
    3. 3.3How it works10 min
    4. 3.4A worked example7 min
    5. 3.5What it needs2 min
    6. 3.6How it compares11 min
    7. 3.7Questions and answers1 min
  4. 04

    Solution 2: Geometry Generates, a Model Ranks

    This page explains a safe way to put a fitted model inside a machine that can break things. It is for a reader who has read the problem chapter and wants to see the simplest solution that learns anything.

    7 sections · 46 min

    1. 4.1What it is7 min
    2. 4.2The code7 min
    3. 4.3How it works15 min
    4. 4.4A worked example4 min
    5. 4.5What it needs1 min
    6. 4.6How it compares11 min
    7. 4.7Questions and answers1 min
  5. 05

    Solution 3: Imitation From Demonstrations

    This chapter explains how to push the glasses apart by copying examples. The arm is shown many good pushes, and a model is trained to copy them. This method is called behaviour cloning. It is the simplest kind of learning. There…

    7 sections · 55 min

    1. 5.1What it is6 min
    2. 5.2The code10 min
    3. 5.3How it works13 min
    4. 5.4A worked example10 min
    5. 5.5What it needs3 min
    6. 5.6How it compares9 min
    7. 5.7Questions and answers4 min
  6. 06

    Solution 4: A World Model, Then Plan With It

    This chapter explains one way to push the glasses apart. The solution first learns what a push does. Then it chooses each push by testing candidates against what it learned, not against the table.

    7 sections · 55 min

    1. 6.1What it is7 min
    2. 6.2The code10 min
    3. 6.3How it works20 min
    4. 6.4A worked example4 min
    5. 6.5What it needs3 min
    6. 6.6How it compares10 min
    7. 6.7Questions and answers1 min
  7. 07

    Solution 5: A Foundation Model as It Downloads

    This chapter answers the problem this book sets in the simplest possible way. It downloads a general robot model and runs it. Nothing is collected, trained or fitted in this cell. The model is shown the table, told in words what…

    7 sections · 40 min

    1. 7.1What it is6 min
    2. 7.2The code7 min
    3. 7.3How it works14 min
    4. 7.4A worked example3 min
    5. 7.5What it needs1 min
    6. 7.6How it compares6 min
    7. 7.7Questions and answers3 min
  8. 08

    Solution 6: The Same Model, Fine-Tuned Here

    This page explains how this solution answers the problem this book sets. It takes a robot foundation model that was trained somewhere else. Then it continues that training on pushes made in this cell. This is called fine-tuning…

    7 sections · 52 min

    1. 8.1What it is7 min
    2. 8.2The code6 min
    3. 8.3How it works18 min
    4. 8.4A worked example5 min
    5. 8.5What it needs3 min
    6. 8.6How it compares11 min
    7. 8.7Questions and answers2 min
  9. 09

    Solution 7: A Newer, Larger Model, Fine-Tuned Here

    This page explains the seventh solution. It is for a reader who has read the chapter on solution 6. Solution 7 is solution 6 with one change: a newer and larger borrowed model. It was fitted three times and did not learn the…

    4 sections · 11 min

    1. 9.1What it is4 min
    2. 9.2What it needs2 min
    3. 9.3Results3 min
    4. 9.4Questions and answers2 min
  10. 10

    The Results

    This page is the scoreboard for the seven solutions in this book. It answers one question. On the same crowded tables, with the same marking, how many glasses did each method clear, and how many did it knock over? It is for a…

    1 section · 19 min

    1. 10.1The seven solutions side by side19 min