What is different

What it uses. It uses PyTorch and LeRobot's ACT, as in the pushing book. It also uses one new small model, written for this problem. Both models start from random numbers. Nothing is downloaded. What it does. It watches the ranked geometry solution clear 900 tables. It records every answer that solution gives. Then it fits two models. The first model chooses the kind of answer and the glass. The second model draws the path for that choice. What it costs. Collecting the examples took about 24 minutes of computer time. Fitting took about 7 minutes a copy for the small model and about 57 minutes a copy for ACT. Three copies of each were fitted.

Contents#

1. Introduction
2. A film of five tables
3. The demonstrations
4. The first version did nothing
5. One model decides, one model draws
6. Teaching the gate that a rare answer counts
7. A push written from its own glass
8. What the gate is told about its own past
9. What the fitting cost

1. Introduction#

This page answers one question. What had to change in the copying solution when the camera moved to the wrist? It is for a reader who already knows the method from the pushing book. That book explains behaviour cloning and action chunking. This chapter does not explain them again.

The teacher is the same as before. It is the ranked geometry solution. It needs no learning, so it can clear as many tables as we like. Every answer it gives is a free example.

The big change is in the examples. Most of the teacher's answers are now moves to look, not pushes. A single copied model learned only the looking. So the solution was rebuilt as two models. This page explains that split and the smaller changes that came with it. What the repairs bought gives the results.

The film comes first, because it shows the solution's main weakness.

2. A film of five tables#

The introduction named the change, and this film shows the solution at work on five held-out tables in a row, 10001 to 10005. It is the first of the three fitted copies.

The film is short because the solution stops early. The project's page for this solution describes it. On the first table, 10001, every answer is a move to look. No answer comes down to a glass. Then the first model says the table is done. But two glasses still have no room. Nothing is broken, because every path stays high above the rims.

The next section asks what the solution was copying, to explain why it acts like this.

3. The demonstrations#

The film shows a solution that mostly looks. This section shows that the examples are mostly looks too.

A demonstration is one pair. The first half is what the teacher was shown. That is the wrist picture, the glasses in it, the jaw's position and the arm time left. The second half is the teacher's answer, which is a path for the jaw. The collector also writes down two new labels. One is the kind of answer: look, probe, push or done. The other is which glass the answer was aimed at, or "none".

The examples come from 900 tables, numbered 1000 to 1899. The table below counts the teacher's answers by kind. Read each row as one kind of answer, with how many times the teacher gave it.

The first two rows are looks. Four answers in five are looks gives the share of each kind. Of the 33,721 answers, 34 were dropped, because a glass went over or left the table area. That leaves 33,687 demonstrations. The counts are in collected.json.

A second, smaller set came from 50 other tables, numbered 8000 to 8049. It holds 1,900 demonstrations. No model is fitted on it. It is used to measure how close a model is on tables it never saw.

Collecting took 1,461.7 seconds of computer time, which is 1.62 seconds a table. In the pushing book a table took 0.74 seconds. A table costs twice as much here, because the teacher must travel to see before it can push. The demonstrations README gives both numbers.

Two things about the stored path are new. Both come from the wrist camera.

• A path keeps its own length. A look is one waypoint, and a push can be up to 190. So each path is padded to a fixed length by repeating its last waypoint. An extra column says where the path really ends. The pushing book squeezed each push into a fixed number of waypoints instead. That changed the spacing, and in this cell the spacing sets the jaw's speed. Padding keeps the teacher's speed, and it keeps the sharp corner at the bottom of a descent.
• The jaw's position is an input. In the pushing book the camera was fixed, so the picture alone placed every glass. Here the same table looks different from each stand. Without the jaw's position, the model cannot place what it sees.

chunks.py builds the path, and readings.py builds the input.

Four answers in five are looks. The next question is what a single copied model made of that.

4. The first version did nothing#

The examples are mostly looks. This section shows what one model learned from them.

The first version was one chunked model, fitted to copy the teacher's paths. On the held-out queue it freed no glasses at all. It said nine of its ten tables were done while they were still crowded. It had never pushed anything, so the shared rule for "done" saw nothing but quiet answers. The solution's README records this.

The cause is shared with the other learned solutions. Fitting learns the average answer, and here the average answer is a look. Why the learned ones failed explains it once for all of them.

That cause points straight at a repair, which the next section describes.

5. One model decides, one model draws#

The first version failed because one model had to give two very different kinds of answer. This section gives the two jobs to two models.

The first model is the gate. It is shown what the policy is shown. It answers two questions. Which kind of answer should be given: look, probe, push or done? And at which glass? It is small: four convolution layers over the picture, and two output heads. A convolution layer is a filter that finds patterns in a picture.

The second model is the policy. It is the pushing book's ACT, with the same settings. It gets one new input, which is the gate's decision. It no longer has to work out whether this is a time to look or to push. It is told. Its only job is to draw the path.

So a choice is now made by a classifier, and a path is drawn by a regressor. A classifier picks one of a few named options. A regressor predicts numbers, such as positions. Their losses do different things. A loss that averages positions cannot say "these two answers are different kinds". A classifier's loss can.

The gate also says when the table is done. The teacher's own "done" answers are in the examples, so the gate learns them. The shared quiet rule is kept only as a backstop. Looking, remembering and stopping explains that rule.

The obvious alternative was to keep one model and give it more examples. That would not help. More examples of a habit teach the same habit. The split has a cost. The gate must now be right twice in a row. It must choose a probe, and then a push, at the same glass. gate.py holds the gate and its argument.

Splitting the models was not enough on its own, and the next section shows why.

6. Teaching the gate that a rare answer counts#

The split gave the decision its own model. But the first gate made the same mistake in a smaller form.

The first gate learned three of the four kinds. It never once answered "push". That is the safest way to be wrong least often, because pushes are so rare. But it is useless, because the push is the answer that changes the table. Two changes fixed it.

• Each kind is weighted by how rare it is. Each kind's part of the loss is divided by how common that kind is. So a missed push costs as much as a missed look. A push given where none was needed costs little. It is a short shove, and the clearance check still stands in front of it.
• The glass head learns only from answers that name a glass. A look and a "done" name no glass. A head fitted on every answer learns mostly to say "none". So answers that name no glass are left out of that head's loss.

The policy gets a matching change. Its training batches are drawn evenly from the kinds, not in the teacher's proportions.

The table below shows how well the three fitted gates did on tables they never saw. Each row is one measure, with the lowest and highest of the three gates.

The gate now finds most of the teacher's pushes. It also offers a push far more often than the teacher does. That is on purpose. Being eager is how a rare answer survives fitting. The numbers are in weights/gate.json.

The gate now names a glass. The next section shows how the policy uses that name.

7. A push written from its own glass#

The gate names the glass to push. This section uses that name to make a push easier to learn.

A push is the same movement wherever it happens. The jaw comes down beside the glass and shoves it away from its neighbour. In table coordinates, each place on the table gives a different set of numbers. So the policy would have to learn the push once for every place. Written as an offset from the named glass, it is one movement. The world model in the pushing book uses the same idea. The push's own frame explains it.

The policy draws the push relative to its glass. The solution then adds the glass's position back on before the arm moves.

The stored examples stay in table coordinates. They are shared with the fine-tuned model, which has no gate and so cannot name a glass. So train.py converts each push into its glass's frame as it builds a batch.

A push from the right glass is still only the last step of a short sequence. The next section asks how the gate can know which step comes next.

8. What the gate is told about its own past#

A push comes after a probe, and a probe comes after the jaw reaches the glass. This section gives the gate what it needs to follow that order.

One picture cannot show what was already tried on a glass. So the gate gets two extra numbers for each glass. One is how many of its own answers were aimed at that glass. The other is how far that glass has moved since it was first measured. These numbers helped every part of the gate, and most of all its finding of probes.

The gate also gets three numbers about its own work on this table. The first is how many answers it has given. The second is how many since it last touched a glass. The third is how far the jaw has travelled since then. These matter when the cell refuses a path. A refused path does not move the arm, so the next picture is the same. Without these numbers the gate would give the same answer for ever. With them, its input changes after every answer, so a stuck gate in the end says "done".

These numbers are facts about the solution, not about the table. That is why the solution may keep them.

With the two models and their inputs in place, the last question is what they cost to fit.

9. What the fitting cost#

The sections above describe two models. This section says what it took to fit three copies of each.

Three copies were fitted, each from a different random start, because one fit is not a measurement. The table below gives the cost. Read each row as one model, with the work for one copy.

The three policy copies took 2 hours and 51 minutes in all. The policy's fit reads every answer that touches a glass, but only 8,000 of the looks, which makes 14,994 frames. Reading all the looks would cost a lot of memory for frames no step would use. The gate reads every answer, because how often each kind happens is part of what it learns. The numbers are in weights/training.json and weights/gate.json.

Answering is cheap: about 21 milliseconds of thinking for each answer.

So the solution is built and fitted. The next page, what the repairs bought, asks what it achieved on the held-out queue.

← Geometry generates, a model ranks — questions and answers · What the repairs bought →