What is different

This page says what changed in the world model solution when the overhead camera and the push counter were taken away. It is for a reader who has read the world model chapter of the pushing book and what every solution had to add. The method is the same. A small network learns what a push does, and a search tries many pushes against it. What a forward model is and the search are explained there and not here. The code is in the project's 04-a-world-model folder. Its README.md holds every number on this page unless a line names another file.

Contents#

1. A film of five tables
2. Why a prediction is worth more here
3. What the model is shown and what it answers
4. What the data cost the arm
5. Why it still looks after every push

1. A film of five tables#

The film comes first, because it shows the one thing this solution does best. It shows tables 10001 to 10005, the five tables every solution in this book is filmed on. The counts below are for the first of them, table 10001.

On table 10001 the solution gives 20 answers and uses 22 arm seconds. Only one of the 20 answers is a push. The rest are moves to look. The six films set this beside the other solutions on the same tables. The film counts come from the project's results/README.md. It separated this table in all three walks of the queue, with 20 answers each time, as results-tables.json shows.

The next section asks why one push can do the work of three here.

2. Why a prediction is worth more here#

The film showed fewer pushes than the reference. This section says what the model predicts that the reference has to find out with the arm.

An answer in this book is a path for the jaw, not a push. A path cannot feel its way to the glass. So the solution must know, before it writes the path, how far the hand reaches in before it touches the glass. This book calls that distance the reach. It is 10 to 30 mm, because the hand meets a stemmed glass at its stem.

The reference finds the reach by poking. It drives 5 mm deeper, looks, and repeats. That is the ranging push, and every round of it costs arm seconds. In the pushing book a look was free, so poking cost nothing. Here the arm's time is the budget. So this solution makes the reach one more input to the model. A prediction replaces the pokes.

One rule is still written by hand. The depth the fingertips may reach is capped by the same lean guard that caps ranging. A camera cannot tell a glass not yet touched from a glass that leans, so the model could not learn this by looking.

The next section asks what the model is given to make this prediction.

3. What the model is shown and what it answers#

The reach is now an input. This section lists the other inputs, the outputs, and how the search reads them.

The model takes 26 numbers. They are the pushed glass's height, widest width and foot. They include the push: where across the glass the hand meets it, how far it pushes, and the reach. They also give the place and size of up to four neighbouring glasses. It answers 11 numbers. They are where the pushed glass ends up, where each neighbour ends up, and the chance that a glass goes over. The code is in encoding.py.

There are three copies of the network, fitted from different starting weights. Where the copies disagree, the model has not seen pushes like this one, as ensembles as a measure of ignorance explains. In the pushing book only the chance of a topple was read from the worst copy. Here every reading is the worst copy's. The search in search.py drops a candidate push in four cases:

• any copy gives it more than 1 chance in 100 of a topple;
• a glass would land outside the glass zone, or the jaw would leave the arm's reach;
• the model predicts a movement longer than the push, which means the model is guessing;
• the hand would sweep a glass it is not aimed at on the way in.

The last filter is new. The model says where glasses end up. It never says what the hand passes through. So the search uses the reference's corridor rule for that. Then the best push must also pass the clearance check before it is answered. The next page says what these filters bought.

The next section asks what it cost to collect the pushes the model learns from.

4. What the data cost the arm#

The model needs real pushes to learn from. This section shows that, without the overhead camera, those pushes cost arm time.

The table below lists each round of pushes. Read each row across: how many tables, how many pushes, and how much arm time they took. The figures are from training.json.

In the pushing book the same kind of data cost no arm time. There, a push was followed by one call to a camera that saw the whole table. Here the arm must walk the ring of stands before it knows where anything is. It must walk again when the pushed glass is hidden. So a collecting table costs about 28 arm seconds and gives about four or five pushes.

The model is also less accurate than the pushing book's model. On the 658 validation pushes, 254 really moved a glass. For those, the model's guess of where the glass lands is 13.3 mm out at the median. It is 47.1 mm out at the 90th percentile. The pushing book's model was 6.5 mm out at the median. The second repair gives how well its topple filter does on the same pushes.

The next section asks whether the model's predictions can also save the look after a push.

5. Why it still looks after every push#

A model that says where every glass ends up could plan a second push without looking. This section explains why that did not save arm time.

The table below compares the two ways on 40 tuning tables. Each figure is the average of three runs. Read it row by row. The first row looks after each push. The second row trusts the model and pushes again without looking.

Trusting the model gives 14% fewer answers. But it uses about 10% more arm time, for the same tables. The reason is where the jaw stands. After a push, the jaw is behind the glass and faces it. So the look that checks the push is usually free: the camera takes another picture from an arm that is not moving. Trusting the model saves a short drive only when the glass ends up hidden. It then spends extra pushes. Those pushes rest on the model's guess of where the glass landed. Before the second repair, that guess was 14 mm out at the median and 50 mm out for the worst tenth. In the pushing book this question did not exist, because every look was free.

This page has shown what the port changed. The next page asks what those changes scored on the held-out queue, and which repairs made the difference.

← Imitation from demonstrations — questions and answers · What the repairs bought →