What Every Solution Had to Add — What fixed them, and what did not

The previous page found why the learned solutions broke glasses or did nothing. This page answers what was done about it. Most of the repairs that worked are one shared piece of arithmetic, which each solution runs over its own answer. The page explains that check first, then how it was tested, then the repairs that failed, and last the changes that worked only in pairs. Each solution's own repairs and results are in its own chapter. This page holds only what several solutions share.

1. The clearance check#

The previous page ended with hands that swept the rims of glasses. This section describes the check that stopped them.

bench/keeping_clear.py is a file of arithmetic in the shared bench. A solution runs it over its own answer, just before it hands the answer over. Nothing in the examiner calls it. A solution that does not call it is treated exactly as before. So the safety comes from the solutions, and the marking is not made any softer.

The rule says that an answer is one of two things.

A look. Every waypoint is at travel height, 300 mm up. Nothing on the table reaches that high, so the path cannot touch a glass however far it goes.
A push at one named glass. The hand moves across the table only at pushing height, where a shove goes through a glass's foot. That height is 75 mm, measured to the middle of the jaw. It comes down to that height straight, while the cell watches the force. The hand is then checked, stretch by stretch, against every glass the solution has measured. It may reach the glass it names and no other.

Anything else is raised to travel height. A raised path goes to the same places and costs the same arm time, but it touches nothing. Before the check, every measured glass is made 4 mm larger, because each reading carries the camera's error.

The first version of the rule had only the counting part. It asked whether a path reached one glass and no other. That was not enough. A path that sweeps across the table at rim height reaches exactly one glass, so it passed. Then it tipped that glass instead of sliding it. So the rule became a rule about the shape of a path too. A path crosses the table high or low, and nothing in between. Going straight down is allowed at any height, because the cell backs the hand off as soon as it touches something.

A side view of the two legal heights: a path either travels high above every glass, or pushes low at the foot of one named glass, and nothing travels between them.

In the picture, see the two heights, 300 mm and 75 mm, and that the band between them is empty. A path that travels in that band is the path the check raises.

The effect was large. On its first walk of the held-out queue, the borrowed model as it downloads broke 20 glasses. With the check, it breaks none. Its chapter gives the rest of its result.

The check worked for answers that are computed. It refused every answer that was drawn by a model. The next section answers why, and how that was fixed.

2. Snapping a path onto two heights#

The check in the section above asks the hand to travel at exactly pushing height. This section is about the solutions that could never meet that demand.

A geometry solution computes its path, so its waypoints sit at exactly pushing height. That is the number it worked from. A learned solution draws its path. It predicts the height, and its guess is out by a centimetre or two. So every push it drew travelled between the two legal heights, and the check refused it. Over twenty tables, the imitation solution asked for between 45 and 109 shoves, and every one was refused for travelling at rim height. This is recorded at the top of bench/keeping_clear.py.

The fix is to snap a failed path before refusing it. To snap a path is to move each of its waypoints onto one of the two legal heights. The function is squared_off, and it works in three steps.

Each waypoint goes up or down. The choice depends on the fingers. Suppose the fingers would be below the top of the named glass. Then the model meant to touch it, so the waypoint goes down to pushing height. Otherwise the waypoint was travelling, so it goes up to travel height.
A corner is put in. Where one stretch moves across the table and also changes height, it becomes two stretches: across at the higher height, then straight down. A jaw moving sideways while it comes down is how a rim gets clipped.
The snapped path is checked again. It must still reach its named glass and no other. If it fails, it is raised and becomes a look.

Snapping cannot make a badly aimed push well aimed, and it does not try. It only fixes the height. After this change, the imitation solution made the first pushes it had ever made. The two borrowed models needed the same step, for the same reason.

Height was the first hole in the check. The next section lists the other holes that were found and closed.

3. The other pieces of the check#

Once drawn pushes passed the height rule, they reached glasses. This section lists the pieces added after that, each for a fault the runs showed.

Each piece below says what it does, and why it was needed. The project's record of changes gives each one's measurement.

A push may not drive its glass into another. The check now works out where the pushed glass would land. It tests that glass's path against every other glass. A glass that already stands too close may still be moved, as long as it ends no closer than it began. Before this, a shove could slide its glass into a neighbour, and the glass that broke was one the hand never came near. Closing this hole changed very little. Driving one glass into another was not how most glasses were lost.
A push needs evidence that the glass slides. A shove that would move a glass is allowed only at a glass this solution has watched move. A shove that only comes to rest against a glass is allowed anywhere, because that is how the evidence is gathered. "Watched move" means moved further than the camera's error, which is the same 4 mm. Shoves in the same file keeps that record for each glass. Why it is needed: nothing can tell in advance that a glass will slide.
Each solution offers its own kind of evidence. The world model never probes. It predicts instead. So its evidence is its own prediction: the least hopeful of its three copies must keep the push under its topple limit. When the rule was first added without this, the world model lost tables it had been separating.
A push is carried on until its glass has room. This is the repair for the copied push that is smaller than the copier's error. driven_home takes a push that passed every check. It extends the push along the model's own line, one step at a time, at the spacing the model chose. It stops when the glass has room, or when the next step would fail a check. The model still chooses everything that is hard: whether to act, which glass, and which way. Only the distance is arithmetic. A run's record counts these extended pushes, so the split is visible in the numbers.
A push is carried no further than the glass has been watched sliding. A probe of 5 mm is evidence about a 5 mm push. It says nothing about a 60 mm push. So a glass seen sliding 5 mm earns a 5 mm extension, and one seen sliding 40 mm earns 40. Every glass the imitation solution lost had gone over on an extended push. This cap was chosen on the tuning tables. On the held-out queue its effect could not be told apart from noise.
The driving stretch is made straight. The cell turns the hand to each stretch's own direction as that stretch starts. A wandering push therefore turns the hand while the fingers press the glass, and a glass twisted at its foot goes over. The teacher's pushes are perfectly straight. The student's wandered by a few millimetres. Straightening them changed nothing that could be told apart from noise. It was kept because it makes the student's push the same shape as the teacher's.
The check uses every measured glass, not only the current picture. One picture from the wrist holds only part of the table. A neighbour out of frame is on no list, so a push could drive a glass into it. Measured keeps the newest reading of every glass seen on the table, as the belief describes. It helped most for the borrowed model, which aims worst, and least for the imitation solution, whose camera usually saw the whole small table anyway.

A repair is worth only the size of the hole it closes. Two of these pieces closed real holes that turned out to be small. That is worth knowing, so that the next effort goes to the large holes.

Every piece above can refuse an answer. The next section answers how the project made sure that a piece did not refuse good answers too.

4. The test against the teacher#

The pieces in the section above each add a reason to refuse. This section describes the one test that each new refusal had to pass.

The teacher is the ranked geometry solution, and it is known to be safe. So its own answers were put through the check without changing what it did. Nearly all of them passed exactly as drawn. A few were snapped onto the heights. None was refused for want of evidence, because the teacher always probes before it pushes. The project's record of changes gives the split.

The lesson is short. A safety rule that refuses the teacher's own answers is a broken rule. And the test is cheap to run.

The project learned this from a real mistake. One measurement once reported that most shoves came down with the hand already over the glass. It treated the hand as a long box reaching back from the fingertips. The question that matters is where the fingertips are against the glass's own circle, and that gave a different answer. A rule built on the wrong reading refused every push in the project, including the teacher's. That is how it was caught. When a new rule refuses the teacher, the measurement behind the rule is what is wrong.

The check and its test are the repairs that worked. The next section lists the ones that did not.

5. The repairs that failed#

The repairs above were kept. This section lists the ones that were built, measured and dropped, because a record of only the successes teaches little.

Each line below names one dropped repair and what it showed. The project's record of changes gives each measurement. The solution chapters say more about their own.

Shorter pushes for the world model, by a higher price per millimetre or a cap on length. Each saved a glass and cost many separated tables.
A probe before every long push. It cut the broken glasses, and it cut the separated tables by about half. A glass that did not keep up with a probe was then left alone.
A tighter limit on the predicted chance of toppling. It never bought anything, at any of the settings tried.
A probe only before the pushes the model's copies disagree about. At every threshold it cost tables and saved no glass.
Collecting extra training pushes where the copies disagree. Those pushes really do topple more often. Fitted on them, the model lost more glasses.
Refitting the world model on its own search's pushes. Its guess about toppling got much better. The solution did not.
A price on push length in the ranked trees. It changed nothing.
Correcting a stale input in the ranked trees. It helped when the arm and clock were taken away, and lost tables under the clock. The ranked chapter explains the input.
Bringing a shove down further back from the glass. The hand then ploughed further along the table, and more glasses went over.
Trying the gate's second choice of glass when the first is refused. It broke more glasses and lost the one separated table. Those second choices were glasses the model itself ranked low.
A solution noting its own broken glasses. This cannot work. The examiner ends a table the moment a glass goes over, so the look that would show it is never taken. The diagnosis now lives in the measuring scripts.
Three times as many training steps for the fine-tuned model. Its paths became more like the teacher's. Its results on the table did not change.

Two lessons cover most of these.

A threshold cannot refuse what the model gives no chance. The pushes that topple glasses are the pushes the model thought were safe. So making the limit tighter has nothing to catch. A small probe has the same problem. It reports what a small push does, and the glasses go over under long pushes.

Caution scores the same as doing nothing. A broken glass costs its whole table. But a table left crowded also scores nothing, and its glasses still count in the total. So a change that trades many unfinished tables for one saved glass is a bad trade. To be safer without being less useful, refuse specific unsafe answers, as the clearance check does. Do not make every answer smaller.

Some of the dropped repairs failed on their own but might have worked beside another change. The next section is about changes of that kind.

6. Changes that only work together#

The failures in the section above were each tested alone. This section shows that some changes fail alone and work only as a pair, so testing them one at a time would reject both.

The world model's two training changes. One change doubled the random pushes the model is fitted on. The other made a missed topple cost ten times as much during fitting. More pushes made the model aim better. They also made it less easily alarmed. The search uses a fixed limit, so a calmer model let more risky pushes through, and more glasses broke. The heavier cost alone made the model refuse so much that it freed far fewer glasses. Together, the model kept the better aim and got its caution back. On the tuning tables the pair separated more tables and lost fewer glasses. On the held-out queue it came out worse. The world model chapter gives the numbers, and why the change was kept anyway.

The imitation solution's chain of repairs. Splitting the solution into a part that decides and a part that draws made pushing learnable. Its chapter describes the split. Yet the first walk after the split still freed nothing. Its pushes were refused for their height until snapping was added. Then they moved their glass nowhere until they were carried on. No single one of these changes was enough. Each was needed, and each would have looked useless if it had been measured alone.

The lesson is that a change can be right and still show nothing until the change it depends on is made. The next page asks the questions this chapter answers, in short form.

← Why the learned ones failed · Questions and answers →