This page is the bill for this solution. It lists the data it needs, the machine it wants, the libraries it uses, and their licences.
1. What it needs#
The worked example showed the solution at work. This section lists what it takes to build and run it.
It needs a physics engine and thousands of pushes in it. This is the main
cost. Every training example is one real push in the simulator: look, push,
look again. The project's
README.md
for the first way records 15,379 of them, collected in two rounds:
- 7,686 random pushes on 4,000 tables;
- then 7,693 pushes on 5,000 new tables. Three quarters of these were chosen by the planner, using the first round's model. This fills the gaps the search would otherwise use.
Every push is also used mirrored, so the 15,379 pushes give 30,758 examples.
Nobody labels any of it. The answer for each example is the difference between two looks, one before the push and one after. So the model learns to predict what the camera will report next, with its error. It does not learn what the world really does. This is the right choice, because the camera's next report is what the planner will see. A model that predicted a truth the arm can never see would predict the wrong thing. It also means the data costs simulator time and no human time. This is the biggest practical advantage over any method trained on demonstrations.
It needs a training run before it can answer anything. The first way trains five small networks. The same README says the pushes and the training together take about half an hour on a laptop processor. The first way needs no accelerator at all. This is because the model is small, and its input is 34 numbers, not a picture.
The second way needs rented hardware. TD-MPC2 is reinforcement learning, and it needs an accelerator. Renting one for a weekend costs about a hundred dollars, which covers one training run. One run is not a measurement, so several training seeds are needed. The realistic cost is a small accelerator for about a month, which is about five hundred dollars.
It needs computing time on every push. Before each push, the search asks
the model about 1,500 candidates for every crowded glass. Each candidate goes to
five networks. The topple question is asked on five readings of the table. The run measured 0.37 seconds
of thinking per push, in
results.json.
The machine was busy with other work, so this number is only rough. A real arm
movement takes longer than this, so the arm does not wait. A two-push search
would cost much more. The results compare
this time with the other five solutions.
It needs weights that match the cell. The weights record what the cell was like when they were trained. Suppose the jaw, the glass zone, the shapes of the glasses or the camera's error changes. Then the weights are out of date, and no test of the code will notice.
It needs libraries, and no borrowed model. Both ways use PyTorch, and MuJoCo through the examiner. The second way needs TD-MPC2. The library this project uses elsewhere has only the earlier method, TD-MPC, and not TD-MPC2. So TD-MPC2 would have to be fetched from its own project. Neither way downloads trained weights, so there is no model licence to meet. Only the libraries' own licences apply, and TD-MPC2's licence is MIT.
Its score has no spread. The first way was trained once and run once. So its results give the time per push but no spread across runs. The test examiner explains why one run is not a full measurement.
The next page weighs these costs against what the solution achieves.