This page answers the questions a reader most often asks about this solution. Each answer is short. Each one links to the section that explains it in full.
Contents#
- What does solution 6 change about solution 5, and why only that one thing?
- What is fine-tuning?
- What is low-rank adaptation, and why is changing a few weights enough?
- What is catastrophic forgetting, and why is it less likely here but still possible?
- Does fine-tuning make the solution slower to run?
- Why did six and a half times as many demonstrations change nothing?
1. What does solution 6 change about solution 5, and why only that one thing?#
It adds training on pushes made in this cell. The model, the weights, the picture and the scorecard stay the same. Only one thing changes, so the gap in score measures what the training bought. Whether anything but the training differs checks this.
2. What is fine-tuning?#
Fine-tuning means you start from weights that someone else trained. Then you train them a little more on your own examples. The model keeps what it knew and moves towards your task. Fine-tuning explains it in full.
3. What is low-rank adaptation, and why is changing a few weights enough?#
Low-rank adaptation leaves every borrowed weight as it is. It learns a small correction beside them, here about four million numbers out of about 450 million. That fits on a laptop. It may not be enough for a task this far from the old ones. Low-rank adaptation explains this.
4. What is catastrophic forgetting, and why is it less likely here but still possible?#
A model trained on a new task tends to lose what it knew before. Here the borrowed weights are never overwritten, so removing the correction gives the old model back. But the corrected model is the one that runs, and it is worse on other robots. Two ways training goes wrong explains this.
5. Does fine-tuning make the solution slower to run?#
No. After training, the correction is added into the borrowed weights. So the trained model does the same work as solution 5's model. Low-rank adaptation explains why. One run looked slower, but it shared the graphics chip with another fit.
6. Why did six and a half times as many demonstrations change nothing?#
The model fails because its jaw comes down onto a glass. The teacher never does that. So no demonstration shows the way out, however many there are. More of the same examples do not teach a new state. More demonstrations gives the measurement.
← How it compares · A newer, larger model fine-tuned here — what it is →