0%

Chapter 9 · Section 2 of 4

What it needs

2 min read59 of 62 in Pushing the Glasses Apart
On this page

This page lists what solution 7 costs. Every number here was measured while the project built and fitted it. The fits did not learn the task, so there is no score. The results say why.

Contents#

  1. What it needs
  2. The measured costs

1. What it needs#

What it is said what the model is. This section lists the parts that must exist before it runs.

  • Solution 6's demonstrations. Solution 7 trains on that folder. So it must be made first.
  • The Evo-1 weights. They download once, the first time training runs.
  • About 100 minutes of training for each seed. A seed is the number that starts the random parts of one training run. There are three seeds. The fitting is done by train.py.

It does not need an NVIDIA graphics card. Evo-1's code names NVIDIA's CUDA system in three places. None of them stops it running on this Mac. Three settings were enough, and no code was changed.

The table lists each setting and the reason for it.

settingwhy
flash attention offan NVIDIA-only part; the model uses ordinary attention instead
mixed precision offthe model already turns it off without CUDA
32-bit numbers, not bfloat16a result should not rest on Metal's bfloat16

The project does not state the licence of the Evo-1 weights. Read their terms before anything trained from them leaves the project.

The next section gives the measured costs.

2. The measured costs#

The parts are listed. This section says how much memory and time the model takes.

The table has one row for each measurement. Most numbers come from the project's README for this solution. The training rows come from each fit's training.json.

numbers in the model775.2 million
numbers fitted122.0 million
memory held while answering3.60 GiB
one forward pass1.7 s
one backward pass0.1 s
memory held while training4.26 to 4.36 GiB
one fit of 1,000 steps98 to 100 minutes

A forward pass runs a picture through the model. A backward pass works out how to change the fitted numbers. Solution 6 fits only 3.8 million numbers, so solution 7 changes about 30 times as many.

The next page, the results, says what the three fits reached.

← What it is · Results →