Power, then heat: what limits the ET-SoC-1's matmul speed

← AI Foundry lab dashboard › Worklog

Experiment E64 (request Q123), run Fri 23 Oct 2026 on aifoundry2, pre-registered in tools/claims-v3/hpl before the first pass; data.

Horace He found that an A100 runs the same matmul about 15% faster on zeros than on random numbers, because the GPU sits at its power cap: data that flips fewer transistors draws fewer watts and earns a higher clock. The ET-SoC-1's firmware has two caps, 65 W of card power and 65 °C on the die, so the same effect could come from either. This experiment pulls them apart on aifoundry2's card and reads every clock step's cause from the service processor's own log.

The answer

Both limits act, each on its own, and which one sets the speed depends on the die's temperature. On a die below 65 °C only the power test acts: every one of the 10 runs whose work drew 67 W or more at 800 MHz was stepped down by the power test first (PASS, rule R1), and none of the 10 runs under 62 W was (PASS, R2). Random normal data on the full card draws about 82 W at 800 MHz, so the governor pulls it back to a mean of 710 MHz, while zeros (35 W) and ones (50 W) stay at the 800 MHz ceiling: Horace's effect, by Horace's mechanism, 13% here against his 15%. On a die above 65 °C only the thermal test acts, even for zeros at a third of the power cap (PASS, R3). For sustained work on this card, heat wins: over 80 s the die settles where its cooling puts it, and on this card the 65 °C line sits at about 50 W of steady board power, below the 65 W level; 5 of the 5 sustained runs that left 800 MHz spent most of that time under the thermal test and 0 under the power test (PASS, R4); zeros ran at 800 MHz for the whole run at 59 °C and 36 W, held back by nothing but the clock table; so a sustained matmul runs 33% faster on zeros than on random data (12.3 against 9.2 fp32 TFLOPS): Horace's effect again, wider, and set by heat.

What was known before

The Horace experiment (September) ran this kernel at a fixed 600 MHz on a hot die and found the same arithmetic drawing 38.3 W on zeros, 46.7 W on ones and 63.4 W on random normal data. From a 62–63 °C die it saw the speed effect too: 11.4–11.8 TFLOPS on zeros against 9.3 on random data, about 25%. But the limits were tangled then. The card idled above 65 °C, random data at 800 MHz touched 88 W and took the die past 65 °C within half a second, and ones (56 W, under the power level) lost 800 MHz to the thermal test alone. Since 9 October a fan bracket keeps aifoundry2's idle die at about 50 °C. So the two limits can now be pulled apart without changing anything on the card.

What the specifications say

Esperanto presented the chip at Hot Chips 2021 as 1,088 RISC-V cores that typically draw under 20 W and "can be adjusted for 10 to 60+ watts under SW control", with an operating range of 300 MHz to 2 GHz (slides; its energy-versus-voltage curve, by its own footnote 6, compares different physical designs, not one chip). The card in the lab is narrower. Its operating-point table has three points, 600, 700 and 800 MHz at 0.517, 0.568 and 0.618 V, so 800 MHz is a hard ceiling. And its firmware (open source, thermal_pwr_mgmt.c in et-platform) has a software governor with two limits that are set, not physical: a 65 °C threshold on the integer mean of 34 on-die sensors, tested first, and a 65 W TDP level, plus a 75 W / 75 °C alarm in the power-management chip. A step down when either is exceeded, a step up when neither is, no dead band.

So a matmul on this card can be held back three ways: by the clock table's top (800 MHz), by the power level, or by the temperature level. The runs below separate them.

On a cool die: the power limit alone

Ten fp32 operand patterns ran on all 1,024 minions, and random normal data on 768, 640 and 512, each for 6 s from a die reading of 55 °C or less, twice in shuffled order. The clock left 800 MHz exactly when the work crossed the power level, whatever made up the watts: random normal data on 768 minions (69 W) was stepped down, on 640 (62 W) it was not, and ones and random powers of 2 under 65 W ran at the ceiling with no power step at all.

The mechanism is a dither. Once a ~1.3 s average of the card's power passes 65 W the governor steps down one point per pass (about 0.24 s), and once it falls under, steps up again, with nothing in between to stop it. For random normal data the card goes 800 → 700 → 600 → 700 → 800, round and round, drawing about 83, 65 and 50 W at the three points, and the average sits on the level. The governor logged its power reading at each of 98 power steps down: 65.01 to 67.85 W, while the board's own reading at 800 MHz was 82 W. A model of that loop, a first-order average with a 1.3 s time constant compared against 65 W once per pass, reproduces the mean clock of all 26 cool-die runs to 3.3 MHz rms.

On a hot die: the temperature limit alone

Work that draws less than 65 W even at 800 MHz, ones and zeros, was launched on a die heated to 63–67 °C. The log of those 12 runs holds 55 thermal steps, every one at a reading of 66 °C, and 0 power steps. Runs that started above the line held 600 MHz until the die read 65 °C (4 of 4), then climbed, heated, and were sent back down; every run that reached 66 °C at 800 MHz was stepped down by the thermal test and not the power test (4 of 4). Zeros, at about a third of the power level, lost speed to heat alone.

Sustained: which limit holds a long matmul

Each number type ran for about 80 s: nine 8 s launch processes back to back from a die of 57 °C or less. A run whose power at 800 MHz is over the level starts in the power dither, but the die keeps warming, and once it reads 66 °C the thermal test takes over: fp32 random data first read 66 °C 8 s in, fp32 ones, under the power level, 28 s in. From then on the thermal test sets the clock. For random fp32 and fp16 data it holds 600 MHz, the bottom of the table, and the die stays above the line anyway (67 °C): the governor has no lower step to take. Only work light enough to keep the die under 65 °C for good holds the ceiling.

• fp32 random normal: 600 MHz in the last 40 s, 0% of it at 800 MHz, the rest 100% thermal and 0% power; die 67.1 °C, board 54.4 W
• fp16 random normal: 600 MHz in the last 40 s, 0% of it at 800 MHz, the rest 100% thermal and 0% power; die 66.1 °C, board 51.3 W
• int8 random (uniform over −128 to 127): 763 MHz in the last 40 s, 66% of it at 800 MHz, the rest 100% thermal and 0% power; die 65.3 °C, board 50.7 W
• fp32 ones: 738 MHz in the last 40 s, 48% of it at 800 MHz, the rest 100% thermal and 0% power; die 65.4 °C, board 49.4 W
• fp16 ones: 737 MHz in the last 40 s, 48% of it at 800 MHz, the rest 100% thermal and 0% power; die 65.4 °C, board 49.3 W
• int8 ones: 800 MHz in the last 40 s, 100% of it at 800 MHz; die 61.7 °C, board 41.8 W
• fp32 zeros: 800 MHz in the last 40 s, 100% of it at 800 MHz; die 59.0 °C, board 36.4 W
• fp16 zeros: 794 MHz in the last 40 s, 97% of it at 800 MHz, the rest 0% thermal and 0% power; die 59.0 °C, board 36.1 W
• int8 zeros: 800 MHz in the last 40 s, 100% of it at 800 MHz; die 59.0 °C, board 37.6 W

The line through the nine runs' last 40 s gives the cooling: the die reading rises 0.47 °C per watt of board power, and reads 65.5 °C at 50 W. That is the sustained power the 65 °C test allows here, and it is below the 65 W level: on aifoundry2's cooling the temperature limit is the tighter of the two for anything that runs long.

What speed to expect, by number type

fp32 and fp16 matmuls retire one 16×16 tile operation every 546 cycles and int8 every 318, so at the 800 MHz ceiling the full card does 12.3 fp32 TFLOPS, 24.6 fp16 TFLOPS and 84.4 int8 TOPS. What it actually delivers depends on the data and on how long it has been running:

Under other power levels the governor model gives the speed each kind of number would hold on a cool die, from that kind's own measured power at 600, 700 and 800 MHz (a model: the level was not changed on the card):

TFLOPS for fp32 and fp16, TOPS for int8, on all 1,024 minions, from the modelled mean clock after the start ramp.

The surface

Put the two axes together: the work's power at 800 MHz (the data: zeros at the left, random normal data at the right) and the power level the governor is given. The fitted governor model gives the mean clock for every pair. Under the card's 65 W the measured runs fall on the model's row. Every 10 W of extra level buys random fp32 data about 60 MHz of mean clock until it reaches the 800 MHz ceiling near 85 W, and on this card the 65 °C threshold sets a lower effective level for sustained work (about 50 W, the amber line). This part is a model: the level itself was never changed (the lab's rule: no TDP, threshold or clock is set on a shared card).

How it was run

• Pre-registered before the first card pass: design, predictions and rules R1–R4 in the README. Amendments: A0, a use-count check that read the runner's own sampler as another user; A1, the trace reader (the lab's earlier one stopped at the first binary record and lost most governor lines); A2, the hot starts (the heater cannot take this card's die past about 67 °C, because the two limits hold it there); A3, the placement of the log's lines on the host clock (the ring keeps only its last ~67 entries, so a dump covers a short run's last few seconds and a sustained run's last 5–60 s, and is anchored at the run's end).
• Nothing was set on the card. The TDP and threshold read 65 W and 65 °C after every run.
• The cause of each step comes from the service processor's 4 KB trace ring, dumped after every run: its CRITICAL lines ("Power throttle down event, current pwr … tdp level …", "Thermal throttle down event, current temperature …"), its "New OP" line at every operating point set, and a 16-byte power-status record (power, throttle state, temperature, target clock), placed on the host clock by matching the operating-point lines to the sampler's clock changes. The ring holds about 67 entries, three per clock step, so the counts above are of the steps it kept; where it had wrapped, the sampler's die readings still say which side of 65 °C each step was on.
• The kernel is the Horace experiment's: one 16×16 TensorFMA per minion on tiles in its L1 scratchpad, back to back. Speeds come from the kernel's own cycle counters.
• Claimed and released in #community-lab; 47 runs, about 32 minutes of card time.