Apple Intelligence Servers Expected to Start Using M4 Chips Next Year After M2 Ultra This Year

Actually this is a good sign for people want even faster Mac Studio and Mac Pro if Apple is building chips for their own AI server, which could lead to more resources being put into making those high performance chips.

But seriously, the base M4 NPU outperforms the M2 Ultra. The upgrade is well deserved.

The base M4 does not outperform the M2 Ultra in NPU function, though if you're not familiar with how TOPS ratings are determined it's easy to misunderstand. The short of it is you need to know what operation the TOPS rating is measuring when comparing.

M1 - M3 Neural Engines were measured using FP16 operations, whereas the M4 chips (and A17 and A18) are measured using INT8 operations. FP16 operations handle about twice as much data per operation than INT8. They're not entirely interchangeable but 20 FP16 operations would equalize out to about 40 INT8 operations.

The M2 Ultra is rated at 31.6 TOPS in FP16, which would be equate to roughly 62-64 TOPS in INT8. The M4 is rated at 38 TOPS in INT8.

Similar confusion occurred with the M3. Apple measured the M3 Neural Engine with FP16 but the corresponding A17 Neural Engine was measured with INT8 for whatever reason, thus making it seem that the A17 had a faster NPU than the M3 when they were essentially the same. The M4 looks like a huge leap over the M3 on paper because of the TOPS figure, but it's actually only about 5-10% faster. The M2 was actually the biggest boost to NPU performance in the four generations of M chip, about 40% faster than the M1.

For the record this is not Apple being sneaky, they made the change because AMD, Intel, and other companies coming out with NPU hardware are measuring in INT8 and it's become something of the de-facto standard benchmark for NPUs. Apple, with good reason, didn't want their NPUs specs to look worse because of a reason like that.

Given the scale of Nvidia GPU sales I'd like to hope the Apple gets back into the server game since there is a lot of money to be made on high memory inference hardware. Assume the M4 Ultra is able to come with 384 GB of memory each it will be quite competitive for the 400B+ parameter models.

Did Apple upgrade to M4 because the base memory was increased? 😄

But seriously, the base M4 NPU outperforms the M2 Ultra. The upgrade is well deserved.

I oughta say, the M4 feels like the very first actual successor to the M1.

I just want to know when this guy's coming back.