Retrospectively Reverse-Engineering Apple's Neural Engine

LoganDark

> what workloads it was designed for and accels at. excels!

CraigJPerry

This isn't ai slop. It's fascinating and well written. But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.

zozbot234

How does this relate to the more recent work on the M4 ANE found at https://maderix.github.io/articles/ ? Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of the same thing? As an aside, the introduction to this article seems to conflate the ANE with the Neural Accelerators (NAX) found in the M5+ (and A-series equivalents) GPUs. These are very different things, and Apple is still working on the ANE - the M6 and A20 will apparently feature doubled ANE blocks.

hn9zmdcaou

Ported a transformer to ANE and the whole job was pretending it was a CNN, 4D tensors with seq in the last axis and 1x1 convs instead of matmuls.

anentropic

Amazing analysis Same author even found a bug in it https://eiln.github.io/posts/ane-dma.html

GeekyBear

It's worth remembering that Apple is releasing a new framework (Core AI) this fall that goes beyond the Pytorch and Tensorflow workloads that the decade old Core ML framework allowed. > Core AI allows your app to use the latest model architectures and inference techniques across the CPU, GPU, and Neural Engine. https://developer.apple.com/documentation/coreai

throw0101a

A lot of folks are/were saying that Apple has missed the boat when it comes to AI, in some (important) aspects that is correct, but I think it's worth remembering that Apple added the Neural Engine to A-series chips in 2017, before the AI hoopla really kicked off: * https://en.wikipedia.org/wiki/Neural_Engine * https://apple.fandom.com/wiki/Neural_Engine "AI" has grown much more since then, and there have been important developments that Apple has not deployed (well), but I think they were looking ahead a little more than most at the time (even if events 'got away' from them subsequently).

brcmthrowaway

I thought random developers weren't even allowed to use ANE, and it's just do the computational photography DoF faking.