While the github contains the code, the article describing the optimisations are here: https://machinelearning.apple.com/research/neural-engine-transformers htt

While the github contains the code, the article describing the optimisations are here: https://machinelearning.apple.com/research/neural-engine-transformers https://machinelearning.apple.com/research/neural-engine-tra.... TL;DR: execution of pytorch models on apple's neural engine and standard data-oriented optimisations (changing matrix layout, chunking to optimise temporal cache locality, and minimising redundant memory copies)