4 ms·

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with

by a2code 3y ago

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

phlipski 3y ago
NXP's new MCX N94x and MCX N54x microcontrollers both have dual Cortex M33 cores and an integrated NPU. Eval boards should be available for purchase any day now.
jononor 3y ago
There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks. The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn https://github.com/espressif/esp-nn Then there are RISCV chips with neural network co processors like Kendryte K210. ARM has also defined a new set of extensions for NN acceleration, and reference designs for cores being ARM Cortex M85. Chips are becoming available this year. ST has announced they will have accelerators in several lines. There are dozens of startups creating accelerator designs and trying to pair them with MCUs. So we have a bit already, with much more to come in the years to come.
- a2code 3y ago
  Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has. On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DSP rather than ML applications. Searching for SIMD returned no arithmetic instructions. Searching for parallel returned instructions for multiply and accumulate. Further, the FPU does not compute any kind of 16-bit floating point numbers. >ARM has also defined a new set of extensions for NN acceleration Can you provide some more info about this?
  - jononor 3y ago
    The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university/Arm-Helium-Technology https://github.com/arm-university/Arm-Helium-Technology There is another chip that is generally available, that has a CNN accelerator/co-processor - the MAX78000 https://www.embedded.com/hardware-conversion-of-convolutional-neural-networks/ https://www.embedded.com/hardware-conversion-of-convolutiona...
    - a2code 3y ago
      Thanks again. I have to correct my previous reply. The ESP32-S3 has an extended instruction set detailed in the technical reference manual. These include vector operations (8, 16, or 32 bit). I'm curious, why do you believe int8 will dominate?

by a2code 3y ago

Thanks. Let me be more specific. The ESP32 included WiFi on the same chip. Is there an MCU with on-chip features for AI? Perhaps an optimized TPU combined with an MCU. Would that be an advantage?

phlipski 3y ago

NXP's new MCX N94x and MCX N54x microcontrollers both have dual Cortex M33 cores and an integrated NPU. Eval boards should be available for purchase any day now.

jononor 3y ago

There are a range of ML acceleration possible on existing chips. The basic 4-wide 8 bit integer SIMD extensions in NEON is available on basically all ARM Cortex M4F chips, which is already available 8+ years. It gives 4-5x speedup for neural networks. The more recent ESP32-S3 has operations with up to 10x speedup, see https://github.com/espressif/esp-nn https://github.com/espressif/esp-nn Then there are RISCV chips with neural network co processors like Kendryte K210. ARM has also defined a new set of extensions for NN acceleration, and reference designs for cores being ARM Cortex M85. Chips are becoming available this year. ST has announced they will have accelerators in several lines. There are dozens of startups creating accelerator designs and trying to pair them with MCUs. So we have a bit already, with much more to come in the years to come.

a2code 3y ago
Thanks for the reply. I did not find public documentation for Kendryte, only a Github repository. At least the code is in English. But the AI examples include an "nncase" library which I could not find on the repository. So I could not see the instructions their accelerator has. On the other hand, esp-nn seems to be code for the xtensa instruction set. I briefly overviewed the instructions. They seem optimized for DSP rather than ML applications. Searching for SIMD returned no arithmetic instructions. Searching for parallel returned instructions for multiply and accumulate. Further, the FPU does not compute any kind of 16-bit floating point numbers. >ARM has also defined a new set of extensions for NN acceleration Can you provide some more info about this?
- jononor 3y ago
  The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university/Arm-Helium-Technology https://github.com/arm-university/Arm-Helium-Technology There is another chip that is generally available, that has a CNN accelerator/co-processor - the MAX78000 https://www.embedded.com/hardware-conversion-of-convolutional-neural-networks/ https://www.embedded.com/hardware-conversion-of-convolutiona...
  - a2code 3y ago
    Thanks again. I have to correct my previous reply. The ESP32-S3 has an extended instruction set detailed in the technical reference manual. These include vector operations (8, 16, or 32 bit). I'm curious, why do you believe int8 will dominate?