5 ms·

FTA > ... in the coming weeks, they will drive new search algorithms based on deep neural networks—artificial intelligence modeled on the structure of the huma

by gradys 10y ago

FTA > ... in the coming weeks, they will drive new search algorithms based on deep neural networks—artificial intelligence modeled on the structure of the human brain—executing this AI several orders of magnitude faster than ordinary chips could.

aab0 10y ago
That raises as many questions as it answers. Why FPGAs and not GPUs which can run just about any deep neural network but usually faster and more efficiently?
- Const-me 10y ago
  One reason is FPGAs are more flexible. Sure, GPUs deliver impressive raw performance. To be useful, the task must benefit from massively parallel hardware. GPU hardware works fantastic for shading polygons, training neural networks, or raytracing. For compression and encryption algorithms however, GPUs aren’t terribly good. Another reason is while a GPU delivers impressive bandwidth on parallel-friendly workloads, it’s usually possible to achieve lower latencies with FPGA. An FPGA doesn’t decode any instructions, and its computing modules exchange data directly.
- Eridrus 10y ago
  GPUs give you great throughput, but they're expensive, eat a lot of power, only work well in batch and aren't tuned to prediction (eg poor INT8 performance and too much VRAM).
- emcq 10y ago
  GPUs have worse performance per watt than a tuned FPGA. Some newer FPGAs can have 400 megabits of on chip RAM - that's huge, significantly larger than the 128-256k cache typically available on chip for a GPU that turns into big energy savings.
  - p1esk 10y ago
    GPUs have worse performance per watt than a tuned FPGA Citation needed. Maxwell Jetson TX1 is claimed to achieve 1TFlops FP16 at <10W, and soon to be released Pascal based replacement will probably be even more efficient.
    - emcq 10y ago
      While I dont have any external publications addressing this general claim, this is taken from my current and past experiences with internal studies focused on neural networks implemented on the TX1, other GPU, custom ASICs, and FPGA approaches. In terms of power efficiency it generally goes ASIC > FPGA > GPU > CPU. If you're doing just fp32 BLAS it's hard to beat a GPU, but it turns out many problems have features that you can optimize for. The TX1 power consumption including DRAM and other subsystems peaks 20-30W. Typical usage is 10-15W if you're running anything useful. That 1 TFLOP counts a FMA instruction as 2 flops - while accurate and useful for say dot products - for other workloads the throughput will be half of this number. As an example of an FPGA performing significantly better than the TX1 is DeepPhi [0]. [0] http://www.deephi.com/en/technology/ http://www.deephi.com/en/technology/
      - p1esk 10y ago
        In that link, where's the comparison of fpga vs TX1?
        emcq 10y ago
        If you click on Papers, there is a link to "Going Deeper with Embedded FPGA for Convolutional Neural Network", which compares against the TK1: https://nicsefc.ee.tsinghua.edu.cn/media/publications/2016/FPGA2016_None_6tAJnDW.pdf https://nicsefc.ee.tsinghua.edu.cn/media/publications/2016/F... While not the TX1 vs FPGA result you want, this is very close. For example they aren't using the latest FPGA or GPU, and are not using TensorRT on the GPU and on the FPGA side they are using fatty 16-bit weights on an older FPGA rather than newer stuff you can do with lower precision (which improves the efficiency of the FPGA having more high speed RAM collocated with computation vs GPU which is primarily off-chip). If you want to learn more about this stuff, I suggest a presentation by one of Bill Dally's students (chief scientist at NVIDIA): http://on-demand.gputechconf.com/gtc/2016/presentation/s6561-song-han-deep-compression.pdf http://on-demand.gputechconf.com/gtc/2016/presentation/s6561...

by gradys 10y ago

aab0 10y ago

That raises as many questions as it answers. Why FPGAs and not GPUs which can run just about any deep neural network but usually faster and more efficiently?

Const-me 10y ago
One reason is FPGAs are more flexible. Sure, GPUs deliver impressive raw performance. To be useful, the task must benefit from massively parallel hardware. GPU hardware works fantastic for shading polygons, training neural networks, or raytracing. For compression and encryption algorithms however, GPUs aren’t terribly good. Another reason is while a GPU delivers impressive bandwidth on parallel-friendly workloads, it’s usually possible to achieve lower latencies with FPGA. An FPGA doesn’t decode any instructions, and its computing modules exchange data directly.
Eridrus 10y ago
GPUs give you great throughput, but they're expensive, eat a lot of power, only work well in batch and aren't tuned to prediction (eg poor INT8 performance and too much VRAM).
emcq 10y ago
GPUs have worse performance per watt than a tuned FPGA. Some newer FPGAs can have 400 megabits of on chip RAM - that's huge, significantly larger than the 128-256k cache typically available on chip for a GPU that turns into big energy savings.
- p1esk 10y ago
  GPUs have worse performance per watt than a tuned FPGA Citation needed. Maxwell Jetson TX1 is claimed to achieve 1TFlops FP16 at <10W, and soon to be released Pascal based replacement will probably be even more efficient.
  - emcq 10y ago
    While I dont have any external publications addressing this general claim, this is taken from my current and past experiences with internal studies focused on neural networks implemented on the TX1, other GPU, custom ASICs, and FPGA approaches. In terms of power efficiency it generally goes ASIC > FPGA > GPU > CPU. If you're doing just fp32 BLAS it's hard to beat a GPU, but it turns out many problems have features that you can optimize for. The TX1 power consumption including DRAM and other subsystems peaks 20-30W. Typical usage is 10-15W if you're running anything useful. That 1 TFLOP counts a FMA instruction as 2 flops - while accurate and useful for say dot products - for other workloads the throughput will be half of this number. As an example of an FPGA performing significantly better than the TX1 is DeepPhi [0]. [0] http://www.deephi.com/en/technology/ http://www.deephi.com/en/technology/
    - p1esk 10y ago
      In that link, where's the comparison of fpga vs TX1?
      - emcq 10y ago
        If you click on Papers, there is a link to "Going Deeper with Embedded FPGA for Convolutional Neural Network", which compares against the TK1: https://nicsefc.ee.tsinghua.edu.cn/media/publications/2016/FPGA2016_None_6tAJnDW.pdf https://nicsefc.ee.tsinghua.edu.cn/media/publications/2016/F... While not the TX1 vs FPGA result you want, this is very close. For example they aren't using the latest FPGA or GPU, and are not using TensorRT on the GPU and on the FPGA side they are using fatty 16-bit weights on an older FPGA rather than newer stuff you can do with lower precision (which improves the efficiency of the FPGA having more high speed RAM collocated with computation vs GPU which is primarily off-chip). If you want to learn more about this stuff, I suggest a presentation by one of Bill Dally's students (chief scientist at NVIDIA): http://on-demand.gputechconf.com/gtc/2016/presentation/s6561-song-han-deep-compression.pdf http://on-demand.gputechconf.com/gtc/2016/presentation/s6561...