andrew-wja

5 ms·

1.
by andrew-wja 8y ago
We have some updated numbers, including benchmarking on Kaby Lake (AVX2): https://www.scss.tcd.ie/~andersan/projects/live/triNNity.htm... I believe I will have access to a Skylake-X machine in the next few da
2.
by andrew-wja 8y ago
Primarily it's down to the fact that most programs have control flow! As you most likely know, DAG selectors have to tile the CFG (which can have back-edges due to control flow) into a forest of DAGs, which are then fed to the instruct
3.
by andrew-wja 8y ago
Thanks for the question, this really gets to the heart of the issue! We do see a fundamental advantage, and I'll try to explain what it is. So TC's kernel evolution is based on autotuning, and what they are autotuning for is the p
4.
by andrew-wja 8y ago
The short answer is that there are dozens of ways to use that GEMM to do convolution (convolution algorithms), and there are NUM_ALGORITHMS * NUM_LAYERS way to implement the network. Our toolkit figures out which of those arrangements are
5.
by andrew-wja 8y ago
You're essentially correct, but there is a bit of a problem with PyTorch and TF specifically, because you don't really have a definition of the model per se. You construct it dynamically using a Python or C++ program. The Caffe .p
6.
by andrew-wja 8y ago
cuDNN's autotuner is only making local decisions; If layer X is connected to layer Y is connected to layer Z, simply choosing the fastest algorithm to implement each layer does not guarantee optimality. That's because the differ
7.
by andrew-wja 8y ago
I just realised the implication in your last question is that a heuristic solution to the PBQP problem is obtained. In fact, for a lot of networks, we get the optimal solution :) DNN DAGs are big, but they are very simple structurally com
8.
by andrew-wja 8y ago
Full disclosure: I'm a compiler person who for funding reasons moved into performance of machine learning systems. None of those things should be called compilers. At best, they are scaffolding for peephole optimization. When you can g
9.
by andrew-wja 8y ago
Yes, that's a big part of it. Also, if you want to do something like (for example) keyword spotting in audio on a small device, like a Cortex-M class processor, the constraints are really really difficult to satisfy: most of them have
10.
by andrew-wja 8y ago
I should mention that we have made Arch Linux packages for all this stuff: trinnity-git and trinnity-optimizer-git (you can just git clone the compiler and the demos to get started). We'd love people to start using and breaking this st
11.
by andrew-wja 8y ago
In the compiler we need to add backends to cuDNN and probably some OpenCL DNN framework -- this is already fairly high on our list of priorities.
12.
by andrew-wja 8y ago
Yes, both! We also are working on ARM Compute and cuDNN backends. Soon :)
13.
by andrew-wja 8y ago
They aren't tied at all -- in fact the optimizer is a totally separate project (triNNity-optimizer) that just does graph optimization. You can add user defined kernels as long as you have some way of microbenchmarking them!
14.
by andrew-wja 8y ago
No changes are required in the library -- you just need to have some way of generating the code for the forward pass using e.g. cuDNN (which already has a heuristic selector!)
15.
by andrew-wja 8y ago
The speedup is for the whole network, as the graph labels show! The point of the compiler is that you produce code that implements the entire forward pass so you can deploy that code where you need to do the inference. I agree we ne
16.
by andrew-wja 8y ago
Hi, author here. There is a staggering amount of low hanging fruit. I have been half-seriously blaming GEMM in correspondence. When you have a problem that looks like GEMM, it's such an attractive hammer to pick up that people just don
17.
by andrew-wja 8y ago
Updated to clarify. Part of the toolkit is written in Haskell, I really oughta know better!
18.
by andrew-wja 8y ago
Fair! Edited.
19.
by andrew-wja 8y ago
If you really want to know more right now , you can read our paper and look at the slides here: https://www.scss.tcd.ie/~andersan/pubs/cgo-2018.html
20.
by andrew-wja 8y ago
If anyone would like to know more about the toolkit, I'll be checking the comment thread here periodically today!
21.
Show HN: High-performance ahead-of-time compiler for Machine Learning(scss.tcd.ie)
83 points by andrew-wja 8y ago|35 comments
22.
Show HN: High-performance header-only C++ template library for CNNs(scss.tcd.ie)
8 points by andrew-wja 9y ago|0 comments
23.
Header-only C++ template library with over 70 DNN convolution algorithms(scss.tcd.ie)
4 points by andrew-wja 9y ago|0 comments