scott-gray

7 ms·

1. 1.▲
   by scott-gray 9y ago

   So a minor correction here. We did explore placing ftz on various instructions inside the matmul ops, but it turns out you don't need anything more than what is already baked into tf by default. All tf gpu primitives are built with -
2. 2.▲
   by scott-gray 9y ago

   Actually if you read section 4.6 of that paper you'll see that denormals are the default on sm_20 and above. But you can see in that same section this this can easily be disabled with the ftz flag. I had to give Jakob custom gemm kern
3. 3.▲
   by scott-gray 10y ago

   I was aware of this deal before leaving. I made out pretty well with my vested shares but was more interested in working on the cutting edge of research than in continuing on with pure hardware optimization and design. I wish Nervana&#x2F