6 ms·
Pretty neat implementation. In general, for these sort of exercises (and even if the intention is to go to prod with custom kernels) I lean towards Triton to wr
by treesciencebot 3y ago
Pretty neat implementation. In general, for these sort of exercises (and even if the intention is to go to prod with custom kernels) I lean towards Triton to write the kernels themselves. It is much more easier to integrate to the tool chain, and allows a level of abstraction that doesn't affect performance even a little bit while providing useful constructs.
- queuebert 3y agoAs a person who finds CUDA extremely easy to write and integrate, what does Triton have to offer?
- whimsicalism 3y agoblock level rather than thread level programming, automatic optimization across hyperparameters, makes it much easier to write fast kernels
- ixaxaar 3y agoYou mean triton the inference server or triton the DSL for cuda?
- whimsicalism 3y agothey mean the dsl (not just necessarily for cuda)
- p1esk 3y agoThe DSL: https://openai.com/research/triton https://openai.com/research/triton
- treesciencebot 3y agotriton the DSL.
- whimsicalism 3y agoyeah even the official flashattention is moving many implementations from cutlass to triton except for the main mha backward/forward pass
- jart 3y agoIt was written with cutlass? No wonder Peter Kim found it valuable and worthwhile to de-obfuscate. Adopting a new programming language invented by OpenAI doesn't sound like a much better alternative. I'd be shocked if either of them were able to build code for AMD GPUs, where it's easy to adapt CUDA code, but not if it's buried in tens of thousands of lines of frameworks. I like open source code to have clarity so I can optimize it for my own production environment myself. When people distribute code they've productionized for themselves, it squeezes out all the alpha and informational value. Just because something's open source doesn't mean it's open source. I think people mostly do it to lick the cookie without giving much away.
- synquid 3y agoTriton has an AMD backend, although work is still ongoing.
- imtringued 3y agoYou will also be able to use Triton to target Ryzen AI.
- fpgamlirfanboy 3y ago> allows a level of abstraction that doesn't affect performance even a little bit The second part of this sentence is true because the first part is false.
- treesciencebot 3y agozero cost abstractions exist. doesn't mean all abstractions are zero-cost. or being zero-cost somehow invalidates their abstractness/genericness. but maybe we differ on the definition of abstractions.
- deleted 3y ago[deleted]
- fpgamlirfanboy 3y ago> zero cost abstractions exist So does perpetual motion :shrug: but my point is Triton is not an abstraction in the least. Source: 1) I spent 6 months investigating targeting other backends 2) Phil himself said he doesn't care to support other backends https://github.com/openai/triton/pull/1797#issuecomment-1730112311 https://github.com/openai/triton/pull/1797#issuecomment-1730...
- fpgamlirfanboy 3y agoIt's amazing how heavily provided hn is. I have a response here that's been deleted that is like 15 words, including a link to source that corroborates my claim but that response contains a transcribed emoji and so it's been deleted by dang or whomever. Lol super rich environment for discourse we've got going here.