9 ms·
Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
- dlphn___xyz 7y agowhats the benefit of rust over julia or C for computation?
- jdc 7y agofrom TFA: >> We chose Rust because: >> It has a very minimal runtime (essentially just bounds checks on arrays) and is easy to embed into other languages such as Java and Python >> It contains functional programming paradigms such as pattern matching that make writing code such as pattern matching compiler optimizations easier >> It has a great community and high quality packages (called “crates” in Rust) that made developing our system easier.
- vmchale 7y agoDid they have to write the runtime in the same language? They could've used an ML with GC and it would've been better (for a compiler). It doesn't really have any functional programming paradigms. Pattern matching is present in imperative languages like past versions of ATS
- fluffything 7y ago> Did they have to write the runtime in the same language? No, they originally wrote the runtime in C++, but ended up re-implementing it in Rust because the C++ runtime had too many bugs. > They could've used an ML with GC and it would've been better (for a compiler). They originally wrote the compiler in Scala with the JVM GC, and they said it was much slower and much harder to embed. > It doesn't really have any functional programming paradigms. Pattern matching is present in imperative languages like past versions of ATS When choosing a language for such a project, there are many engineering trade-offs that must be evaluated. The Weld project has dozens of developers that need onboarding, documentation, examples, tooling, etc. Rust has a lower barrier of entry than ATS. One of the main things Weld does is interfacing with LLVM: this is one of the main things the Rust compiler does and Rust has great libraries for this. Another thing Weld does is interfacing with many dynamic languages (Python, R, etc.). Rust does not just have "C FFI". It also has _a lot_ of great tooling for automatically generating and validating all the boilerplate automatically. Finally, performance and code size of Weld is one of the main advantages over the alternatives. Rust generates reasonable code with LLVM, ATS has its own code machine code generator, which while reasonable, isn't as good. Finally, it is hard to find people who enjoy writing ATS code. They exist, but are not many. Even when you do find them, they often don't like collaborating with people (I only know one ATS user, vmchale on github, and they don't really like interacting with others). OTOH it is trivial to find lots of people that enjoy writing Rust with others. It doesn't matter if this is due to technical reasons, marketing, or hype, but it's a fact that you have to consider if you want a project to grow fast.
- TurboHaskal 7y agoDid you realise you are responding to the same Github user you're mentioning?
- rch 7y agoI like the memory safety, concurrency, and zero-based indices.
- giancarlostoro 7y agoRust performance is pretty much in terms of C/C++ performance but promises stability in regards to memory management due to the borrow checker. Rust is very impressive on its own. If you havent taken the time to research Rust because "its yet another language" you really ought to honestly.
- agumonkey 7y agoin this particular case it's to be a drop in replacement for the python datascientist, the rest is irrelevant I believe
- asjw 7y agoNone On the other side you'll gain all the instability that a young language suffer from, like in 6 months it could be impossible to compile this code and you'll need to refactor it It is also heavily used only at Mozilla , which is not a strong guarantee of longevity
- onei 7y agoThe Rust developers are working hard to ensure that no breaking changes ever appear in the language. As such old code would always be compilable, even if there's better ways to do it with modern features. Rust learns the lessons that older languages taught us. Why would you think they'd fall into such an obvious pitfall given how long they took to get to v1?
- jcranmer 7y agoRust guarantees stability on any feature in the stable release of the compiler. And anything in the standard library will remain in there in perpetuity, even if deprecated.
- shpongled 7y agoNot to sound like a member of the Rust evangelism strike force, but after using Rust for a couple years, I don't have any desire to go back to C - sum types alone are worth the switch to me, not to mention iterators, concurrency story, etc.
- shaklee3 7y agoC++ has sum types as of several years ago with std::variant.
- codr7 7y agoThe way Python has macros, sure :) You would have to keep everything in variants, or wrap/unwrap manually all over the place to get similar functionality. And C has tagged unions.
- shaklee3 7y agoCan you elaborate more? What do other languages have over std::variant/visit?
- atroche 7y agoPattern matching syntax (eg in Rust) makes a big difference.
- fluffything 7y agoThat would be like explaining C++ Concepts to an assembly programmer from the 60s that had never used a "function" as a way of abstracting code. If you really want to know, spend one afternoon learning any programming language with built in support for that (Rust, Ocaml, Haskell, ...). ADTs is one of the first things one learns. In Rust, the features you'd need to learn are enums, patterns, and pattern matching. But be warned that using C++ variant and std::visit will feel like you are being forced to only write C instead of C++ for the rest of your life, knowing that life could be much better. Once you learn this, there is no way to un-learn it.
- noobermin 7y agohttps://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html Not quite Julia, but for C/Fortran particularly for computational uses, if your issue is multiple array scans, it might be time to look at rolling it yourself and then calling your algo from python.
- mkl 7y agoThe computation here is actually not done in Rust. The Rust code is performing stages of compilation of the original source code, into an intermediate representation that LLVM finishes compiling. The fully compiled code is what does the computation.
- vmchale 7y agoThis is different from Julia. I don't really know why you'd write a project of this sort in C.
- the_duke 7y agoSee also this interesting talk on Weld at RustConf 2019: https://www.youtube.com/watch?v=AZsgdCEQjFo&t=1430s https://www.youtube.com/watch?v=AZsgdCEQjFo&t=1430s
- westurner 7y agoThere's also RustPython, a Rust implementation of CPython 3.5+: https://news.ycombinator.com/item?id=20686580 https://news.ycombinator.com/item?id=20686580 > https://github.com/RustPython/RustPython https://github.com/RustPython/RustPython
- tomrod 7y agoIs this basically what Cython and PyPy are trying to do, but with Rust?
- sp332 7y agoPyPy is a JIT compiler. RustPython is an interpreter.
- yorwba 7y agoAnd Cython is an AOT compiler for a superset of Python. RustPython seems to be modestly aiming for a reimplementation of CPython.
- laughinghan 7y agoMaybe "dialect" would be more accurate than "superset"? I don't think Cython is technically a superset of Python, since I think runtime metaprogramming features like __dict__ and monkey-patching are significantly altered or restricted?
- loeg 7y agoYeah, I was going to make a similar comment. It's a dialect of CPython, and certainly there are extensions required to make it usable. But I'm not sure it is a strict superset of the full Python language.
- infinite8s 7y agoCython is a reification of the interpretation of a python program. Ie it converts the python code into the equivalent CPython API calls (which are all in C) thereby allowing the developer to interperse real C code. Anything you could do in python you could technically do in Cython, although it would be much more verbose.
- stereosteve 7y agoThis project has similar goals to the MLIR project: https://github.com/tensorflow/mlir https://github.com/tensorflow/mlir https://www.youtube.com/watch?v=qzljG6DKgic https://www.youtube.com/watch?v=qzljG6DKgic Exciting times for the future of parallel computing!
- xiphias2 7y agoI saw a performance comparision with XLA, and it's interesting that Weld is faster, because XLA is supposed to optimize the code using the known tensor sizes during compile time. Weld and XLA seem to have similar optimization steps though.
- sppalkia 7y agoXLA and Weld do have similar optimizations -- at their core, one of the main things they do is removing inefficiencies like unnecessary scans over data, common subexpressions, etc. across many operators. The speedup in the benchmark you're referring to actually involved some NumPy code too for pre-processing, and the reason Weld outperformed XLA is because Weld could perform those kinds of optimizations across TensorFlow operators and NumPy functions (whereas XLA only optimizes the TensorFlow part of the application). I also want to mention that this benchmark is from a while back (around 2017 I believe), so its possible improvements in both XLA and Weld will make the numbers look different today :)
- davmre 7y agoFor what it's worth, Jax (github.com/google/jax) now lets you use XLA to compile Numpy code. It'd be cool to see how that would stack up in a modern comparison.
- mlthoughts2018 7y agoVery bizarre there is no discussion of numba here, which has been around and used widely for many years, achieves faster speedups than this, and also emits an LLVM IR that is likely a much better starting point for developing a “universal” scientific computing IR than doing yet another thing that further complicates it with fairly needless involvement of Rust. https://numba.pydata.org/ https://numba.pydata.org/
- unbalancedparen 7y agoHi, I am the interviewer. I think I saw numba once but forgot about it. I will check it and probably ask to interview them too. We are preparing interviews about RAPIDS and other similar projects too.
- FreakLegion 7y agoNumba is the option used in Lectures in Quantitative Economics with Python, posted and highly upvoted here yesterday: https://news.ycombinator.com/item?id=21022620 https://news.ycombinator.com/item?id=21022620.
- blts 7y agoNumba is amazing. +1 for numba
- objektif 7y agoBut does it really speed up numerical libraries like numpy and pandas? I thought it only works on pure python code.
- sppalkia 7y agoI'm one of the developers of Weld -- Numba is indeed very cool and is a great way to compile numerical Python code. Weld performs some additional optimizations specific to data science that Numba doesn't really target right now (e.g., fusing parallel loops across independently written functions, parallelizing hash table operations, etc.). We're also working on adding the ability to call Python functions from within Weld, which will allow a data science program expressed in Weld to call out to other optimized functions (e.g., ones compiled by Numba). We additionally have a system called split annotations under development that can schedule chains of such optimized functions in a more efficient way without an IR, by keeping datasets processed by successive function calls in the CPU caches (check it out here: https://github.com/weld-project/split-annotations https://github.com/weld-project/split-annotations). Overall, we think that the accelerating the kinds of data science apps Weld and Numba target will not only involve tricks such as compilation that make user-defined code faster, but also systems that can just schedule and call code that people have already hand-optimized in a more efficient and transparent way (e.g., by pipelining data).
- axegon_ 7y agoI have said multiple times that Rust has an incredible potential in the data analysis world. And Weld is a great example.
- rolltiide 7y agoIf only you could get paid for porting open source libraries, maintaining them and trying to get a community to use it Have fun entertaining your Patreon
- axegon_ 7y agoTrue that, hence the reason why I do it so little. In fact I personally do it out of pure boredom.
- the_duke 7y agoWeld is a compiler/JIT/runtime though, something Rust is very well suited for, and which is very different code from data analysis/ML. I think Julia is a more interesting language for this space, with the built in matrix support, easier prototyping, a REPL, etc...
- sppalkia 7y agoRust is great, but this is an important comment! We used it to implement Weld's compiler and runtime, but we don't expect data scientists who use languages such as Python, Julia, or R to switch over to it; the idea is that these data scientists continue using APIs in these languages, and under the hood, Weld will perform optimizations and compilation for decreasing execution time (and these "under the hood" components are the ones that we wrote in Rust).
- fluffything 7y agoWould Weld be able to do a better job if these scientist were using a Rust library instead ? A lot of people would like to use Rust for data-analysis / machine learning, but there are not really any good batteries-included frameworks for getting started with this.
- spenrose 7y ago"the first implementation was in Scala, which was chosen because of its algebraic data types and powerful pattern matching. This made writing the optimizer, which is the core part of the compiler, very easy. Our original optimizer was based on the design of Catalyst, which is Spark SQL’s extensible optimizer. We moved away from Scala because it was too difficult to embed a JVM-based language into other runtimes and languages." There is an important "contemporary history of computing" article to write about the evolution of the Spark project from "let's build a distributed filesystem for MapReduce in Java because we read those early Google papers" to "SQL is the right model for working with data so DataFrames" to "meet data scientists where they are: Python (and R)" to "make machine learning easy" and now to "LLVM, but for crunching big numeric arrays".
- deleted 7y ago[deleted]
- dpflan 7y agoVery interesting. Do you have any references to share relevant to the article you suggest to be written?
- vmchale 7y agoNot that you should replace Rust with Haskell, but Haskell would've been a better choice than Scala. It has its own runtime, but it's not difficult to call Haskell code from C or ATS or whatever.
- tmostak 7y agoAlso worth checking out OmniSci (formerly MapD), which features an LLVM query compiler to gain large speedups executing SQL on both CPU and GPU: https://github.com/omnisci/omniscidb https://github.com/omnisci/omniscidb . And here's a link to a blog post giving a high level overview of the advantages of JIT compilation of queries over an interpreter: https://devblogs.nvidia.com/mapd-massive-throughput-database-queries-llvm-gpus/ https://devblogs.nvidia.com/mapd-massive-throughput-database... .
- nautilus12 7y agoThat tweetmap is impressive
- roadbeats 7y agoInteresting. It looks like Rust and Swift will be competitors in this field.
- adrien-treuille 7y agoThis post combines pretty much every technology I'm obsessed with right now: Python, Rust, Pandas, Numpy, and LLVM. Yess!!!
- whoevercares 7y agoJust a word of caution, always obsess with product and customer needs first :) In ML/data science tech first normally won’t end up well
- cortesoft 7y agoUnless they are just talking about personal projects?
- mkl 7y agoPeople aren't all or always working on products or serving customers.
- d33 7y agoIt's important to enjoy your work, which is - among other causes - about having right tools. Also, some of us actually get to have some influence over what language we write our projects in.
- ekianjo 7y agoI think you misunderstand the parent. The obsession to always focus on tools is I think, what they described. In the end of the day what matters is what you produce, not what tools you used. Nobody cares about what you used, apart from engineers.
- naniwaduni 7y agoYou care.
- toss1 7y ago
- riboflavin 7y agoSounds a lot like Gandiva (part of Apache Arrow) as well. https://www.dremio.com/announcing-gandiva-initiative-for-apache-arrow/ https://www.dremio.com/announcing-gandiva-initiative-for-apa.... Cool!
- RocketSyntax 7y agoyou had me at keras
- syrusakbary 7y agoThis is awesome. The quality of work behind it it's incredible. I think there might be something interesting for this strategy also in the WebAssembly space :)
- Myrmornis 7y ago> The motivation behind Weld is to provide bare-metal performance for applications that rely on existing high-level APIs such as NumPy and Pandas. With regard to Pandas this makes me pause slightly, since, while pandas contains lots of high quality and high performance implementations, the API of pandas in some places doesn’t feel well-designed (the most obvious example is indexing of data frames via square brackets and the various properties like iloc).
- janered 7y agoInteresting that initial implementation was in Scala but then they switched to Rust because of minimal runtime, language embeddability, functional paradigms, community and high quality packages. The hype bandwagon is so real in here. So basically one could say the same for several other well established languages, e.g. Haskell. Also what saddens me is that everyone forgets about D which has the same benefits and a syntax that does not make scratch your eyes out, especially when it comes to FP. Also D has not actually "skipped the leg day" ;)
- ris 7y agoSo... this requires cooperation from the underlying libraries (numpy, pandas...) - what is the likelihood of said libraries adopting this upstream vs Weld having to maintain their own shadow implementations for the foreseeable future? Numpy et al of course already have N python acceleration frameworks hammering at their doorsteps to integrate more closely...
- sgillen 7y agoHow much cooperation is needed though? It seems to me that all that numpy pandas etc. need to do is maintain a stable API, which they already do AFAIK.
- objektif 7y agoCan anyone pls tell me if there are any other tools out there to increase performance of pandas?
- alcidesfonseca 7y agoModin is an alternative pandas implementation for distributed processing using Ray or Dask: https://github.com/modin-project/modin https://github.com/modin-project/modin
- xtat 7y agoThis would have made so much of my work so much faster