7 ms·
That's an interesting point that I should perhaps have covered in the original article. The real code that this is based on is in fact hidden behind a GenServe
by clarkema 3y ago
That's an interesting point that I should perhaps have covered in the original article.
The real code that this is based on is in fact hidden behind a GenServer for this exact reason -- to maintain the expectations of other Elixir code that has to interact with it. The advantage of the escape hatch, as another commenter mentions, is allowing efficient sparse mutations of a large chunk of data, without having to pay a copy penalty every time. I definitely wouldn't recommend sharing the db handle widely.
- lvass 3y agoHave you measured performance? If mutating from Elixir like this can bring serious benefits, maybe there's a place for mutable versions of libraries like Explorer and Nx.
- clarkema 3y agoExplorer does actually use Rust (and polars) for a lot of its work -- its one on the libraries I looked at while figuring out my memory management issues.
- lvass 3y agoBut would it benefit from mutating the value of one reference? At the moment it does not do that, right?
- clarkema 3y agoNo, it doesn't -- looking at the website that's an explicit trade-off of pure performance vs 'Elixir-ish-ness'. It would certainly break a lot of expectations to have data mutating like that without it being hidden away somewhere, so I can understand why they went that way. In my case the data I'm dealing with is more of a store than a single data item, so I'm leaning on the example of things like ETS. Also it's within a single application rather than being a large generally-available library, so the trade-offs are different. It would be interesting to know if they did tests though.
- lawik 3y agoProbably not ideal since Polars builds on Apache Arrow and that tends to want to treat the structure as immutable if I recall correctly.
- rkangel 3y agoDid you consider a port (written in Rust) instead of a NIF? When you're presenting a GenServer like message passing interface a port is a natural fit, with none of the risks related to linking a NIF into the VM itself. (admittedly those risks are much lower with Rust than C)
- clarkema 3y agoIn our case one of these NIF stores is created per user for a specific task; ironically, with the amount of polish that Rustler puts around NIFs I suspect it would have been more work and more risk to go down the port route and manage everything manually.