6 ms·
It is possible to start external processes from BEAM and interact with them. I've blogged a bit about it at http://theerlangelist.com/article/outside_elixir htt
by sasa555 10y ago
It is possible to start external processes from BEAM and interact with them. I've blogged a bit about it at http://theerlangelist.com/article/outside_elixir http://theerlangelist.com/article/outside_elixir
You can also write NIFs (native implemented functions) which run in BEAM process (see http://andrealeopardi.com/posts/using-c-from-elixir-with-nifs/ http://andrealeopardi.com/posts/using-c-from-elixir-with-nif...). The latter option should be the last resort though, because it can violate safety guarantees of BEAM, in particular fault-tolerance and fair scheduling.
So using BEAM facing language as a "controller plane" while resorting to other languages in special cases is definitely a viable option.
- sashaafm 10y agoLove your blog and book Sasa. Could elaborate on the fair scheduling disruption by NIFs? Don't recall ever reading about that
- sasa555 10y agoThanks, nice to hear that! Basically a NIF blocks the scheduler, so if you run a tight loop for a long time, there will be no preemption. Therefore, invoking foo(), where a foo is a NIF which runs for say 10 seconds, means a single process will get 10 seconds of uninterrupted scheduler time, which is way more than other processes not calling that NIF. There are ways of addressing that (called dirty schedulers), but the thing is that you need to be aware of the issue in the first place. If due to some bug a NIF implementation ends up in an infinite loop, then the scheduler will be blocked forever, and the only way to fix it is to restart the whole system. That is btw. a property of all cooperative schedulers, so it can happen in Go as well. In contrast, if you're not using NIFs, I can't think of any Erlang/Elixir program that will block the scheduler forever, and assuming I'm right, that problem is completely off the table.
- kyrra 10y agoAs linked elsewhere here, tight loops that never preempt are being fixed in Go 1.8/1.9[0]. Looks like a flag may been added to Go 1.8 called "GOEXPERIMENT=preemptibleloops" that adds a preemptible point at the end of a loop. It's behind a flag for performance/testing reasons, but they are working on it. [0] https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958
- pmarreck 10y agoWon't pre-emptible loops lead to more irreproducible race conditions as a negative consequence, unless the preemption is done deterministically?
- Callmenorm 10y agoI spent 30 minutes looking at NIF, but I was scared away. My understanding is that if the NIF crashes then BEAM crashes. Which leads me to think that if you need NIF then you need safety guarantees on the Native side that C can't provide.
- sasa555 10y agoPrecisely, which is why I always advise to consider ports first :-) However, in some situations the overhead of communicating with a port might be too large, so then you have two options: 1. Move more code to another language which you run as a port. 2. Use a NIF It's hard to generalize, but I'd likely consider option 1 first. If you go for a NIF, you can try to keep its code as simple as possible which should reduce the chances of crashing. You can also consider extracting out the minimum BEAM part which uses the NIF into a separate BEAM node which runs on the same machine. That will reduce the failure surface if the NIF crashes. I've also seen people implementing NIFs in Rust for better safety, so that's another option to consider. So there are a lot of options, but as I said, NIF would usually be my last choice precisely for the reason you mention :-)
- lenkite 10y agoApparently people are working on this using Rust for writing NIFs https://github.com/hansihe/rustler https://github.com/hansihe/rustler
- derefr 10y ago