5 ms·

Side question as I am following zig only losely and do Go Programming: Zig has the feature that you can drop in your allocator from the caller. Now with 0.16 y

by weitzj 3mo ago

Side question as I am following zig only losely and do Go Programming: Zig has the feature that you can drop in your allocator from the caller. Now with 0.16 you also "bring your own IO" implementation with you. And for my understanding this looks like the pattern Go uses with its Context package, where you pass in transitive data, cancellation signals and timers to for example stop an SQL query in server B, since a user canceled a web request in their browser before hitting server A, whilst all elements delegate the Context. Then yesterday was total unrelated article about NUMA architecture, and I remember somewhere that the creator of Erlang mentioned (Joe Armstrong) that you cannot get around physics and it takes time to call a function between servers, therefore so not try to hide the latency between the calls. And now to my question: Would that in any way make sense for zig to go even more in this direction, where you pass in your allocator, IO, but now with something similar to Google Go context, but have it even more fine grained? So that your functions could actually in their interface expose somehow the time in between CPU cores (NUMA) up to the request cancellation as Google go context is used for timeout signals, cancellation signals. Also probably making time an external dependency as well. So in essence , every function would be treated as a remote procedure calls, whereas remote would mean: "other cpu", other server/service

dnautics 3mo ago
not really sure what you're asking. but time is now part of the io interface, so you can pretty easily write your own io implementation that just stubs out to the stdlib's default io content and has custom time functions if you want to try something funky. This is pretty close to what you are asking for (and in terms of functionality, equivalent). I say go for it and see what happens, report back!
- weitzj 3mo ago
  Thank you for your response. So time as you mentioned can be made explicit. I am probably looking for something like the request context with timers and cancellation signals propagating through the call chain like in gRPC but more fine granular. So you could implement arbitrary response times of functions. So you could say like: This whole request must finish in 5 seconds, otherwise abort. And these 5 seconds can be made very low so that you reach latency of NUMA vs Single CPU. I don't know how you call this, maybe "latency budget"
- jsrcout 3mo ago
  > time is now part of the io interface [...] Nice, didn't know that! Sounds like this could be extremely useful for some of the simulators we work on (though we mostly work in C/C++).
noelwelsh 3mo ago
What Zig is doing is called "capability passing". I don't know if the Zig team is aware of this field of work, or have independently arrived upon it, but that's what is achieved by passing IO, memory allocators, and other stuff around. The core idea is that you create a "capability" for any action you want to track, such as using IO, allocating memory, or in your example making cross core or cross server calls. Now to perform these actions code has to have access to the capability. It's a very simple, but powerful, model. The Effekt language formalizes this, and adds safety properties: https://effekt-lang.org/ https://effekt-lang.org/ Scala 3 also has this. The papers are pretty readable, if you ignore the middle bits which go into the formal models.
- epolanski 3mo ago
  That looks like a convoluted way to describe dependency injection to me, what am I missing.
  - noelwelsh 3mo ago
    You're right, they are related. One difference is simply lineage. Capabilities come out of the erights / security world (e.g. [0]), while dependency injection comes from the XP / agile / Martin Fowler world. More interesting is that to do capabilities correctly you need some type system extensions, namely capture checking. Essentially, this means you delimit where a program can hold a reference to a value derived from a capability. So if you have a function that allocates memory, you can say "there are no references to any allocated memory outside this function call" and hence no use-after-free bugs. It gives a form of resource management that is simpler than Rust's lifetimes. See [1] and [2]. (Technically it's a modal type system versus Rust's substructural type system.) To my mind it's an obvious thing for Zig to add. Shameless plug. If you're interested in more on this, for a programmer's rather than academic perspective, this is going into my book [3]. I'm writing the chapter of capability passing right now. Back to writing! [0]: https://blog.acolyer.org/2016/02/16/capability-myths-demolished/ https://blog.acolyer.org/2016/02/16/capability-myths-demolis... [1]: https://effekt-lang.org/tour/captures https://effekt-lang.org/tour/captures [2]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html https://docs.scala-lang.org/scala3/reference/experimental/cc... [3]: https://functionalprogrammingstrategies.com/ https://functionalprogrammingstrategies.com/

by weitzj 3mo ago

dnautics 3mo ago

not really sure what you're asking. but time is now part of the io interface, so you can pretty easily write your own io implementation that just stubs out to the stdlib's default io content and has custom time functions if you want to try something funky. This is pretty close to what you are asking for (and in terms of functionality, equivalent). I say go for it and see what happens, report back!

weitzj 3mo ago
Thank you for your response. So time as you mentioned can be made explicit. I am probably looking for something like the request context with timers and cancellation signals propagating through the call chain like in gRPC but more fine granular. So you could implement arbitrary response times of functions. So you could say like: This whole request must finish in 5 seconds, otherwise abort. And these 5 seconds can be made very low so that you reach latency of NUMA vs Single CPU. I don't know how you call this, maybe "latency budget"
jsrcout 3mo ago
> time is now part of the io interface [...] Nice, didn't know that! Sounds like this could be extremely useful for some of the simulators we work on (though we mostly work in C/C++).

noelwelsh 3mo ago

What Zig is doing is called "capability passing". I don't know if the Zig team is aware of this field of work, or have independently arrived upon it, but that's what is achieved by passing IO, memory allocators, and other stuff around. The core idea is that you create a "capability" for any action you want to track, such as using IO, allocating memory, or in your example making cross core or cross server calls. Now to perform these actions code has to have access to the capability. It's a very simple, but powerful, model. The Effekt language formalizes this, and adds safety properties: https://effekt-lang.org/ https://effekt-lang.org/ Scala 3 also has this. The papers are pretty readable, if you ignore the middle bits which go into the formal models.

epolanski 3mo ago
That looks like a convoluted way to describe dependency injection to me, what am I missing.
- noelwelsh 3mo ago
  You're right, they are related. One difference is simply lineage. Capabilities come out of the erights / security world (e.g. [0]), while dependency injection comes from the XP / agile / Martin Fowler world. More interesting is that to do capabilities correctly you need some type system extensions, namely capture checking. Essentially, this means you delimit where a program can hold a reference to a value derived from a capability. So if you have a function that allocates memory, you can say "there are no references to any allocated memory outside this function call" and hence no use-after-free bugs. It gives a form of resource management that is simpler than Rust's lifetimes. See [1] and [2]. (Technically it's a modal type system versus Rust's substructural type system.) To my mind it's an obvious thing for Zig to add. Shameless plug. If you're interested in more on this, for a programmer's rather than academic perspective, this is going into my book [3]. I'm writing the chapter of capability passing right now. Back to writing! [0]: https://blog.acolyer.org/2016/02/16/capability-myths-demolished/ https://blog.acolyer.org/2016/02/16/capability-myths-demolis... [1]: https://effekt-lang.org/tour/captures https://effekt-lang.org/tour/captures [2]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html https://docs.scala-lang.org/scala3/reference/experimental/cc... [3]: https://functionalprogrammingstrategies.com/ https://functionalprogrammingstrategies.com/