8 ms·

Erlang absolutely has closures, you are mistaken. What you are referring to are "function captures", which bind a function reference as a value, and there is no

Erlang absolutely has closures, you are mistaken. What you are referring to are "function captures", which bind a function reference as a value, and there is no environment to close over with those. However, you can define closures which as you'd expect, can close over bindings in the environment in which the closure is defined. The interaction between hot reloads and function captures in general is a bit subtle, particularly when it comes to how a function is captured. A fully qualified function capture is reloaded normally, but a capture using just a local name refers to the version of the module at the time it was captured, but is force upgraded after two consecutive hot upgrades, as only two versions of a module are allowed to exist at the same time. For this reason, you have to be careful about how you capture functions, depending on the semantics you want.

toast0 2y ago
> but is force upgraded after two consecutive hot upgrades, as only two versions of a module are allowed to exist at the same time. Force upgraded is maybe misleading. When a module is loaded for the 3rd time, any processes that still have the first version in their stack are killed. That may result in a supervisor restarting them with new code, if they're supervised.
- bitwalker 2y ago
  Ah right, good point - I was trying to remember the exact behavior, but couldn't recall if an error is raised (and when), or if the underlying module is just replaced and "jesus take the wheel" after that.
Muromec 2y ago
What does is it look like? I was talking about this thing: Val = 1, SumFun = fun(X) -> X + Val end, SumFun(2). It looks like you define arity 1 function that captures Val, while in fact you define arity 2 function and bind 1 as a first argument. Since you can't redefine Val anyway, it's as good as a closure, but technically it doesn't capture the environment. Maybe I'm mistaken and there is another way to express it?
- bitwalker 2y ago
  The example you've given here does not work the way you think it does. I would agree however that the mechanics of closure environments is simpler in Erlang due to the fact that values are immutable, as opposed to closures in other languages where mutability must be accounted for. I would also note that, for the example you've given, the compiler _could_ constant-fold the whole thing away, but for the sake of argument, let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound. The closure will be constructed at the point it is captured, using the `make_fun` BIF, with a given number of free var slots (in this case, 1 for the capture of `Val`). `Val` is written to the slot in the closure environment at this time as well. See the implementation of the BIF [here](https://github.com/erlang/otp/blob/6cefa05a2a977864150908feb8c3f2ffe29e43ae/erts/emulator/beam/emu/instrs.tab#L543 https://github.com/erlang/otp/blob/6cefa05a2a977864150908feb...) if you are curious. At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars. In my own Erlang compiler, the closure environment was given via pointer, as the first argument to the function, and then instructions were emitted to load free variables relative to that pointer. I believe BEAM does the same thing, but it may differ in the specific details, but conceptually that is how it works. The compiler obviously must generate a new free function definition for closures with free variables (hence the name of the function you see in the interactive shell, or in debug output). The captured MFA of the closure is this generated function. The runtime distinguishes between the two types of closures (function captures vs actual closures) based on the metadata of the func value itself. Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard.
  - Muromec 2y ago
    > let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound. That was exactly the case I was talking about, because otherwise there is no need to even make arity 2 function. If the value is known at compile time, the constant is embedded into the body of inlined function. >At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars. To my understanding, no it doesn't, as the value is resolved when the function pointed is created, not when the underlying function executes, which the code you linked shows too. I know it uses the "env" as a structure field, but it's partial application, not the actual closure which has access to parent scope. Consider two counter examples in python: for x in range(1,10): ret.append(partial(lambda y: y*2, x)) # that's what erlang does for x in range(1,10): ret.append(partial(x, lambda y: y*2)) # that's an actual closure, as all lambdas will return 18 because x is captured from the parent context But then again, it doesn't matter since variables are assigned only once. >Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard. I was looking into it a week ago, and erlc does what I described when it can't figure out the constant at compile time. add: If we are at it, BEAM doesn't even know about variables, only values and registers anyway, so it has nothing to capture anyway.

toast0 2y ago

> but is force upgraded after two consecutive hot upgrades, as only two versions of a module are allowed to exist at the same time. Force upgraded is maybe misleading. When a module is loaded for the 3rd time, any processes that still have the first version in their stack are killed. That may result in a supervisor restarting them with new code, if they're supervised.

bitwalker 2y ago
Ah right, good point - I was trying to remember the exact behavior, but couldn't recall if an error is raised (and when), or if the underlying module is just replaced and "jesus take the wheel" after that.

Muromec 2y ago

What does is it look like? I was talking about this thing: Val = 1, SumFun = fun(X) -> X + Val end, SumFun(2). It looks like you define arity 1 function that captures Val, while in fact you define arity 2 function and bind 1 as a first argument. Since you can't redefine Val anyway, it's as good as a closure, but technically it doesn't capture the environment. Maybe I'm mistaken and there is another way to express it?

bitwalker 2y ago
The example you've given here does not work the way you think it does. I would agree however that the mechanics of closure environments is simpler in Erlang due to the fact that values are immutable, as opposed to closures in other languages where mutability must be accounted for. I would also note that, for the example you've given, the compiler _could_ constant-fold the whole thing away, but for the sake of argument, let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound. The closure will be constructed at the point it is captured, using the `make_fun` BIF, with a given number of free var slots (in this case, 1 for the capture of `Val`). `Val` is written to the slot in the closure environment at this time as well. See the implementation of the BIF [here](https://github.com/erlang/otp/blob/6cefa05a2a977864150908feb8c3f2ffe29e43ae/erts/emulator/beam/emu/instrs.tab#L543 https://github.com/erlang/otp/blob/6cefa05a2a977864150908feb...) if you are curious. At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars. In my own Erlang compiler, the closure environment was given via pointer, as the first argument to the function, and then instructions were emitted to load free variables relative to that pointer. I believe BEAM does the same thing, but it may differ in the specific details, but conceptually that is how it works. The compiler obviously must generate a new free function definition for closures with free variables (hence the name of the function you see in the interactive shell, or in debug output). The captured MFA of the closure is this generated function. The runtime distinguishes between the two types of closures (function captures vs actual closures) based on the metadata of the func value itself. Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard.
- Muromec 2y ago
  > let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound. That was exactly the case I was talking about, because otherwise there is no need to even make arity 2 function. If the value is known at compile time, the constant is embedded into the body of inlined function. >At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars. To my understanding, no it doesn't, as the value is resolved when the function pointed is created, not when the underlying function executes, which the code you linked shows too. I know it uses the "env" as a structure field, but it's partial application, not the actual closure which has access to parent scope. Consider two counter examples in python: for x in range(1,10): ret.append(partial(lambda y: y*2, x)) # that's what erlang does for x in range(1,10): ret.append(partial(x, lambda y: y*2)) # that's an actual closure, as all lambdas will return 18 because x is captured from the parent context But then again, it doesn't matter since variables are assigned only once. >Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard. I was looking into it a week ago, and erlc does what I described when it can't figure out the constant at compile time. add: If we are at it, BEAM doesn't even know about variables, only values and registers anyway, so it has nothing to capture anyway.