10 ms·

Alpaca – Functional programming inspired by ML for the Erlang VM

217 points by knv 9y ago

sctb 9y ago
Related discussion from last year: https://news.ycombinator.com/item?id=11992773 https://news.ycombinator.com/item?id=11992773.
platz 9y ago
I continue to be believe even as a static typing fan that static types are fundamentally incompatible with OTP and it's goals. Distributed systems just seem to too thorny for static types to subjugate/bend to their will. Sure, you can declare global invariants ahead of time that your cluster must uphold, but it's a bit less "distributed" in a real sense then
- brightball 9y ago
  I tend to agree with you. Message passing and static types don't mesh unless there is some type of contract between the sender and receiver. It would be a nightmare.
  - msangi 9y ago
    How does dynamic typing help if there is no contract between sender and receiver? Even in that case they must agree on the content of message. Even if you insist in keeping the message untyped, with a static type system one could always convert (and possibly reject) messages as soon as they are received into a more precise type. That would keep the code that the compiler can't verify to the edges of the system.
    - platz 9y ago
      Distributed systems are not like traditional programs, because there is not just one "edge of the system". Every node becomes an "edge" in it's own right, and doesn't necessarily have global coherence with the rest of the system.
      - louthy 9y ago
        True, but if the sender wants the receiver to do something of value then it will need to meet a contract that the receiver enforces. That doesn't require a central repository of contracts, one node can diverge, but you must understand that parts of your network of services will start to fail. From that point of view it starts to look very much like the linker phase of a compilation, and that the types need to match up to the data structures being instantiated. It's just this 'linking' phase is in the programmers heads, and not particularly useful. A distributed system that can validate itself is a much more valuable concept.
        platz 9y ago
        Absolutely, the rubber meets the road at some point, nodes must understand/assume "contracts" about the data they are working with. There already are static typed actor systems ( e.g. Orleans) which work well, but my point is that I believe OTP is more flexible for better or worse. Whether that flexibility is worth it to you for what you get is another matter. Also I'm not sure how to think about binary compatibility between upgrades in such a system
        louthy 9y ago
        > There already are static typed actor systems (e.g. Orleans) Yep, I develop one myself. And have gone to the extent of not allowing senders to even post a message if it's of the wrong type (processes in nodes publish the types they accept to a central store). I initially went along with the 'accept anything' approach (which Akka really majors on too), but found that for the large systems I was developing that it became a real headache to deal with. > but my point is that I believe OTP is more flexible for better or worse. Whether that flexibility is worth it to you for what you get is another matter. Yep, fair enough, if it works for you, who am I to complain? It's not worth it for me, because I feel quite strongly that the code I write should understand the types it's working with. It feels like this super-late binding can give false positives, appear to work, when in fact it's not. That scares the shit out of me when systems get large.
  - jordwest 9y ago
    I would say there should always be a contract between the sender and receiver, whether that's using static types or otherwise. Not having a contract is a nightmare. For example, say a satellite sends a number to the throttle control in feet/second, but the throttle control thinks its in m/s. To each of those systems, they're just passing a number and don't know any better.
    - brightball 9y ago
      Every JSON API call currently works without a contract. In theory it should have one, but in reality it doesn't unless the server (hopefully) validates. Either can change at any time without informing each other. WSDL based APIs on the other hand have clearly defined contracts at both ends but there's more overhead involved.
      - msangi 9y ago
        It's not an explicit contract, but there must be an implicit one for stuff not to break
- sgrove 9y ago
  Would you mind elaborating, or sharing some papers on the subject? I'm particularly interested in a dialect of ReasonML that would use the BuckleScript compiler + ConcurrentML but target the BEAM VM, and I'd love to know how bad of an idea it might be. Maybe because it lacks e.g. session types it's hopeless, but I'm not sure. So, would love to hear specifics!
  - platz 9y ago
    There are parts of OTP patterns that seem inherently dynamic. Message passing is only one aspect. There are also deployment/upgrade concerns with a running system. Actors can receive messages that change their behavior entirely ( http://erlang.org/doc/man/gen_server.html http://erlang.org/doc/man/gen_server.html ). Features like this are not there by accident. Actors can hot-upgrade code their dynamically while the process is running. For example, if an actor is hot-upgrading I'm not sure how it would work, if the types of the old state and the new state don't exactly match. Sure, you could write functions to do this, but you see the picture is much more complicated. I don't think I've presented the best arguments off the top of my head here here, but if you think more about the deployment/upgrade scenarios, along with partial updates along in certain nodes of the system, you can think about how complex it could get. Basically, never assume that you get to take the whole cluster down to do an upgrade. Comprehensive "red/black" deployment strategies used by other non-distributed languages are not really the OTP way of doing deployment/upgrades.
    - sitkack 9y ago
      I have version N of a struct and then having version N+1 of a struct in-flight at the same time is almost impossible with current statically typed languages. In a dynamically typed language, as long as the contents of version N+k struct are additive and don't change the semantics, old code can read new data. What needs to happen is both, immutable code, and versioned structs with pure functions that can upgrade and possibly downgrade structs as needed. The larger the distributed system, the versions of a struct (message) will be in-flight at a time. Services need to contain no state, so that they can be micro-rebooted and brought up with the new version. Joe Armstrong had a comment on globally accessible but immutable code, which I think would go a long way towards the ability to statically type the inputs to a function in a distributed system. Interposition and routing would be the only way to upgrade or deprecate old code paths.
- jrobn 9y ago
  I also agree with this premise. I favor the gradual typing philosophy more and more. For me at least, productivity wise, being able to write something, play around with it, make changes, etc without worrying to much about satisfying type requirements is great. When the idea and and implementation feels solid go back and gradually add in type requirements. I would love to see Erlang get a LLVM based JIT compiler backend. I think this http://llvm.org/devmtg/2014-04/PDFs/Talks/drejhammar.pdf http://llvm.org/devmtg/2014-04/PDFs/Talks/drejhammar.pdf is the latest work done in that area.
  - di4na 9y ago
    There was a talk at the eef17 this week from the OTP team on it. It is on erlang solutions YouTube channel. Sadly on phone so can hardly link it
  - di4na 9y ago
    Found it ! https://youtu.be/PtgD5WRzcy4 https://youtu.be/PtgD5WRzcy4
- jerf 9y ago
  You have to model sending a message across the cluster as marshaling into a binary form and unmarshaling it again. I don't mean that you "should" model it that way... you have to model it that way, because that's what is happening. Therefore, when receiving a message, you really only ever get a Maybe Message or Either Message Error or whatever you want to model it as. The act of marshaling the message back into the local representation is also when you check it for whether it conforms to the type restrictions you think it should have. Because you must already model this as a process that can fail, I don't think it does break the static typing model at all. In fact I routinely "statically type" messages coming from things that were actually emitted by dynamic languages! What gets tricky is if you try to model this as a process that can't fail. But the problem there isn't static typing, it's a specific instance of the general principle that you can not build robust systems based on the principle that networks can't fail. I also think this is an instance of the general misunderstanding about static types, which I understand deeply because I once held it, that static types somehow prevent errors. They don't. What they do is provide a gateway that says "in order to get into this type, you must meet these criteria, and the compiler is going to statically check that you've verified these criteria". A static typing system doesn't force things through that gateway, it forces you to check whether things fit through that gateway, and do something with the things that don't. Then, it also allows you to strictly declare that everything that uses that type is statically checked to be "behind" that gateway, so there are no other ways around it to get in, thus creating a space in which you can count on the fact that the values have been checked for certain properties and you can now write code that counts on those without constantly checking them. A statically typed system faced with the task of, say, parsing a number out of a string, does not prevent a user from sending me a string of "xyz"; it just prevents me from just sending it through the system as-is.
  - platz 9y ago
    > everything that uses that type is statically checked to be "behind" that gateway In a distributed system, the largest the "gateway" can reliable be is a single node, because you don't get guarantees about the code that other nodes in the system are running. Even the single node case poses difficulties, because I believe in OTP the upgrade path means you have to transfer state during upgrades. What if the types of the state during the upgrade don't exactly match? Can multiple types of a thing exist simultaneously? How is these types versioned? etc... it gets complicated. > Therefore, when receiving a message, you really only ever get a Maybe Message or Either Message Error or whatever you want to model it as. Sure, you can receive messages as "Object" and then cast/parse them inside the node. Does that mesh with the vision of what people have when they want to bring static typing to erlang? --- The hard part about thinking about OTP is not just the message passing, but also the myriad deployment & upgrade & versioning scenarios. I am a fan of static typing over dynamic typing in everything else , i.e. normal programs.. just not _OTP-style_ erlang for distributed systems. Even thinking about something like a gen_server (http://erlang.org/doc/man/gen_server.html http://erlang.org/doc/man/gen_server.html) makes my head hurt... though if someone can figure out a way to do it that's faithful, more power to them.
- dalailambda 9y ago
  While I agree that the OTP perhaps is not as easily statically typed, since it was built with Erlang in mind, I do think that static typing adds a layer of robustness to distributed systems, especially if you design it that way upfront. In my experience the problem comes when you try to apply static typing to a dynamic system.
  - leshow 9y ago
    > In my experience the problem comes when you try to apply static typing to a dynamic system. As in, because it compiles down to a dynamic system, it's no good? There are plenty of languages that give us strong static guarantees and compile down to dynamic or untyped languages. Look at Purescript, Elm, etc. They all do quite well compiling down to JS. Don't forget that assembly isn't strongly typed either, and most languages compile down to that. I don't see anything wrong with a static typed layer that compiles to dynamic code, the interface you're providing is still type safe.
    - dalailambda 9y ago
      In regards to Purescript/Typescript, they're both statically typed and that results in friction when trying to integrate with the existing JavaScript ecosystem/libraries. Erlang/OTP might be different, but there will probably be situations where the type system is either incompatible with a certain library, or the type system is made less strict (e.g. an any type).
      - leshow 9y ago
        That wasn't what I got out of the previous post, it seemed to be saying there was something inherently unsafe about compiling down to a dynamic language.
- nv-vn 9y ago
  I suggest you look into Session Typing, specifically Multiparty Session Types as these provide a refreshing approach to the problem of communicating threads. A lot of it is still kind of experimental but there's some good traction being made for sure and it's probably as expressive as you'd need to get to model 95% of the type information in an Erlang program. Type inference obviously isn't a choice yet, but I think a good language offering some of these features on the BEAM VM is all that is needed to make them hit the mainstream and actually get used for real software so that more work can go into the theory, etc. The problem is being solved on the bleeding edge of things, just not as fast as Erlang itself is progressing.
- runeks 9y ago
  > Distributed systems just seem to too thorny for static types to subjugate/bend to their will. The more I've learned to leverage types, the more I realize that it's my limited knowledge of type systems that prevents me from expressing something in it. Types do not bend to the will of programs; programs bend to the will of types (in statically typed languages). > Sure, you can declare global invariants ahead of time that your cluster must uphold, but it's a bit less "distributed" in a real sense then I don't understand. The components of distributed systems communicate via protocols. What prevents the implementation of these protocols from leveraging type safety, thus transforming a runtime error into a compile-time one? Static typing is about catching programmer mistakes, by communicating your intent to a compiler -- "I expect the type of this to be a Maybe Int, fail if that's not the case". There's no essential difference between a test informing you that a value-level property doesn't hold up at runtime, and a type error, informing you that a type-level property doesn't hold up at compile-time.
  - platz 9y ago
    > What prevents the implementation of these protocols from leveraging type safety Global invariants of a running distributed system are different than local invariants in a single program that you can stop, deploy re-compiled binaries to, and then start again. Now, you can use static types in actor systems, and they are some of these that exist. These typed actor systems don't do all the same things that erlang/OTP does (that may be ok - maybe you don't need them). If your use case fits into what the typed actor systems actor systems provide, by all means, one of those are probably a better fit for you.
- Tarean 9y ago
  I think it is necessary to go with an erlang like style if one wants to get a remotely acceptable cost model. I am not sure how much static typing actually hinders and how much that is a matter of tooling, though. Maybe static typing could do things like checking whether the new version will be compatible with other nodes before deploying?
salimmadjd 9y ago
Really hoping this project gets more traction. I'm learning Elm now and I'm really liking the syntax to the level that other languages feel rather cluttered to me now. The more I'm playing with types and learning to leverage them, the more I appreciate their power (yes, I'm late to the game) so making this statically typed is very interesting. However, there seem to be a saturation of new languages and not sure if there is enough eyeballs left for a new language that does not have a large corporate backing (FB, Google, Apple) or happens not to arrive on a perfect time with the right set of answers. Maybe BEAM, ML/Elm syntax and static typing is what everyone else is looking for. Edit: Video posted today of creator of Alpaca (Jeremy Pierre) giving a talk at Erlang Factory. It gives a nice overview of the state of the language - https://www.youtube.com/watch?v=cljFpz_cv2E https://www.youtube.com/watch?v=cljFpz_cv2E
- platz 9y ago
  > enough eyeballs left for a new language that does not have a large corporate backing It's a solid point if the goal is winner-take-all style competitive victory. But I'm not sure software should co-op SV-startup-business exponential growth-or-die mindset. What happened to hacker culture? Are open source developers corporatist now? /end-speculative-rant
  - bbcbasic 9y ago
    Open source is massively corporatist. Developers working for $0 to create tool chains that other developers spend their weekend learning so that the shareholders of their employers can increase their wealth.
  - alextheparrot 9y ago
    There does seem to be a significant push to contribute to open-source at many large companies. Having money and people contributing as part (or all) of their day job can be quite a boon for projects. I agree with the premise of your point, though.
  - salimmadjd 9y ago
    The point I was making is for something to get enough traction, so that it would get active contributors who help mature the language, tools, etc. I think there are only handful of people out there who can contribute in a meaningful way for a project like this. If they are consumed working on open source Swift or doing pull request on many things pushed by FB or Google or working contributing to existing projects like GHC, etc. Then the Alpaca project wont get the contributors it needs to show progress. If there is no progress, it falls into a vicious circle of no progress -> no traction -> no contributors -> no progress
sergiotapia 9y ago
>Apache License, Version 2.0 Pardon my ignorance, but why not make it MIT and completely avoid any licensing issues?
- geofft 9y ago
  The Apache license has an explicit patent grant (the MIT license says "permission to use", which isn't a copyright grant, so it probably has an implicit patent grant), and an explicit statement that patches intentionally submitted for merge are submitted back under the Apache license. The reason we have licenses at all instead of the Unlicense or similar is to make things unambiguous for courts and lawyers. Explicit is better than implicit. The length of the MIT license isn't actually a feature. (And the contribution section seems like it avoids licensing issues that MIT doesn't.)
  - dmm 9y ago
    License bikeshedding is fun, so please indulge me. People dislike apache because it's complicated and requires annoying notices on distribution of modified versions. Debian and fsf say it's free. OpenBSD believes the patent provisions are non-free and refuses to include apache licensed software. I think a project is better off having non-trivial contributers sign explicit license grants, even you admit that explicit is better than implicit.
    - liveoneggs 9y ago
      until they are taken to court the licenses are all just assumptions. Did OpenBSD have a lawyer review this decision and post the language somewhere?
    - geofft 9y ago
      I don't understand how the OpenBSD project defines "free", so I can't usefully comment on how they consider the Apache license "non-free". It sounds from https://softwareengineering.stackexchange.com/questions/263227/why-are-apache-2-0-works-excluded-from-openbsd https://softwareengineering.stackexchange.com/questions/2632... like they are reading the existence of an explicit patent clause in the Apache license (regardless of what that clause is!) as an "additional restriction". I want to know whether they believe the MIT license has a patent grant, and if not, what they think "Permission to use" means.
haspok 9y ago
I think this is great news for the Erlang VM: while you wouldn't want to use static typing for any program you write, there is a very specific use-case where you definitely want to do that: embedding business logic in your application. I've been there, done that: encoding business rules in Erlang is no fun, hard to test, and definitely hard to read and modify later. In this particular domain the constraint of types does not slow you down, in fact, it speeds up development. A large amount of unit tests can become unnecessary just because of the type checking. And the more expressive your type system, the fewer tests you need - and the code and the remaining tests can concentrate on validating business logic instead of validating programming language logic ("here is a map - do I have a value with key X in it?" - maybe a bad example because of pattern matching, but I hope you get the idea). You definitely have to be able to interface with OTP, but I don't see it as a huge problem - parts of your application could and should be written in Erlang, there is nothing wrong with that.
- im_down_w_otp 9y ago
  I'm not sure I'm following you regarding the difficulty of encoding business logic into your code in Erlang. Erlang's function-head matching system is extremely close to being a Prolog-style logic programming system when used a certain way. I've found it extremely easy to take what would normally be a big weird database of rules and values and instead precompile every possible route through the system into a bunch of generated function-head matched function calls + guard clauses. It makes assuring that given inputs will definitely produce correct outputs very easy, and makes processing the rules extremely fast.
Athas 9y ago
I am intrigued by this snippet from the README: type messages 'x = 'x | Fetch pid 'x This appears to define a sum type where one of the variants is left with an implicit constructor. How do you pattern match on that? How do you do type inference?
- j14159 9y ago
  That's a good question and the short answer is that we're deliberately breaking decidability to allow people to use types that are relatively in line with what we're used to in Erlang (I get into this a bit in the talk linked above). There's a reasonable argument to be made that we should knock this off of course but I'm generally biased in favour of making interoperation with the ecosystem simpler :) We should actually clean up that example you pointed out, matching on the `Fetch` constructor first. What we're really trying to support is unions like: type number = int | float There's a yet un-had argument about the utility of this as well of course and we may want to remove the ability to do this entirely. The way we type this is by actually using type-checking guard functions like `is_integer` to convey information to the typer at compile time.
pka 9y ago
There's also purerl [0], an Erlang backend for PureScript. Would be cool if the two projects could join forces. [0] - https://github.com/purerl/purescript https://github.com/purerl/purescript
jmcdiesel 9y ago
Can anyone explain why it seems like so many new languages are reinventing things that seemingly have little effect, but they seem to be changing them "just because" We have comments in code for decades now. // and /* */ are easily the bigest standard, with # coming in second. Why '--' in this language? Why "``" in another language i saw recently? I cant imagine this gives any real benifit to the coder or the compiler, and it seems to be more difficult because now the IDEs have to be configured for a(nother) new comment type, it has be become muscle memory again, its yet another "common ground" peice of code that requires context switching to change between languages... I get doing new things with the functional features of a language, im all for trying new things and seeing what works... just seems wierd to have so many ways to comment code... such an insignificant part, why change?
- SmellyGeekBoy 9y ago
  SQL uses '--' for comments.
- yawaramin 9y ago
  You may be pleased to hear that ReasonML chose /* */ for its OCaml syntax refresh just for familiarity.
- lepoetemaudit 9y ago
  It's -- in both Elm and Haskell for comments, so it's not new and part of the heritage of the language.
- SEMW 9y ago
  > why change? They didn't. The syntax is explicitly stated to be a mixture of OCaml and Elm. The -- comment syntax is from Elm. Elm in turn got it from Haskell.

217 points by knv 9y ago

sctb 9y ago

Related discussion from last year: https://news.ycombinator.com/item?id=11992773 https://news.ycombinator.com/item?id=11992773.

platz 9y ago

I continue to be believe even as a static typing fan that static types are fundamentally incompatible with OTP and it's goals. Distributed systems just seem to too thorny for static types to subjugate/bend to their will. Sure, you can declare global invariants ahead of time that your cluster must uphold, but it's a bit less "distributed" in a real sense then

brightball 9y ago
I tend to agree with you. Message passing and static types don't mesh unless there is some type of contract between the sender and receiver. It would be a nightmare.
- msangi 9y ago
  How does dynamic typing help if there is no contract between sender and receiver? Even in that case they must agree on the content of message. Even if you insist in keeping the message untyped, with a static type system one could always convert (and possibly reject) messages as soon as they are received into a more precise type. That would keep the code that the compiler can't verify to the edges of the system.
  - platz 9y ago
    Distributed systems are not like traditional programs, because there is not just one "edge of the system". Every node becomes an "edge" in it's own right, and doesn't necessarily have global coherence with the rest of the system.
    - louthy 9y ago
      True, but if the sender wants the receiver to do something of value then it will need to meet a contract that the receiver enforces. That doesn't require a central repository of contracts, one node can diverge, but you must understand that parts of your network of services will start to fail. From that point of view it starts to look very much like the linker phase of a compilation, and that the types need to match up to the data structures being instantiated. It's just this 'linking' phase is in the programmers heads, and not particularly useful. A distributed system that can validate itself is a much more valuable concept.
      - platz 9y ago
        Absolutely, the rubber meets the road at some point, nodes must understand/assume "contracts" about the data they are working with. There already are static typed actor systems ( e.g. Orleans) which work well, but my point is that I believe OTP is more flexible for better or worse. Whether that flexibility is worth it to you for what you get is another matter. Also I'm not sure how to think about binary compatibility between upgrades in such a system
        louthy 9y ago
        > There already are static typed actor systems (e.g. Orleans) Yep, I develop one myself. And have gone to the extent of not allowing senders to even post a message if it's of the wrong type (processes in nodes publish the types they accept to a central store). I initially went along with the 'accept anything' approach (which Akka really majors on too), but found that for the large systems I was developing that it became a real headache to deal with. > but my point is that I believe OTP is more flexible for better or worse. Whether that flexibility is worth it to you for what you get is another matter. Yep, fair enough, if it works for you, who am I to complain? It's not worth it for me, because I feel quite strongly that the code I write should understand the types it's working with. It feels like this super-late binding can give false positives, appear to work, when in fact it's not. That scares the shit out of me when systems get large.
- jordwest 9y ago
  I would say there should always be a contract between the sender and receiver, whether that's using static types or otherwise. Not having a contract is a nightmare. For example, say a satellite sends a number to the throttle control in feet/second, but the throttle control thinks its in m/s. To each of those systems, they're just passing a number and don't know any better.
  - brightball 9y ago
    Every JSON API call currently works without a contract. In theory it should have one, but in reality it doesn't unless the server (hopefully) validates. Either can change at any time without informing each other. WSDL based APIs on the other hand have clearly defined contracts at both ends but there's more overhead involved.
    - msangi 9y ago
      It's not an explicit contract, but there must be an implicit one for stuff not to break
sgrove 9y ago
Would you mind elaborating, or sharing some papers on the subject? I'm particularly interested in a dialect of ReasonML that would use the BuckleScript compiler + ConcurrentML but target the BEAM VM, and I'd love to know how bad of an idea it might be. Maybe because it lacks e.g. session types it's hopeless, but I'm not sure. So, would love to hear specifics!
- platz 9y ago
  There are parts of OTP patterns that seem inherently dynamic. Message passing is only one aspect. There are also deployment/upgrade concerns with a running system. Actors can receive messages that change their behavior entirely ( http://erlang.org/doc/man/gen_server.html http://erlang.org/doc/man/gen_server.html ). Features like this are not there by accident. Actors can hot-upgrade code their dynamically while the process is running. For example, if an actor is hot-upgrading I'm not sure how it would work, if the types of the old state and the new state don't exactly match. Sure, you could write functions to do this, but you see the picture is much more complicated. I don't think I've presented the best arguments off the top of my head here here, but if you think more about the deployment/upgrade scenarios, along with partial updates along in certain nodes of the system, you can think about how complex it could get. Basically, never assume that you get to take the whole cluster down to do an upgrade. Comprehensive "red/black" deployment strategies used by other non-distributed languages are not really the OTP way of doing deployment/upgrades.
  - sitkack 9y ago
    I have version N of a struct and then having version N+1 of a struct in-flight at the same time is almost impossible with current statically typed languages. In a dynamically typed language, as long as the contents of version N+k struct are additive and don't change the semantics, old code can read new data. What needs to happen is both, immutable code, and versioned structs with pure functions that can upgrade and possibly downgrade structs as needed. The larger the distributed system, the versions of a struct (message) will be in-flight at a time. Services need to contain no state, so that they can be micro-rebooted and brought up with the new version. Joe Armstrong had a comment on globally accessible but immutable code, which I think would go a long way towards the ability to statically type the inputs to a function in a distributed system. Interposition and routing would be the only way to upgrade or deprecate old code paths.
jrobn 9y ago
I also agree with this premise. I favor the gradual typing philosophy more and more. For me at least, productivity wise, being able to write something, play around with it, make changes, etc without worrying to much about satisfying type requirements is great. When the idea and and implementation feels solid go back and gradually add in type requirements. I would love to see Erlang get a LLVM based JIT compiler backend. I think this http://llvm.org/devmtg/2014-04/PDFs/Talks/drejhammar.pdf http://llvm.org/devmtg/2014-04/PDFs/Talks/drejhammar.pdf is the latest work done in that area.
- di4na 9y ago
  There was a talk at the eef17 this week from the OTP team on it. It is on erlang solutions YouTube channel. Sadly on phone so can hardly link it
- di4na 9y ago
  Found it ! https://youtu.be/PtgD5WRzcy4 https://youtu.be/PtgD5WRzcy4
jerf 9y ago
You have to model sending a message across the cluster as marshaling into a binary form and unmarshaling it again. I don't mean that you "should" model it that way... you have to model it that way, because that's what is happening. Therefore, when receiving a message, you really only ever get a Maybe Message or Either Message Error or whatever you want to model it as. The act of marshaling the message back into the local representation is also when you check it for whether it conforms to the type restrictions you think it should have. Because you must already model this as a process that can fail, I don't think it does break the static typing model at all. In fact I routinely "statically type" messages coming from things that were actually emitted by dynamic languages! What gets tricky is if you try to model this as a process that can't fail. But the problem there isn't static typing, it's a specific instance of the general principle that you can not build robust systems based on the principle that networks can't fail. I also think this is an instance of the general misunderstanding about static types, which I understand deeply because I once held it, that static types somehow prevent errors. They don't. What they do is provide a gateway that says "in order to get into this type, you must meet these criteria, and the compiler is going to statically check that you've verified these criteria". A static typing system doesn't force things through that gateway, it forces you to check whether things fit through that gateway, and do something with the things that don't. Then, it also allows you to strictly declare that everything that uses that type is statically checked to be "behind" that gateway, so there are no other ways around it to get in, thus creating a space in which you can count on the fact that the values have been checked for certain properties and you can now write code that counts on those without constantly checking them. A statically typed system faced with the task of, say, parsing a number out of a string, does not prevent a user from sending me a string of "xyz"; it just prevents me from just sending it through the system as-is.
- platz 9y ago
  > everything that uses that type is statically checked to be "behind" that gateway In a distributed system, the largest the "gateway" can reliable be is a single node, because you don't get guarantees about the code that other nodes in the system are running. Even the single node case poses difficulties, because I believe in OTP the upgrade path means you have to transfer state during upgrades. What if the types of the state during the upgrade don't exactly match? Can multiple types of a thing exist simultaneously? How is these types versioned? etc... it gets complicated. > Therefore, when receiving a message, you really only ever get a Maybe Message or Either Message Error or whatever you want to model it as. Sure, you can receive messages as "Object" and then cast/parse them inside the node. Does that mesh with the vision of what people have when they want to bring static typing to erlang? --- The hard part about thinking about OTP is not just the message passing, but also the myriad deployment & upgrade & versioning scenarios. I am a fan of static typing over dynamic typing in everything else , i.e. normal programs.. just not _OTP-style_ erlang for distributed systems. Even thinking about something like a gen_server (http://erlang.org/doc/man/gen_server.html http://erlang.org/doc/man/gen_server.html) makes my head hurt... though if someone can figure out a way to do it that's faithful, more power to them.
dalailambda 9y ago
While I agree that the OTP perhaps is not as easily statically typed, since it was built with Erlang in mind, I do think that static typing adds a layer of robustness to distributed systems, especially if you design it that way upfront. In my experience the problem comes when you try to apply static typing to a dynamic system.
- leshow 9y ago
  > In my experience the problem comes when you try to apply static typing to a dynamic system. As in, because it compiles down to a dynamic system, it's no good? There are plenty of languages that give us strong static guarantees and compile down to dynamic or untyped languages. Look at Purescript, Elm, etc. They all do quite well compiling down to JS. Don't forget that assembly isn't strongly typed either, and most languages compile down to that. I don't see anything wrong with a static typed layer that compiles to dynamic code, the interface you're providing is still type safe.
  - dalailambda 9y ago
    In regards to Purescript/Typescript, they're both statically typed and that results in friction when trying to integrate with the existing JavaScript ecosystem/libraries. Erlang/OTP might be different, but there will probably be situations where the type system is either incompatible with a certain library, or the type system is made less strict (e.g. an any type).
    - leshow 9y ago
      That wasn't what I got out of the previous post, it seemed to be saying there was something inherently unsafe about compiling down to a dynamic language.
nv-vn 9y ago
I suggest you look into Session Typing, specifically Multiparty Session Types as these provide a refreshing approach to the problem of communicating threads. A lot of it is still kind of experimental but there's some good traction being made for sure and it's probably as expressive as you'd need to get to model 95% of the type information in an Erlang program. Type inference obviously isn't a choice yet, but I think a good language offering some of these features on the BEAM VM is all that is needed to make them hit the mainstream and actually get used for real software so that more work can go into the theory, etc. The problem is being solved on the bleeding edge of things, just not as fast as Erlang itself is progressing.
runeks 9y ago
> Distributed systems just seem to too thorny for static types to subjugate/bend to their will. The more I've learned to leverage types, the more I realize that it's my limited knowledge of type systems that prevents me from expressing something in it. Types do not bend to the will of programs; programs bend to the will of types (in statically typed languages). > Sure, you can declare global invariants ahead of time that your cluster must uphold, but it's a bit less "distributed" in a real sense then I don't understand. The components of distributed systems communicate via protocols. What prevents the implementation of these protocols from leveraging type safety, thus transforming a runtime error into a compile-time one? Static typing is about catching programmer mistakes, by communicating your intent to a compiler -- "I expect the type of this to be a Maybe Int, fail if that's not the case". There's no essential difference between a test informing you that a value-level property doesn't hold up at runtime, and a type error, informing you that a type-level property doesn't hold up at compile-time.
- platz 9y ago
  > What prevents the implementation of these protocols from leveraging type safety Global invariants of a running distributed system are different than local invariants in a single program that you can stop, deploy re-compiled binaries to, and then start again. Now, you can use static types in actor systems, and they are some of these that exist. These typed actor systems don't do all the same things that erlang/OTP does (that may be ok - maybe you don't need them). If your use case fits into what the typed actor systems actor systems provide, by all means, one of those are probably a better fit for you.
Tarean 9y ago
I think it is necessary to go with an erlang like style if one wants to get a remotely acceptable cost model. I am not sure how much static typing actually hinders and how much that is a matter of tooling, though. Maybe static typing could do things like checking whether the new version will be compatible with other nodes before deploying?

salimmadjd 9y ago

Really hoping this project gets more traction. I'm learning Elm now and I'm really liking the syntax to the level that other languages feel rather cluttered to me now. The more I'm playing with types and learning to leverage them, the more I appreciate their power (yes, I'm late to the game) so making this statically typed is very interesting. However, there seem to be a saturation of new languages and not sure if there is enough eyeballs left for a new language that does not have a large corporate backing (FB, Google, Apple) or happens not to arrive on a perfect time with the right set of answers. Maybe BEAM, ML/Elm syntax and static typing is what everyone else is looking for. Edit: Video posted today of creator of Alpaca (Jeremy Pierre) giving a talk at Erlang Factory. It gives a nice overview of the state of the language - https://www.youtube.com/watch?v=cljFpz_cv2E https://www.youtube.com/watch?v=cljFpz_cv2E

platz 9y ago
> enough eyeballs left for a new language that does not have a large corporate backing It's a solid point if the goal is winner-take-all style competitive victory. But I'm not sure software should co-op SV-startup-business exponential growth-or-die mindset. What happened to hacker culture? Are open source developers corporatist now? /end-speculative-rant
- bbcbasic 9y ago
  Open source is massively corporatist. Developers working for $0 to create tool chains that other developers spend their weekend learning so that the shareholders of their employers can increase their wealth.
- alextheparrot 9y ago
  There does seem to be a significant push to contribute to open-source at many large companies. Having money and people contributing as part (or all) of their day job can be quite a boon for projects. I agree with the premise of your point, though.
- salimmadjd 9y ago
  The point I was making is for something to get enough traction, so that it would get active contributors who help mature the language, tools, etc. I think there are only handful of people out there who can contribute in a meaningful way for a project like this. If they are consumed working on open source Swift or doing pull request on many things pushed by FB or Google or working contributing to existing projects like GHC, etc. Then the Alpaca project wont get the contributors it needs to show progress. If there is no progress, it falls into a vicious circle of no progress -> no traction -> no contributors -> no progress

sergiotapia 9y ago

>Apache License, Version 2.0 Pardon my ignorance, but why not make it MIT and completely avoid any licensing issues?

geofft 9y ago
The Apache license has an explicit patent grant (the MIT license says "permission to use", which isn't a copyright grant, so it probably has an implicit patent grant), and an explicit statement that patches intentionally submitted for merge are submitted back under the Apache license. The reason we have licenses at all instead of the Unlicense or similar is to make things unambiguous for courts and lawyers. Explicit is better than implicit. The length of the MIT license isn't actually a feature. (And the contribution section seems like it avoids licensing issues that MIT doesn't.)
- dmm 9y ago
  License bikeshedding is fun, so please indulge me. People dislike apache because it's complicated and requires annoying notices on distribution of modified versions. Debian and fsf say it's free. OpenBSD believes the patent provisions are non-free and refuses to include apache licensed software. I think a project is better off having non-trivial contributers sign explicit license grants, even you admit that explicit is better than implicit.
  - liveoneggs 9y ago
    until they are taken to court the licenses are all just assumptions. Did OpenBSD have a lawyer review this decision and post the language somewhere?
  - geofft 9y ago
    I don't understand how the OpenBSD project defines "free", so I can't usefully comment on how they consider the Apache license "non-free". It sounds from https://softwareengineering.stackexchange.com/questions/263227/why-are-apache-2-0-works-excluded-from-openbsd https://softwareengineering.stackexchange.com/questions/2632... like they are reading the existence of an explicit patent clause in the Apache license (regardless of what that clause is!) as an "additional restriction". I want to know whether they believe the MIT license has a patent grant, and if not, what they think "Permission to use" means.

haspok 9y ago

I think this is great news for the Erlang VM: while you wouldn't want to use static typing for any program you write, there is a very specific use-case where you definitely want to do that: embedding business logic in your application. I've been there, done that: encoding business rules in Erlang is no fun, hard to test, and definitely hard to read and modify later. In this particular domain the constraint of types does not slow you down, in fact, it speeds up development. A large amount of unit tests can become unnecessary just because of the type checking. And the more expressive your type system, the fewer tests you need - and the code and the remaining tests can concentrate on validating business logic instead of validating programming language logic ("here is a map - do I have a value with key X in it?" - maybe a bad example because of pattern matching, but I hope you get the idea). You definitely have to be able to interface with OTP, but I don't see it as a huge problem - parts of your application could and should be written in Erlang, there is nothing wrong with that.

im_down_w_otp 9y ago
I'm not sure I'm following you regarding the difficulty of encoding business logic into your code in Erlang. Erlang's function-head matching system is extremely close to being a Prolog-style logic programming system when used a certain way. I've found it extremely easy to take what would normally be a big weird database of rules and values and instead precompile every possible route through the system into a bunch of generated function-head matched function calls + guard clauses. It makes assuring that given inputs will definitely produce correct outputs very easy, and makes processing the rules extremely fast.

Athas 9y ago

I am intrigued by this snippet from the README: type messages 'x = 'x | Fetch pid 'x This appears to define a sum type where one of the variants is left with an implicit constructor. How do you pattern match on that? How do you do type inference?

j14159 9y ago
That's a good question and the short answer is that we're deliberately breaking decidability to allow people to use types that are relatively in line with what we're used to in Erlang (I get into this a bit in the talk linked above). There's a reasonable argument to be made that we should knock this off of course but I'm generally biased in favour of making interoperation with the ecosystem simpler :) We should actually clean up that example you pointed out, matching on the `Fetch` constructor first. What we're really trying to support is unions like: type number = int | float There's a yet un-had argument about the utility of this as well of course and we may want to remove the ability to do this entirely. The way we type this is by actually using type-checking guard functions like `is_integer` to convey information to the typer at compile time.

pka 9y ago

There's also purerl [0], an Erlang backend for PureScript. Would be cool if the two projects could join forces. [0] - https://github.com/purerl/purescript https://github.com/purerl/purescript

jmcdiesel 9y ago

Can anyone explain why it seems like so many new languages are reinventing things that seemingly have little effect, but they seem to be changing them "just because" We have comments in code for decades now. // and /* */ are easily the bigest standard, with # coming in second. Why '--' in this language? Why "``" in another language i saw recently? I cant imagine this gives any real benifit to the coder or the compiler, and it seems to be more difficult because now the IDEs have to be configured for a(nother) new comment type, it has be become muscle memory again, its yet another "common ground" peice of code that requires context switching to change between languages... I get doing new things with the functional features of a language, im all for trying new things and seeing what works... just seems wierd to have so many ways to comment code... such an insignificant part, why change?

SmellyGeekBoy 9y ago
SQL uses '--' for comments.
yawaramin 9y ago
You may be pleased to hear that ReasonML chose /* */ for its OCaml syntax refresh just for familiarity.
lepoetemaudit 9y ago
It's -- in both Elm and Haskell for comments, so it's not new and part of the heritage of the language.
SEMW 9y ago
> why change? They didn't. The syntax is explicitly stated to be a mixture of OCaml and Elm. The -- comment syntax is from Elm. Elm in turn got it from Haskell.