6 ms·

Rust RAII is better than the Haskell bracket pattern

238 points by setra 8y ago

kccqzy 8y ago
The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. Rust drops (runs destructor and deallocates) values as soon as they go out of scope; C++ too. In Haskell you depend on the whims of the GC, which makes RAII unusable. (The Haskell approach of not guaranteeing destructors being called does have its merits; when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2)). Therefore the RAII style wouldn't really work in Haskell. The current bracket approach is still better than RAII in Haskell. That said, the ST-style trick of a phantom type variable is pretty well-known. Unfortunately not many people knew the same trick can be used for non-ST as well. I feel like as a community we should be encouraging this style more often. UPDATE: I wrote the original comment with the incorrect assumption that drop functions will always be called in Rust. This is wrong. Please see child comments.
- annabellish 8y ago
  I don't know if it's fair to call that "the Haskell approach", per-se. That destructors are not guaranteed to run, or run predictably, is generally a property of all fast garbage collectors. If you want a GC that can run quickly, which in a language like haskell where you're going to get lots of small allocations in contexts it would be difficult-to-impossible to efficiently determine the exact moment scopes die, or get the programmer to, you absolutely do, then one of the costs of that is you can't afford to run code for every destroyed object. The linked post is interesting, because I didn't realise "RAII is a much better way of managing resources than destructors" was controversial. It absolutely is, RAII is fast, predictable, and flexible. It's also one of the tradeoffs some languages make to achieve more flexibility in their design by enabling performant automatic garbage collection that doesn't require perfect escape analysis.
  - pjmlp 8y ago
    You can have both, GC, static allocation and RAII, like in Modula-3 for example. Which .NET is finally arriving to, thanks to Midori outcomes. And Java might eventually get there as well, depending on how Project Valhalla ends up. As for languages like Haskell, a mix of bracket and linear types might be the way to go.
    - naasking 8y ago
      I googled "Midori outcomes", but I only found your posts mentioning it. Have a better keyword to search or a link?
      - tveita 8y ago
        http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ is a good set of articles about the Midori project.
      - pjmlp 8y ago
        It is a path to enlightenment with multiple stops. :) Start with Joe Duffy blog posts about Midori architecture. http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ Then hop on to his talks. "Safe Systems Programming in C# and .NET" https://www.infoq.com/presentations/csharp-systems-programming https://www.infoq.com/presentations/csharp-systems-programmi... "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" https://www.youtube.com/watch?v=EVm938gMWl0 https://www.youtube.com/watch?v=EVm938gMWl0 Then you can watch "Inside .NET Native" from Channel 9 https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-Native https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-Native Finally there are the specs and related discussions that lead up to C# 7.3 design. https://github.com/dotnet/corefxlab/tree/master/docs/specs https://github.com/dotnet/corefxlab/tree/master/docs/specs The TL;DR; version, basically async/await, the UWP AOT compiler, improved handling of value types, spans (aka slices), improved GC (TryStartNoGCRegion()) have their roots in System C# used in Midori. Also there are some influences of Singularity, namely Bartok and MDIL, on the WP 8.x AOT compiler, but that is not longer relevant.
- dan00 8y ago
  > The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. There's also no guarantee for Rust/C++ destructors to be called. It's certainly less of an issue then depending on the GC to being called, but if you need absolute correctness, then you shouldn't rely on the destructors.
  - quietbritishjim 8y ago
    If a variable has block scope in C++ (i.e. it is a local variable in a function) then its destructor is guaranteed to be called when the block is finished, regardless of whether that is due to a `return` statement or an exception being thrown (or a `break` or `continue`). In what sense do you disagree? If you allocate an object on the heap with `new` then its destructor isn't called automatically unless you make it so through some other mechanism, but GP comment clearly want claiming that. There are some situations where objects with block scope do not have their destructor called e.g. `_exit()` called, segfault, power cable pulled out. But in that sense nothing is guaranteed.
    - DanWaterworth 8y ago
      If you are consuming an API that provides an object with a destructor, you are correct, you can determine when destructors will be called. The issue is when you produce an API that contains objects with destructors. Since you are handing these entities off to unknown code, you cannot ensure that they will be dropped. This was a problem in scoped threads in Rust.
      - siscia 8y ago
        Can you please dig deeper, that I am not sure I follow. In which case in rust you cannot be sure that "the drop" will be called?
        richardwhiuk 8y ago
        A Rc cycle causing a leak. See the very excellent http://cglab.ca/~abeinges/blah/everyone-poops/ http://cglab.ca/~abeinges/blah/everyone-poops/
- DanWaterworth 8y ago
  > The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. However, this is different than the bracket pattern that the article is taking about. No one in the Haskell community advocates cleaning up resources (like file descriptors, etc) using only destructors.
  - kccqzy 8y ago
    You misunderstood me. I'm explaining why simply adopting RAII is inappropriate in Haskell, even though the author thinks it's a better approach. I've edited my comment to make this clearer.
    - thesz 8y ago
      Author of the article discusses a library - two approaches of different (parts of) libraries. It is quite possible you may need to have RAII somewhere in Haskell code and that's where things like parametrized monads are good: http://blog.sigfpe.com/2009/02/beyond-monads.html http://blog.sigfpe.com/2009/02/beyond-monads.html It is a library and I keep saying that what is usually programming language feature is just a library in Haskell.
- jcelerier 8y ago
  > when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2) thank god they do this. how many times did I have to manually force linux to release sockets because badly coded C programs which opened sockets forgot to release them causing them to hang up for ~5 minutes after the process ended. With proper RAII classes this does not happen.
  - foldr 8y ago
    That has nothing to do with deallocating memory. Of course there are other kinds of resources which are not automatically freed when a program exits.
  - nineteen999 8y ago
    Do you mean orphaned sockets, stuck in FIN_WAIT? Surely what objects are are meant to do is call shutdown(2) syscall - or shutdown(3) C library function - on the socket in their destructor or whatever to prevent that. But I don't think the same applies for memory, once the process is destroyed the kernel should reclaim all memory in the process page tables automatically. Otherwise you'd end up with a pretty trivial way of disabling the system by exhausting all the memory...
    - jcelerier 8y ago
      > Surely what objects are are meant to do is call shutdown(2) syscall - or shutdown(3) C library function well, the problem with non-RAII solutions is that you depend on the whims and talent of the programmer to call shutdown at some point. With a RAII solution like in C++ or Rust you know that if your socket opened successfully, a call to close will necessarily be issued.
      - nineteen999 8y ago
        Maybe I'm being dumb here, but with RAII in C++ at least, doesn't shutdown() and then close() have to be called on the socket by the programmer explicitly in the destructor for the class?
- AnthonyMouse 8y ago
  > when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2) This isn't useless because memory allocation can happen during destruction/exit, e.g. to write some data to the filesystem. Suppose you have a container with a billion objects. The container's destructor iterates over each object, doing some housekeeping that requires making a copy and then deleting the original before moving on to the next object. That requires memory equivalent to one additional object because an original is destroyed following each copy. Stop dellocating memory during destruction/exit and the total memory required doubles, because you have all the copies but still all the originals. There are also some helpful things that happen during deallocation. For example, glibc has double free detection, which strongly implies potential UAF but it's only detected if the second free() actually gets called.
- deleted 8y ago
  [deleted]
EugeneOZ 8y ago
Would be interesting to know why memory leaks are possible in Rust, if RAII is so deeply integrated into language.
- devit 8y ago
  Because the standard library has reference counting with no static checking to avoid cycles, and it was decided to also have safe mem::forget since it can be (mostly) emulated with the former. It has no such static checking because it was deemed to reduce expressiveness, while not impacting memory safety.
- syn_rst 8y ago
  Like this: fn leak() { // Create a 1KiB heap-allocated vector let b = Box::new(vec![0u8; 1024]); // Turn it into a raw pointer let p = Box::into_raw(b); // Then leak the pointer } Obviously that's kind of blatant, but there are more subtle ways to leak memory. Memory leaks aren't considered unsafe, so even though they're undesirable the compiler doesn't guarantee you won't have any. Reference cycles when using Rc<T> are a big one, but generally it's pretty hard to cause leaks by accident. I've only run into one instance of leaking memory outside of unsafe code, and that was caused by a library issue.
  - the_mitsuhiko 8y ago
    The most obvious ways to leak are calling `Box::leak` which is also a very useful API and `mem::forget` (the latter is mostly useful for working with unsafe code).
- pdpi 8y ago
  The same way that memory leaks are possible in Java: rather than a technical bug (you forgot to `free` some buffer), instead you have a semantical bug (you're holding on to a pointer to the data after you're done with it, and that keeps the data alive). Granted, the ownership/borrowing semantics of rust make this a lot harder, but anything that uses Rc/Arc can easily fall prey to it — you can use those to create a reference cycle.
Animats 8y ago
How do you handle errors at resource release? When you close a file, the final writes take place, and they can fail. What's the idiom in Rust for getting them out? Python's "with" clause, and the way it interacts with exceptions, is the only system I've seen that gets this right for the nested case.
- pjmlp 8y ago
  Python is not the only language with a kind of "with" clause. It is done properly in other languages as well, specially if they allow for trailing lambdas.
- DanWaterworth 8y ago
  Python's "with" construct is analogous to the bracket pattern in Haskell that the article is talking about. It also works in the nested case in the presence of exceptions. Furthermore, the issue that Michael has with the bracket pattern in Haskell can also happen in Python.
  - anentropic 8y ago
    True, but in Python the coding mistake would stand out much more because the with block is syntax sugar - it does not look like regular function application, whereas in the Haskell example there is nothing to tell you that withMyResource is using the 'bracket pattern' (except by reading the src) Also I guess in Haskell there is more expectation that the type system should prevent you from expressing runtime errors
    - DanWaterworth 8y ago
      I can see why you might think that, being built into the language, using 'with' in Python in a broken way would be easier to spot. However, having used both languages extensively, I can tell you that, at least for me, there's no discernible difference. I think the reason for this is might be that, in Haskell, a function starting with 'with' is, by convention, using the bracket pattern and the way that you might use such a function would be very similar in structure to the Python way. Something that is often said about C++ is that, you're only ever using 10% of the language, but that everyone uses a different 10% and it's true, but it's true of every language to differing degrees. Everyone has their own way of forming programs, just like everyone has their own slightly different style of playing chess, cooking or forming sentences. When you have a well developed style, you will quickly spot any deviations from it. At that point, it doesn't matter if your style was forced on you by the language or whether it's just a convention that you use. It's certainly true that Haskellers expect a lot from the type system, even compared to other static languages, let alone Python.
T-R 8y ago
Something to keep in mind - linear types are on their way[1], with exactly this usecase in mind. Simon Peyton Jones gave an excellent presentation on the topic[2], briefly discussing exceptions, as well as giving a mention to ResourceT and the phantom type solution in the article (described as channel-passing). [1] https://arxiv.org/abs/1710.09756 https://arxiv.org/abs/1710.09756 [2] https://www.youtube.com/watch?v=t0mhvd3-60Y https://www.youtube.com/watch?v=t0mhvd3-60Y
- thesz 8y ago
  Please, don't add them to the language. Use the library approach instead, it is much more Haskellish.
  - radarsat1 8y ago
    Is it possible? I mean, to add linear types via a library? I feel like it would have been done already if it were.
    - bjoli 8y ago
      I am always impressed by what the ocaml/Haskell people can do compared to my language of choice (scheme). Iirc Oleg Kiselyov implemented proper delimited continuations in ocaml as a library, without touching the runtime or compiler. Something similar has been done in Haskell. I doubt fully dependent types can be implemented in Haskell without extra help by ghc. There has been lots of work in the area, and last time I checked you could simulate DT to some degree, but it never was as powerful as the dependant types in idris. Iirc t were some edge cases where the typing became undecidable.
      - fasquoika 8y ago
        >Iirc Oleg Kiselyov implemented proper delimited continuations in ocaml as a library, without touching the runtime or compiler. To clarify this, the library you're talking about implements most of the functionality in C, reusing the runtime's exception mechanism. So it doesn't require any upstream change to compiler or runtime, but it also can't be implemented in pure OCaml.
hardwaresofton 8y ago
tl;dr - Try rust. The mechanic point of this article is pretty clear: - it's possible to be unsafe in both Haskell and Rust when dealing with resource cleanup - Rust does a bit of a better job in the general case though it has it's own warts (see the other comments, it's hard to deal with issues during `drop`-triggered cleanup) I want to make a muddier meta point -- Rust is the best systems language to date (does anyone know a better one I can look at?). - The person who wrote this article Michael Snoyman[0] is mainly a haskell developer, he's the lead developer behind arguably the most popular web framework, yesod[1]. - Haskell developers generally have a higher standard for type systems, and spend a lot of time (whether they should or not) thinking about correctness due to the pro-activity of the compiler. - These are the kind of people you want trying to use/enjoy your language, if only because they will create/disseminate patterns/insight that make programming safer and easier for everyone down the line -- research languages (Haskell is actually probably tied for the least "researchy" these days in the ML camp) are the Mercedes Benz's of the programming world -- the safety features trickle down from there. - Rust is not a ML family language -- it's a systems language - People who write Haskell on a daily basis are finding their way to rust, because it has a pretty great type system When was the last time you saw a systems language with a type system so good that people who are into type systems were working with it? When was the last time you saw a systems language that scaled comfortably and gracefully from embedded systems to web services? When have you last seen a systems language with such a helpful, vibrant, excited community (TBH I don't think this can last), backed by an organization with values Mozilla's? You owe it to yourself to check it out. As far as I see it rust has two main problems: - Learning curve for one of it's main features (ownership/borrowing) - Readability/Ergonomics (sigils, etc can make rust hard to read) Admittedly, I never gave D[2] a proper shake, and I've heard it's good, but the safety and the emphasis on zero-cost abstractions Rust offers me makes it a non-starter. Rust is smart so I can be dumb. C++ had it's chance and it just has too much cruft for not enough upside -- there's so much struggle required to modernize, to make decisions that rust has had from the beginning (because it's so new). It might be the more stable choice for a x hundred people big corporate project today or next month, but I can't imagine a future where Rust isn't the premier backend/systems language for performance critical (and even those that are not critical) programs in the next ~5 years. I'll go even one step further and say that I think that how much rust forces you to think about ownership/borrowing and how memory is shared around your application is important. Just as Haskell might force you to think about types more closely/methodically (and you're often better for it), Rust's brand of pain seems instructive. [0]: https://www.snoyman.com/ https://www.snoyman.com/ [1]: https://www.yesodweb.com/ https://www.yesodweb.com/ [2]: https://dlang.org/ https://dlang.org/
rbehrends 8y ago
You can also have the exact opposite problem with RAII, where a resource survives the end of a transaction, because there is still a live reference to it hidden away somewhere (say, due to some debugging code holding on to it). This is a classical liveness vs. safety dualism. "Something good will eventually happen" and "nothing bad will ever happen" are promises whose solutions are often in conflict with one another. The general problem — to make transactional state changes and transactional control flow (i.e. expectations about these state changes) match up precisely — is not easy to solve in the general case, especially once you move on to things that are less trivial than simple resource acquisition/release matching.
- jstimpfle 8y ago
  That's also a problem with garbage collection, by the way. GC means memory safety, but not necessarily correctness. In fact, it invites sloppiness, and a kind of sloppiness that sits at a more conceptual level, which could be harder to fix.
  - Ooveex2C 8y ago
    At least in java the recommended way is to only use GC as safety net for resource management. Print a warning when it's cleaned up due to finalizers (or cleaner-refs since v9) and use scope-based cleaning instead where possible.
- hawkice 8y ago
  Oddly, Rust's ownership system really does solve these problems, and Non-lexical lifetimes should eliminate accidental scope-broadening. Unless you are doing some mega-schenanigans, an e.g. MutexGuard gets released precisely when you think. Your point about this being difficult to solve in the general case is true, it's just worth pointing out Rust intends to do that hard thing anyway.
  - steveklabnik 8y ago
    Non-lexical lifetimes do not affect something that implements Drop, so the MutexGuard will, by default, last till the end of its lexical scope. You can still call drop on it manually to release it earlier, though.
platz 8y ago
"RAII" is not "Rust"; "the bracket pattern" is not "Haskell"
- steveklabnik 8y ago
  This is true, but the blog post uses these two as concrete examples, so the title is still accurate.
mark_l_watson 8y ago
Interesting! Michael is one of the more prolific writings and practitioners in the Haskell space (I read just about everything he writes) so it is interesting to also read his take on Rust.
it 8y ago
Isn't this just because withMyResource returns IO a rather than IO ()? It doesn't seem reasonable for it to return the resource.

238 points by setra 8y ago

The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. Rust drops (runs destructor and deallocates) values as soon as they go out of scope; C++ too. In Haskell you depend on the whims of the GC, which makes RAII unusable. (The Haskell approach of not guaranteeing destructors being called does have its merits; when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2)). Therefore the RAII style wouldn't really work in Haskell. The current bracket approach is still better than RAII in Haskell. That said, the ST-style trick of a phantom type variable is pretty well-known. Unfortunately not many people knew the same trick can be used for non-ST as well. I feel like as a community we should be encouraging this style more often. UPDATE: I wrote the original comment with the incorrect assumption that drop functions will always be called in Rust. This is wrong. Please see child comments.

annabellish 8y ago
I don't know if it's fair to call that "the Haskell approach", per-se. That destructors are not guaranteed to run, or run predictably, is generally a property of all fast garbage collectors. If you want a GC that can run quickly, which in a language like haskell where you're going to get lots of small allocations in contexts it would be difficult-to-impossible to efficiently determine the exact moment scopes die, or get the programmer to, you absolutely do, then one of the costs of that is you can't afford to run code for every destroyed object. The linked post is interesting, because I didn't realise "RAII is a much better way of managing resources than destructors" was controversial. It absolutely is, RAII is fast, predictable, and flexible. It's also one of the tradeoffs some languages make to achieve more flexibility in their design by enabling performant automatic garbage collection that doesn't require perfect escape analysis.
- pjmlp 8y ago
  You can have both, GC, static allocation and RAII, like in Modula-3 for example. Which .NET is finally arriving to, thanks to Midori outcomes. And Java might eventually get there as well, depending on how Project Valhalla ends up. As for languages like Haskell, a mix of bracket and linear types might be the way to go.
  - naasking 8y ago
    I googled "Midori outcomes", but I only found your posts mentioning it. Have a better keyword to search or a link?
    - tveita 8y ago
      http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ is a good set of articles about the Midori project.
    - pjmlp 8y ago
      It is a path to enlightenment with multiple stops. :) Start with Joe Duffy blog posts about Midori architecture. http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ Then hop on to his talks. "Safe Systems Programming in C# and .NET" https://www.infoq.com/presentations/csharp-systems-programming https://www.infoq.com/presentations/csharp-systems-programmi... "RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy" https://www.youtube.com/watch?v=EVm938gMWl0 https://www.youtube.com/watch?v=EVm938gMWl0 Then you can watch "Inside .NET Native" from Channel 9 https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-Native https://channel9.msdn.com/Shows/Going+Deep/Inside-NET-Native Finally there are the specs and related discussions that lead up to C# 7.3 design. https://github.com/dotnet/corefxlab/tree/master/docs/specs https://github.com/dotnet/corefxlab/tree/master/docs/specs The TL;DR; version, basically async/await, the UWP AOT compiler, improved handling of value types, spans (aka slices), improved GC (TryStartNoGCRegion()) have their roots in System C# used in Midori. Also there are some influences of Singularity, namely Bartok and MDIL, on the WP 8.x AOT compiler, but that is not longer relevant.
dan00 8y ago
> The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. There's also no guarantee for Rust/C++ destructors to be called. It's certainly less of an issue then depending on the GC to being called, but if you need absolute correctness, then you shouldn't rely on the destructors.
- quietbritishjim 8y ago
  If a variable has block scope in C++ (i.e. it is a local variable in a function) then its destructor is guaranteed to be called when the block is finished, regardless of whether that is due to a `return` statement or an exception being thrown (or a `break` or `continue`). In what sense do you disagree? If you allocate an object on the heap with `new` then its destructor isn't called automatically unless you make it so through some other mechanism, but GP comment clearly want claiming that. There are some situations where objects with block scope do not have their destructor called e.g. `_exit()` called, segfault, power cable pulled out. But in that sense nothing is guaranteed.
  - DanWaterworth 8y ago
    If you are consuming an API that provides an object with a destructor, you are correct, you can determine when destructors will be called. The issue is when you produce an API that contains objects with destructors. Since you are handing these entities off to unknown code, you cannot ensure that they will be dropped. This was a problem in scoped threads in Rust.
    - siscia 8y ago
      Can you please dig deeper, that I am not sure I follow. In which case in rust you cannot be sure that "the drop" will be called?
      - richardwhiuk 8y ago
        A Rc cycle causing a leak. See the very excellent http://cglab.ca/~abeinges/blah/everyone-poops/ http://cglab.ca/~abeinges/blah/everyone-poops/
DanWaterworth 8y ago
> The thing is that, in Haskell, even when you attach a function to run during destruction, the runtime doesn't guarantee that the function will be called promptly, or even at all. However, this is different than the bracket pattern that the article is taking about. No one in the Haskell community advocates cleaning up resources (like file descriptors, etc) using only destructors.
- kccqzy 8y ago
  You misunderstood me. I'm explaining why simply adopting RAII is inappropriate in Haskell, even though the author thinks it's a better approach. I've edited my comment to make this clearer.
  - thesz 8y ago
    Author of the article discusses a library - two approaches of different (parts of) libraries. It is quite possible you may need to have RAII somewhere in Haskell code and that's where things like parametrized monads are good: http://blog.sigfpe.com/2009/02/beyond-monads.html http://blog.sigfpe.com/2009/02/beyond-monads.html It is a library and I keep saying that what is usually programming language feature is just a library in Haskell.
jcelerier 8y ago
> when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2) thank god they do this. how many times did I have to manually force linux to release sockets because badly coded C programs which opened sockets forgot to release them causing them to hang up for ~5 minutes after the process ended. With proper RAII classes this does not happen.
- foldr 8y ago
  That has nothing to do with deallocating memory. Of course there are other kinds of resources which are not automatically freed when a program exits.
- nineteen999 8y ago
  Do you mean orphaned sockets, stuck in FIN_WAIT? Surely what objects are are meant to do is call shutdown(2) syscall - or shutdown(3) C library function - on the socket in their destructor or whatever to prevent that. But I don't think the same applies for memory, once the process is destroyed the kernel should reclaim all memory in the process page tables automatically. Otherwise you'd end up with a pretty trivial way of disabling the system by exhausting all the memory...
  - jcelerier 8y ago
    > Surely what objects are are meant to do is call shutdown(2) syscall - or shutdown(3) C library function well, the problem with non-RAII solutions is that you depend on the whims and talent of the programmer to call shutdown at some point. With a RAII solution like in C++ or Rust you know that if your socket opened successfully, a call to close will necessarily be issued.
    - nineteen999 8y ago
      Maybe I'm being dumb here, but with RAII in C++ at least, doesn't shutdown() and then close() have to be called on the socket by the programmer explicitly in the destructor for the class?
AnthonyMouse 8y ago
> when many C++ and Rust programs are about to end, they spend the last few cycles uselessly deallocating memory that would've immediately been freed via _exit(2) This isn't useless because memory allocation can happen during destruction/exit, e.g. to write some data to the filesystem. Suppose you have a container with a billion objects. The container's destructor iterates over each object, doing some housekeeping that requires making a copy and then deleting the original before moving on to the next object. That requires memory equivalent to one additional object because an original is destroyed following each copy. Stop dellocating memory during destruction/exit and the total memory required doubles, because you have all the copies but still all the originals. There are also some helpful things that happen during deallocation. For example, glibc has double free detection, which strongly implies potential UAF but it's only detected if the second free() actually gets called.
deleted 8y ago
[deleted]

EugeneOZ 8y ago

Would be interesting to know why memory leaks are possible in Rust, if RAII is so deeply integrated into language.

devit 8y ago
Because the standard library has reference counting with no static checking to avoid cycles, and it was decided to also have safe mem::forget since it can be (mostly) emulated with the former. It has no such static checking because it was deemed to reduce expressiveness, while not impacting memory safety.
syn_rst 8y ago
Like this: fn leak() { // Create a 1KiB heap-allocated vector let b = Box::new(vec![0u8; 1024]); // Turn it into a raw pointer let p = Box::into_raw(b); // Then leak the pointer } Obviously that's kind of blatant, but there are more subtle ways to leak memory. Memory leaks aren't considered unsafe, so even though they're undesirable the compiler doesn't guarantee you won't have any. Reference cycles when using Rc<T> are a big one, but generally it's pretty hard to cause leaks by accident. I've only run into one instance of leaking memory outside of unsafe code, and that was caused by a library issue.
- the_mitsuhiko 8y ago
  The most obvious ways to leak are calling `Box::leak` which is also a very useful API and `mem::forget` (the latter is mostly useful for working with unsafe code).
pdpi 8y ago
The same way that memory leaks are possible in Java: rather than a technical bug (you forgot to `free` some buffer), instead you have a semantical bug (you're holding on to a pointer to the data after you're done with it, and that keeps the data alive). Granted, the ownership/borrowing semantics of rust make this a lot harder, but anything that uses Rc/Arc can easily fall prey to it — you can use those to create a reference cycle.

Animats 8y ago

How do you handle errors at resource release? When you close a file, the final writes take place, and they can fail. What's the idiom in Rust for getting them out? Python's "with" clause, and the way it interacts with exceptions, is the only system I've seen that gets this right for the nested case.

pjmlp 8y ago
Python is not the only language with a kind of "with" clause. It is done properly in other languages as well, specially if they allow for trailing lambdas.
DanWaterworth 8y ago
Python's "with" construct is analogous to the bracket pattern in Haskell that the article is talking about. It also works in the nested case in the presence of exceptions. Furthermore, the issue that Michael has with the bracket pattern in Haskell can also happen in Python.
- anentropic 8y ago
  True, but in Python the coding mistake would stand out much more because the with block is syntax sugar - it does not look like regular function application, whereas in the Haskell example there is nothing to tell you that withMyResource is using the 'bracket pattern' (except by reading the src) Also I guess in Haskell there is more expectation that the type system should prevent you from expressing runtime errors
  - DanWaterworth 8y ago
    I can see why you might think that, being built into the language, using 'with' in Python in a broken way would be easier to spot. However, having used both languages extensively, I can tell you that, at least for me, there's no discernible difference. I think the reason for this is might be that, in Haskell, a function starting with 'with' is, by convention, using the bracket pattern and the way that you might use such a function would be very similar in structure to the Python way. Something that is often said about C++ is that, you're only ever using 10% of the language, but that everyone uses a different 10% and it's true, but it's true of every language to differing degrees. Everyone has their own way of forming programs, just like everyone has their own slightly different style of playing chess, cooking or forming sentences. When you have a well developed style, you will quickly spot any deviations from it. At that point, it doesn't matter if your style was forced on you by the language or whether it's just a convention that you use. It's certainly true that Haskellers expect a lot from the type system, even compared to other static languages, let alone Python.

T-R 8y ago

Something to keep in mind - linear types are on their way[1], with exactly this usecase in mind. Simon Peyton Jones gave an excellent presentation on the topic[2], briefly discussing exceptions, as well as giving a mention to ResourceT and the phantom type solution in the article (described as channel-passing). [1] https://arxiv.org/abs/1710.09756 https://arxiv.org/abs/1710.09756 [2] https://www.youtube.com/watch?v=t0mhvd3-60Y https://www.youtube.com/watch?v=t0mhvd3-60Y

thesz 8y ago
Please, don't add them to the language. Use the library approach instead, it is much more Haskellish.
- radarsat1 8y ago
  Is it possible? I mean, to add linear types via a library? I feel like it would have been done already if it were.
  - bjoli 8y ago
    I am always impressed by what the ocaml/Haskell people can do compared to my language of choice (scheme). Iirc Oleg Kiselyov implemented proper delimited continuations in ocaml as a library, without touching the runtime or compiler. Something similar has been done in Haskell. I doubt fully dependent types can be implemented in Haskell without extra help by ghc. There has been lots of work in the area, and last time I checked you could simulate DT to some degree, but it never was as powerful as the dependant types in idris. Iirc t were some edge cases where the typing became undecidable.
    - fasquoika 8y ago
      >Iirc Oleg Kiselyov implemented proper delimited continuations in ocaml as a library, without touching the runtime or compiler. To clarify this, the library you're talking about implements most of the functionality in C, reusing the runtime's exception mechanism. So it doesn't require any upstream change to compiler or runtime, but it also can't be implemented in pure OCaml.

hardwaresofton 8y ago

tl;dr - Try rust. The mechanic point of this article is pretty clear: - it's possible to be unsafe in both Haskell and Rust when dealing with resource cleanup - Rust does a bit of a better job in the general case though it has it's own warts (see the other comments, it's hard to deal with issues during `drop`-triggered cleanup) I want to make a muddier meta point -- Rust is the best systems language to date (does anyone know a better one I can look at?). - The person who wrote this article Michael Snoyman[0] is mainly a haskell developer, he's the lead developer behind arguably the most popular web framework, yesod[1]. - Haskell developers generally have a higher standard for type systems, and spend a lot of time (whether they should or not) thinking about correctness due to the pro-activity of the compiler. - These are the kind of people you want trying to use/enjoy your language, if only because they will create/disseminate patterns/insight that make programming safer and easier for everyone down the line -- research languages (Haskell is actually probably tied for the least "researchy" these days in the ML camp) are the Mercedes Benz's of the programming world -- the safety features trickle down from there. - Rust is not a ML family language -- it's a systems language - People who write Haskell on a daily basis are finding their way to rust, because it has a pretty great type system When was the last time you saw a systems language with a type system so good that people who are into type systems were working with it? When was the last time you saw a systems language that scaled comfortably and gracefully from embedded systems to web services? When have you last seen a systems language with such a helpful, vibrant, excited community (TBH I don't think this can last), backed by an organization with values Mozilla's? You owe it to yourself to check it out. As far as I see it rust has two main problems: - Learning curve for one of it's main features (ownership/borrowing) - Readability/Ergonomics (sigils, etc can make rust hard to read) Admittedly, I never gave D[2] a proper shake, and I've heard it's good, but the safety and the emphasis on zero-cost abstractions Rust offers me makes it a non-starter. Rust is smart so I can be dumb. C++ had it's chance and it just has too much cruft for not enough upside -- there's so much struggle required to modernize, to make decisions that rust has had from the beginning (because it's so new). It might be the more stable choice for a x hundred people big corporate project today or next month, but I can't imagine a future where Rust isn't the premier backend/systems language for performance critical (and even those that are not critical) programs in the next ~5 years. I'll go even one step further and say that I think that how much rust forces you to think about ownership/borrowing and how memory is shared around your application is important. Just as Haskell might force you to think about types more closely/methodically (and you're often better for it), Rust's brand of pain seems instructive. [0]: https://www.snoyman.com/ https://www.snoyman.com/ [1]: https://www.yesodweb.com/ https://www.yesodweb.com/ [2]: https://dlang.org/ https://dlang.org/

rbehrends 8y ago

You can also have the exact opposite problem with RAII, where a resource survives the end of a transaction, because there is still a live reference to it hidden away somewhere (say, due to some debugging code holding on to it). This is a classical liveness vs. safety dualism. "Something good will eventually happen" and "nothing bad will ever happen" are promises whose solutions are often in conflict with one another. The general problem — to make transactional state changes and transactional control flow (i.e. expectations about these state changes) match up precisely — is not easy to solve in the general case, especially once you move on to things that are less trivial than simple resource acquisition/release matching.

jstimpfle 8y ago
That's also a problem with garbage collection, by the way. GC means memory safety, but not necessarily correctness. In fact, it invites sloppiness, and a kind of sloppiness that sits at a more conceptual level, which could be harder to fix.
- Ooveex2C 8y ago
  At least in java the recommended way is to only use GC as safety net for resource management. Print a warning when it's cleaned up due to finalizers (or cleaner-refs since v9) and use scope-based cleaning instead where possible.
hawkice 8y ago
Oddly, Rust's ownership system really does solve these problems, and Non-lexical lifetimes should eliminate accidental scope-broadening. Unless you are doing some mega-schenanigans, an e.g. MutexGuard gets released precisely when you think. Your point about this being difficult to solve in the general case is true, it's just worth pointing out Rust intends to do that hard thing anyway.
- steveklabnik 8y ago
  Non-lexical lifetimes do not affect something that implements Drop, so the MutexGuard will, by default, last till the end of its lexical scope. You can still call drop on it manually to release it earlier, though.

platz 8y ago

"RAII" is not "Rust"; "the bracket pattern" is not "Haskell"

steveklabnik 8y ago
This is true, but the blog post uses these two as concrete examples, so the title is still accurate.

mark_l_watson 8y ago

Interesting! Michael is one of the more prolific writings and practitioners in the Haskell space (I read just about everything he writes) so it is interesting to also read his take on Rust.

it 8y ago

Isn't this just because withMyResource returns IO a rather than IO ()? It doesn't seem reasonable for it to return the resource.