11 ms·

Taming Go’s memory usage, or how we avoided rewriting our client in Rust

359 points by jeanyang 5y ago

notamy 5y ago
Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves. Isn't part of the point of Rust that you don't manage memory yourself, and rather that the compiler is smart enough to manage it for you?
- Spartan-S63 5y ago
  I feel that this is one of those common misconceptions about Rust. Rust's memory management is nothing like C or non-modern C++'s with malloc/free or new/delete. Rust uses modern-C++'s RAII model, typically, to allocate memory. The compiler is smart enough to know when to call drop() (which is essentially free/delete, but with the possibility of additional behavior). You can also call drop() yourself. What I think people _should_ focus on with Rust versus Go (et al) is that Rust allows you to choose where you _place_ memory. You can choose the stack or the heap. The placement can matter in hot regions of code. Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries.
  - angelzen 5y ago
    Tangentially, I did a bit of Rust work recently. I was sadly unable to find a concise credible answer to a rather elementary best-practices question: How does ownership interact with nested datastructures? Is it possible to build a heap tree without Boxing every node explicitly?
    - steveklabnik 5y ago
      You'd do the same stuff you'd do in C++ here; allocate every node explicitly, use an arena, whatever you want.
    - miloignis 5y ago
      This question is a bit subtle, it depends on exactly what you mean. You could make a tree using only borrow checked references and the compiler would make sure that parent nodes go out of scope at the same time or before the child nodes they point to, but I don't think that's what you're talking about. In general, if it's a datastructure where you have to use pointers, you'll have them Box'ed, but you would try to avoid that if you can. In your example of a heap, you'd want to use an array-based implementation, probably backed by a growable Vec, and use indexes internally. A peek function would still return a normal Rust reference to the data, and the borrow checker would make sure that you don't mutate the heap's backing array while that reference was still in use, etc.
      - slaymaker1907 5y ago
        I never thought about using a Vec for these, but that is a great idea for keeping the memory management sane for tree/linked lists. One thing I would add that you need to be wary of destructors with large pointer data structures in Rust since it can easily stack overflow. When using Option<Box<T>> you need to be careful to call Option::take on the pointers in a loop to avoid stack overflow.
    - bobbylarrybobby 5y ago
      You might be interested in this: https://rust-unofficial.github.io/too-many-lists/ https://rust-unofficial.github.io/too-many-lists/
      - angelzen 5y ago
        Thanks. Saw that before, but the credibility/length ratio wasn't high enough to read it more carefully. It appears that we do have to Box/Rc/Arc nodes in a recursive datastructure. Doable, but a bit on the inconvenient side. struct Node { elem: i32, next: Option<Box<Node>>, }
  - jcelerier 5y ago
    It kills me that RAII is considered modern c++. It's there since 1983 aha, what do you think fstream and std::vector are if not RAII wrappers over files or memory
    - oconnor663 5y ago
      I think before the introduction of move semantics in C++11, there were a lot of cases where you needed new and delete to get basic things working. (Moving an fstream around is a relevant example.) So the modern rule of "don't use new and delete in application code" really wasn't practical before that.
      - jcelerier 5y ago
        No, pretty much everything could be done with swap (like moving an fstream as you say). Sure, it's a bit more cumbersome, but it was still RAII.
      - nyanpasu64 5y ago
        I suppose RAII is an old concept, but move semantics allowing RAII to transfer ownership and avoid manual new/free of non-copied resources was uncommon until C++11.
    - bluGill 5y ago
      before unique_ptr we didn't have a good way to handle raii for a lot of things. I wrote a lot of RAII wrappers for various things (still do, but a lot less). Attempts like auto_ptr show just how hard it is to make raii work well before C++11. Yes we had RAII, but it didn't work for a lot of cases where we needed it.
  - brink 5y ago
    > Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries. Use channels whenever possible.
    - kinjba11 5y ago
      Channels are not always the best solution (unless you're referring to Rust channels?) https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-should-feel-bad/ https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s...
      - brink 5y ago
        Yeah, Rust's crossbeam channels are actually really good.
- steveklabnik 5y ago
  Yes, Rust kinda doesn't fit super cleanly into a very black/white binary here. It is automatic in the sense that you do not generally call malloc/free. The compiler handles this for you. At the same time, you have a lot more control than you do in a language with a GC, and so to some people, it feels more manual. It's also like, a perception thing in some sense. Imagine someone writes some code. They get a compiler error. There are two ways to react to this event: "Wow the compiler didn't make this work, I have to think about memory all the time." "Ah, the compiler caught a mistake for me. Thank goodness I don't have to think about this for myself." Both perceptions make sense, but seem to be in complete and total opposition.
  - throwaway894345 5y ago
    "Manual vs automatic" is mostly just a semantic problem IMHO. We could say "runtime versus compile time" to be more precise, but maybe there are problems there as well. The more interesting question to me is "how much time/energy do I spend thinking about memory management, and is that how my time is best spent?". In cases of high performance code, you might spend more time fighting with the GC than you would with the borrow checker to get the performance you need, but for everything else the hot paths are so few and far between you're most likely better off fighting with the GC 1% of the time and not fighting anything the other 99%. The Rust community has done laudable work in bringing down the cognitive threshold of "manual / compile-time" memory management, but I think we're finding out that the returns are diminishing quickly and there's still quite a chasm between borrow checking and GC with respect to developer velocity.
    - steveklabnik 5y ago
      "developer velocity" is also, in some sense, a semantic question. I am special, of course, but basically, if you include things like "time fixing bugs that would have been prevented in Rust in the first place", my velocity is higher in Rust than in many GC'd languages I've used in the past. It just depends on so many factors it's impossible to say definitively one way or another.
      - 5y ago
- dgb23 5y ago
  You are still managing memory in Rust, it’s just more constrained, statically checked and inferred. Within those constraints you have full control.
- mullr 5y ago
  > Isn't part of the point of Rust that you don't manage memory yourself, and rather that the compiler is smart enough to manage it for you? For trivial cases, kind of. But once you start to do anything remotely sophisticated, no. Everything you do in Rust is checked w.r.t. memory management, but you still need to make many choices about it. All the stuff about lifetimes, borrowing, etc: that's memory management. The compiler's checking it for you, but you still need to design stuff sanely, with memory management (and the checking thereof) in mind. It's easy to back yourself into a corner if you ignore this.
- NovemberWhiskey 5y ago
  Go = you do no explicit memory management and the GC/runtime takes care of it for you Rust = when writing your code, you explicitly describe the ownership and lifetime of your objects and how your functions are allowed to consume/copy etc. them and get safety as a result C = when writing your code, you explicitly allocate and free your objects and you get no assistance from the language about when it is safe to copy/dereference/free/etc. a pointer/allocation
  - throwaway894345 5y ago
    I prefer to think that in Go you don't do explicit memory management by default, while in Rust you do. Although you can laboriously opt out of explicit memory management (e.g., by tagging everything Rc<> or Gc<> and all of the ceremony that entails).
- slaymaker1907 5y ago
  While some commenters have pointed out that you still need to deal with lifetimes/thinking about where stuff lives, in practice you can avoid almost all of this by using Rc<Type> instead of Type everywhere (or Arc in a multithreaded scenario). Yes Rc and equivalents have a performance overhead, but for many use cases the overhead really isn't that bad since you typically aren't creating tons of copies. In practice, I've found one can ignore lifetimes in almost all cases even when using references except when storing them in structs or closures. So really you would just need to increment the Rc counter for structs/closures outside of allocation/deallocation which is dominated by calls to malloc/free.
  - throwaway894345 5y ago
    I've tried this before and it was so laborious that I regretted it. I'm not sure I saved myself any time over writing "vanilla" Rust or whatever one might call the default alternative. If I was really interested in writing Rust more quickly, I would just clone everything rather than Rc it, but in whichever case you're still moving quite a lot slower than you would in Go.
    - nyanpasu64 5y ago
      I've tried writing Rc-oriented Rust (for gtk-rs) too, and struggled hard with the pervasive cloning/aliasing needed, having to use weak references to avoid leaking memory, and the clone!() macro turning off rustfmt for all code in the method body. In fact, I'd rather deal with Qt-style memory management, with single QObject ownership, QPointer (which is kinda like a weak pointer), and praying you don't use-after-free. (Normally I use subclassing in Qt to associate extra state with a widget, but gtk-rs's subclassing API was arcane and boilerplate-heavy. Perhaps there's alternative paradigms for state management that follows Rust's single ownership principle better. Some people take a React/Elm-style approach, but I don't think virtual DOMs and diffing the entire UI tree on each user interaction are the last word on GUI interactivity and updates, and I don't find the added memory of virtual DOMs and CPU of generating/diffing them acceptable, but rather "pure overhead" to be eliminated in favor of minimal targeted UI state updates.)
- pjc50 5y ago
  You can also kind of do your own management of memory in GC languages, you just have to be extremely careful in code review to spot inadvertant allocations in the hot path. A great example is the "LMAX Disruptor" in Java: https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ The trick is to pre-allocate all your objects and buffers and reuse them in a ring buffer. Similar techniques work in zero-malloc embedded C environments.
- sreque 5y ago
  I'm not a rust user, but I would argue you are still managing memory manually, you're just doing a lot of it through rust's type system, which can check for errors at compile time, rather than through runtime APIs like the C or C++ standard library. The question then becomes whether it is easier to manage memory through Rust's type system versus via standard runtime APIs. From what I've read, Rust memory management actually requires more work but provides fantastic safety guarantees. This could mean that rust actually lowers productivity at first, but as the complexity of the code base grows, some of that productivity is restored or even supercedes C/C++ because you spend no time chasing runtime memory bugs. For some products or projects, the costs of shipping a security flaw caused by a memory bug exploit could be high enough that a drop in productivity from Rust relative to C is still more than justified due to external costs that Rust mitigates.
- oconnor663 5y ago
  I think sometimes the "compiler manages memory for you" concept gets overplayed a bit. It's not as complex as that description makes it sound. If you understand C++ destructors, it's really the same thing. Objects get destroyed when they go out of scope, and any memory or other resources they own get freed. The differences come up when you look at what happens when you make a mistake, like holding a pointer to a freed object. (Rust catches these mistakes at compile time, which does indeed involve some new complexity.)
  - pjmlp 5y ago
    Try to implement a data structure that works across async runtimes, or a couple of GUI widgets, then you will get the point why some of us complain about the borrow checker, even with decades of experience in C and C++.
    - chakkepolja 5y ago
      Or rather, acting as if rust is positioned to replace general purpose languages.
- saghm 5y ago
  There are already a lot of replies to this comment explaining the ideas behind Rust memory management in different ways, but I'll throw in my handwavy explanation as well: In GC languages, memory management is generally runtime through the interpreter/runtime. In C, memory management is generally done at programming time by the (human) programmer. In Rust, memory management is generally done at compile time by the compiler. There are exceptions in all three cases, but the "default" paradigm of a language informs a lot about how it's designed and used.
- sgift 5y ago
  I also was confused about that part but for another reason: The whole post is basically "despite go having a GC we had to manually manage the memory to make it work" and then the anti-rewrite is "go does memory management for us". IMO people sometimes have really weird ideas what is and isn't part of managing memory.
- bilboa 5y ago
  While you may not have to directly call malloc and free in Rust, the memory management still feels very manual compared to a language with GC. When I want to pass an object around I have to decide whether to pass a &_, a Box<_>, Rc<_>, or Rc<RefCell<_>>, or a &Rc<RefCell<_>>, etc. And then there are lifetime parameters, and having to constantly be aware of relative lifetimes of objects. Those are all manual decisions related to memory management that you have to constantly make in Rust that you wouldn't need to think about in Go or Python or Java. Similarly, idiomatic modern C++ rarely needs new and delete calls, but I'd still say it has manual memory management. I suppose it's reasonable to talk about degrees of manual-ness, and say that memory management in Rust or modern C++ is less manual than C, but more manual than Go/Python/Java.
- bugmen0t 5y ago
  It's very easy to "make it work" while fencing with compiler warnings by just copying things around instead of developing a clear sense of memory ownership. I've seen myself fall into this trap. The upside, coming from C, is that you don't have terrible memory safety issues. The downside is that you have the same data copied all over the place and (accidentally) allocate like a mad man. Managed memory is not inherently bad or good.
srcreigh 5y ago
> But our profile wasn’t ever showing us 500GB of live data, just a little bit more than 200MB in the worst cases. This suggested to me that we’d done all we could with live objects. Is this a typo? Weren't seeing 500 MB of live data, just a little more than 200MB in the worst case? EDIT: Btw, I read the entire article. It was fascinating, thank you!
- deleted 5y ago
  [deleted]
- markgritter 5y ago
  Yes, that's a typo, thanks!
void_mint 5y ago
Rebuilding in a different language is just trading one problem set for another. Better using the tools you've already taken on is a much better strategy if you don't have the money to hire a whole new set of devs or a year to burn onboarding onto a new language.
wrs 5y ago
Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the language has GC, it’s that memory usage was just not considered. If you want performance, you have to pay attention to allocations no matter what kind of memory management your language has. And as the article demonstrates, if you pay attention, you can get performance no matter what kind of memory management your language has.
- zamadatix 5y ago
  Rewrites can definitely help but rushing into them before doing these other things is going to net you a lot less gain for the time.
- xondono 5y ago
  > Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.
  - coliveira 5y ago
    This is less an argument for a rewrite than an argument for redesigning parts of your codebase, which can be done much more easily than a complete rewrite.
    - xondono 5y ago
      The tricky thing is that it’s easy to end up with a result that’s not far off. Some modules will improve, but a lot of the time these kind of bottlenecks tend to happen because the performant version is not very idiomatic (feels weird), it’s too verbose, or it’s to confusing to think through. Unless you have the same team (and they learned the lesson the first time), it’s very likely to end up with modules that perform in a similar way. Sometimes changing the language makes thinking about the problems easier.
henning 5y ago
> Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves. No.
- arsome 5y ago
  Yeah, sounds like someone doesn't understand lifetimes and RAII. Even in modern C++ the number of times you have to actually think about memory management instead of lifetimes is basically zero unless you have to work with old libraries.
  - david422 5y ago
    Even then, just add a wrapper and off you go.
  - tsimionescu 5y ago
    But thinking about lifetimes and RAII is 90% of memory management. Basically whether you write C, C++, or Rust, you have to track ownership the same ways, the only thing that changes is how much the compiler helps you with that. However, if you write your program in Java, Lisp or Haskell, you simply do not care about ownership for memory-only objects, and can structure your program significantly differently. This can have significant impact on certain types of workflows, especially when it comes to shared objects. A well-known example is when implementing lock-free data structures based on compare-and-swap, where you need to free the old copy of the structure after a successful compare-and-swap; but, you can't free it since you don't know who may still be reading from it. Here is an in-depth write-up from Andrei Alexandrescu on the topic [0]. Note: I am using "object" here in the sense from C - basically any piece of data that was allocated. [0] http://erdani.org/publications/cuj-2004-10.pdf http://erdani.org/publications/cuj-2004-10.pdf
    - bluGill 5y ago
      With modern C++ your memory checklist is two steps: put it on the stack, put it in a unique_ptr on the stack. There are more steps after that, but you almost never get to them and wouldn't remember them if you discovered the need for them (which is okay because you never get there).
CraigJPerry 5y ago
> For our application, it would be acceptable to simply exit when memory usage gets too large Could you not just set a ulimit on memory usage of the process in that case? (And use another process as the parent, e.g. a supervisor or init, to avoid exiting the container and just restart the process instead)
geodel 5y ago
Well good for author that they were able to fix the issue. However I think writing efficient code in even in managed memory languages for large, heavily used service is kind of normal thing and not above and beyond normal work.
tptacek 5y ago
The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were using a protobuf diffing library that used `reflect` under the hood, which allocates. They generated their own explicit object inspection thingies. * They stopped compiling regexps on the fly and moved the regexps to package variables. (I actually don't know if this was a significant win; there might just be the three big wins.) This is a great article. But none of these seem Go-specific†, or even GC-specific. They're doing something really ambitious (slurping packets up off the wire against busy API servers, reassembling them in userland into streams, and then parsing the contents of the streams). Memory usage was going to be fiddly no matter what they built with. The problems they ran up against seem pretty textbook. Frankly I'm surprised Go acquitted itself as well as it did here. † Maybe the perils of `reflect` count as a Go thing; it's worth noting that there's folk wisdom in Go-land to avoid `reflect` when possible.
- kevingadd 5y ago
  Reflection APIs seem to be pretty messy and slow in every runtime I've ever used, perhaps because the idea of optimizing them might encourage more use. The C# reflection APIs also allocate a lot.
  - aidenn0 5y ago
    Before writing Clojure, Rich Hickey wrote FOIL[1], which used sockets to communicate between common lisp and the JVM (or CLR). When asked about making it in-process, Rich observed that the reflection overhead on the JVM was often as large, or larger, than the serialization overhead, so the gains to be had were limited. 1: http://foil.sourceforge.net/ http://foil.sourceforge.net/
    - hinkley 5y ago
      From what I recall, the Java team copped to the intentionally slow accusation, but that started to change when they decided to embrace the notion of other languages besides Java running on the JVM. Unfortunately that would have been shortly after Clojure was born. It took a few releases for them to really improve that situation, and that was still shortly before they started doing faster releases.
typical182 5y ago
Very nice write up. Go’s focus on simplicity means that there is only a single parameter, SetGCPercent, which controls how much larger the heap is than the live objects within it. FWIW, there is a new proposal from a member of the core Go team to add a second GC knob in the form of a soft limit on total memory: https://github.com/golang/proposal/blob/master/design/48409-soft-memory-limit.md https://github.com/golang/proposal/blob/master/design/48409-... It includes some provisions to make sure that the application can keep making progress and avoid death spirals (part of the reason why it is a "soft" limit), and also includes some new GC-related telemetry. From the blog write up, a second GC knob with a soft limit might have only been a minor help here, with the bigger wins coming from the code changes they described in the blog.
- _ph_ 5y ago
  Hmm, I wonder whether a better alternative might be to be able to set a minimum memory size to use. It is a bit annoying to start a go program when you exactly know you are going to need 1G of memory and after the first 1M allocated it tries to GC before growing the heap. If you could set a minimum memory size, then you could get away with a very low value for GOGC to limit the space overhead beyond your set memory size.
option_greek 5y ago
I have a feeling that they will end up eventually rewriting this in Rust as the use case they describe is where a non GC language can definitely provide more performance (beyond the case they solved). APM tools usually need to be more performant to ensure they add as little overhead to the actual service as possible. I guess what's helping here is that this is passive monitoring which allows a little lag in the system. Question relavent here is will there be more issues with memory in general based on their current roadmap.
brundolf 5y ago
"How we avoided rewriting in Rust" feels like clickbait given that the answer is "our problems were algorithmic, not language-specific"
- throwaway894345 5y ago
  I assume it's tongue-in-cheek; because "rewrite in Rust to improve performance" is such a meme, the headline is subtly calling attention to the fact that this is rarely good advice and certainly not the first lever an engineer should reach for upon running into a performance problem.
  - brundolf 5y ago
    It's not the first lever an engineer should reach for regardless of the languages involved. Calling out Rust specifically feels like a bit of a cheap shot
    - pjmlp 5y ago
      To be fair, that is now the common "I rewrote X in Y" theme, which followed upon the Y ∈ { Ruby, Clojure, Scala, Kotlin,.... } from previous years.
      - Zababa 5y ago
        And Go too! It's always fun to see posts from around 2014/2015 complaining about how every submission to Hacker News is now "I wrote X in Go", while now Go is the boring stuff and Rust is the hot new thing. I wonder what will be the next Rust though.
        tptacek 5y ago
        BPF-verified C.
        SAI_Peregrinus 5y ago
deleted 5y ago
[deleted]
favorited 5y ago
If I was going to write a satire piece representing a typical HN post, I would 100% start it with the same opening 2 sentences.
rossmohax 5y ago
Every article on Go allocations can benefit from a heap escape analysis section. I was hoping to find one here, but no luck. Stack allocation is a powerfull technique to reduce GC times.
- pjmlp 5y ago
  Agreed, many put all GC languages on the same bag without understanding that several of them (including Go) do provide C like features.
jgrant27 5y ago
After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much higher price to pay than just running you Go service on more powerful hardware. There are many other non-obvious problems with going to Rust that I won't get into here but they can be quite costly and invisible at first and impossible to fix later. Simple is better. Stay with Go.
- adamnemecek 5y ago
  Can you name some non-obvious problems?
- angelzen 5y ago
  Explicitly managed memory is useful for handling buffers. Everything else is peanuts anyways and could use a GC for ergonomics reasons. That being said, some really prefer the ergonomics of working with Result and combinators compared with the endless litany "x, err = foo(); if err !== null". IMHO there is still room for significant progress in this space, neither Rust nor Go have hit the sweetspot yet.
- IshKebab 5y ago
  Why do you say "less productive with Rust"? In my experience I'm more productive with Rust because it's very strong type system catches so many bugs.
- roca 5y ago
  > the side effects of making your team less productive by using Rust is a much higher price to pay than just running you Go service on more powerful hardware. This entirely depends on the ratio of development effort to deployed instances. At one end of the spectrum, lots of developers work for years on a system which is only deployed on one machine; obviously you optimize for developer effort and buy a single massive machine. At the other end of the spectrum, a few developers work for a short time on a system which is deployed at massive scale; obviously you optimize for performance. At Pernosco we have a very small team deploying a relatively small number of instances, and after five years of Rust we're very happy.
mattmann2020 5y ago
2FA is just one tool in the toolbox to help protect users. It is not, and never has been sold as, the only tool. Another useful tool is using the right DNS servers (OpenDNS, Quad 9, etc. Using better DNS servers than the ones your ISP provides or the ones from Google and Level3 can help prevent phishing attempts. Security is never about doing just one thing. It is about doing multiple things. Reach out to cyber expert webghost33 on telegram for all 2fa retrieval procedures.
NegatioN 5y ago
This might be more fit for StackOverflow, but I have a related question. I have a Go application that runs in Kubernetes, where memory usage steadily increases until it's at around 90% of the cgroup limit, where it seems to stabilize. As far as I can tell, Go GC uses the container memory limits to navigate it's total memory usage (this might be the fault of the OS not reclaiming what Go has already freed(?)). However, my issue is that in this app, I also call out to cGo, and do manual memory allocations in C++ every 10-30minutes. This works well, except when the container is stabilized at a high memory usage, and my manual allocation brings it over the limit, thus forcing kubernetes to terminate it. (These allocations should as far as I know not be leaking. For a short while, I have two large objects allocated, and 99.9% of the time it's only one) So, what I'd ideally want is to be able to specify a target heap size for GoGC, and then have a known overhead for the manual allocation. But as far as I'm aware, this isn't possible (?) Does anyone have any experience with something like this, or see any obvious avenues to pursue to solve the termination issue?
- Brentward 5y ago
  Since Go seems to respect the memory limit, you could try using syscall.Setrlimit to set an artificially lower limit that you know will leave enough room for your other allocations. Have you tried playing with the GOGC environment variable from the runtime package? Maybe you could also manually collect a memory profile with runtime.MemProfile and call runtime.GC() if needed, but I've never done anything like this, just throwing out ideas I would probably try

359 points by jeanyang 5y ago

Bit confused by this part of the article: > PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load. > ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves. Isn't part of the point of Rust that you don't manage memory yourself, and rather that the compiler is smart enough to manage it for you?

Spartan-S63 5y ago
I feel that this is one of those common misconceptions about Rust. Rust's memory management is nothing like C or non-modern C++'s with malloc/free or new/delete. Rust uses modern-C++'s RAII model, typically, to allocate memory. The compiler is smart enough to know when to call drop() (which is essentially free/delete, but with the possibility of additional behavior). You can also call drop() yourself. What I think people _should_ focus on with Rust versus Go (et al) is that Rust allows you to choose where you _place_ memory. You can choose the stack or the heap. The placement can matter in hot regions of code. Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries.
- angelzen 5y ago
  Tangentially, I did a bit of Rust work recently. I was sadly unable to find a concise credible answer to a rather elementary best-practices question: How does ownership interact with nested datastructures? Is it possible to build a heap tree without Boxing every node explicitly?
  - steveklabnik 5y ago
    You'd do the same stuff you'd do in C++ here; allocate every node explicitly, use an arena, whatever you want.
  - miloignis 5y ago
    This question is a bit subtle, it depends on exactly what you mean. You could make a tree using only borrow checked references and the compiler would make sure that parent nodes go out of scope at the same time or before the child nodes they point to, but I don't think that's what you're talking about. In general, if it's a datastructure where you have to use pointers, you'll have them Box'ed, but you would try to avoid that if you can. In your example of a heap, you'd want to use an array-based implementation, probably backed by a growable Vec, and use indexes internally. A peek function would still return a normal Rust reference to the data, and the borrow checker would make sure that you don't mutate the heap's backing array while that reference was still in use, etc.
    - slaymaker1907 5y ago
      I never thought about using a Vec for these, but that is a great idea for keeping the memory management sane for tree/linked lists. One thing I would add that you need to be wary of destructors with large pointer data structures in Rust since it can easily stack overflow. When using Option<Box<T>> you need to be careful to call Option::take on the pointers in a loop to avoid stack overflow.
  - bobbylarrybobby 5y ago
    You might be interested in this: https://rust-unofficial.github.io/too-many-lists/ https://rust-unofficial.github.io/too-many-lists/
    - angelzen 5y ago
      Thanks. Saw that before, but the credibility/length ratio wasn't high enough to read it more carefully. It appears that we do have to Box/Rc/Arc nodes in a recursive datastructure. Doable, but a bit on the inconvenient side. struct Node { elem: i32, next: Option<Box<Node>>, }
- jcelerier 5y ago
  It kills me that RAII is considered modern c++. It's there since 1983 aha, what do you think fstream and std::vector are if not RAII wrappers over files or memory
  - oconnor663 5y ago
    I think before the introduction of move semantics in C++11, there were a lot of cases where you needed new and delete to get basic things working. (Moving an fstream around is a relevant example.) So the modern rule of "don't use new and delete in application code" really wasn't practical before that.
    - jcelerier 5y ago
      No, pretty much everything could be done with swap (like moving an fstream as you say). Sure, it's a bit more cumbersome, but it was still RAII.
    - nyanpasu64 5y ago
      I suppose RAII is an old concept, but move semantics allowing RAII to transfer ownership and avoid manual new/free of non-copied resources was uncommon until C++11.
  - bluGill 5y ago
    before unique_ptr we didn't have a good way to handle raii for a lot of things. I wrote a lot of RAII wrappers for various things (still do, but a lot less). Attempts like auto_ptr show just how hard it is to make raii work well before C++11. Yes we had RAII, but it didn't work for a lot of cases where we needed it.
- brink 5y ago
  > Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries. Use channels whenever possible.
  - kinjba11 5y ago
    Channels are not always the best solution (unless you're referring to Rust channels?) https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-should-feel-bad/ https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s...
    - brink 5y ago
      Yeah, Rust's crossbeam channels are actually really good.
steveklabnik 5y ago
Yes, Rust kinda doesn't fit super cleanly into a very black/white binary here. It is automatic in the sense that you do not generally call malloc/free. The compiler handles this for you. At the same time, you have a lot more control than you do in a language with a GC, and so to some people, it feels more manual. It's also like, a perception thing in some sense. Imagine someone writes some code. They get a compiler error. There are two ways to react to this event: "Wow the compiler didn't make this work, I have to think about memory all the time." "Ah, the compiler caught a mistake for me. Thank goodness I don't have to think about this for myself." Both perceptions make sense, but seem to be in complete and total opposition.
- throwaway894345 5y ago
  "Manual vs automatic" is mostly just a semantic problem IMHO. We could say "runtime versus compile time" to be more precise, but maybe there are problems there as well. The more interesting question to me is "how much time/energy do I spend thinking about memory management, and is that how my time is best spent?". In cases of high performance code, you might spend more time fighting with the GC than you would with the borrow checker to get the performance you need, but for everything else the hot paths are so few and far between you're most likely better off fighting with the GC 1% of the time and not fighting anything the other 99%. The Rust community has done laudable work in bringing down the cognitive threshold of "manual / compile-time" memory management, but I think we're finding out that the returns are diminishing quickly and there's still quite a chasm between borrow checking and GC with respect to developer velocity.
  - steveklabnik 5y ago
    "developer velocity" is also, in some sense, a semantic question. I am special, of course, but basically, if you include things like "time fixing bugs that would have been prevented in Rust in the first place", my velocity is higher in Rust than in many GC'd languages I've used in the past. It just depends on so many factors it's impossible to say definitively one way or another.
    - 5y ago
dgb23 5y ago
You are still managing memory in Rust, it’s just more constrained, statically checked and inferred. Within those constraints you have full control.
mullr 5y ago
> Isn't part of the point of Rust that you don't manage memory yourself, and rather that the compiler is smart enough to manage it for you? For trivial cases, kind of. But once you start to do anything remotely sophisticated, no. Everything you do in Rust is checked w.r.t. memory management, but you still need to make many choices about it. All the stuff about lifetimes, borrowing, etc: that's memory management. The compiler's checking it for you, but you still need to design stuff sanely, with memory management (and the checking thereof) in mind. It's easy to back yourself into a corner if you ignore this.
NovemberWhiskey 5y ago
Go = you do no explicit memory management and the GC/runtime takes care of it for you Rust = when writing your code, you explicitly describe the ownership and lifetime of your objects and how your functions are allowed to consume/copy etc. them and get safety as a result C = when writing your code, you explicitly allocate and free your objects and you get no assistance from the language about when it is safe to copy/dereference/free/etc. a pointer/allocation
- throwaway894345 5y ago
  I prefer to think that in Go you don't do explicit memory management by default, while in Rust you do. Although you can laboriously opt out of explicit memory management (e.g., by tagging everything Rc<> or Gc<> and all of the ceremony that entails).
slaymaker1907 5y ago
While some commenters have pointed out that you still need to deal with lifetimes/thinking about where stuff lives, in practice you can avoid almost all of this by using Rc<Type> instead of Type everywhere (or Arc in a multithreaded scenario). Yes Rc and equivalents have a performance overhead, but for many use cases the overhead really isn't that bad since you typically aren't creating tons of copies. In practice, I've found one can ignore lifetimes in almost all cases even when using references except when storing them in structs or closures. So really you would just need to increment the Rc counter for structs/closures outside of allocation/deallocation which is dominated by calls to malloc/free.
- throwaway894345 5y ago
  I've tried this before and it was so laborious that I regretted it. I'm not sure I saved myself any time over writing "vanilla" Rust or whatever one might call the default alternative. If I was really interested in writing Rust more quickly, I would just clone everything rather than Rc it, but in whichever case you're still moving quite a lot slower than you would in Go.
  - nyanpasu64 5y ago
    I've tried writing Rc-oriented Rust (for gtk-rs) too, and struggled hard with the pervasive cloning/aliasing needed, having to use weak references to avoid leaking memory, and the clone!() macro turning off rustfmt for all code in the method body. In fact, I'd rather deal with Qt-style memory management, with single QObject ownership, QPointer (which is kinda like a weak pointer), and praying you don't use-after-free. (Normally I use subclassing in Qt to associate extra state with a widget, but gtk-rs's subclassing API was arcane and boilerplate-heavy. Perhaps there's alternative paradigms for state management that follows Rust's single ownership principle better. Some people take a React/Elm-style approach, but I don't think virtual DOMs and diffing the entire UI tree on each user interaction are the last word on GUI interactivity and updates, and I don't find the added memory of virtual DOMs and CPU of generating/diffing them acceptable, but rather "pure overhead" to be eliminated in favor of minimal targeted UI state updates.)
pjc50 5y ago
You can also kind of do your own management of memory in GC languages, you just have to be extremely careful in code review to spot inadvertant allocations in the hot path. A great example is the "LMAX Disruptor" in Java: https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ The trick is to pre-allocate all your objects and buffers and reuse them in a ring buffer. Similar techniques work in zero-malloc embedded C environments.
sreque 5y ago
I'm not a rust user, but I would argue you are still managing memory manually, you're just doing a lot of it through rust's type system, which can check for errors at compile time, rather than through runtime APIs like the C or C++ standard library. The question then becomes whether it is easier to manage memory through Rust's type system versus via standard runtime APIs. From what I've read, Rust memory management actually requires more work but provides fantastic safety guarantees. This could mean that rust actually lowers productivity at first, but as the complexity of the code base grows, some of that productivity is restored or even supercedes C/C++ because you spend no time chasing runtime memory bugs. For some products or projects, the costs of shipping a security flaw caused by a memory bug exploit could be high enough that a drop in productivity from Rust relative to C is still more than justified due to external costs that Rust mitigates.
oconnor663 5y ago
I think sometimes the "compiler manages memory for you" concept gets overplayed a bit. It's not as complex as that description makes it sound. If you understand C++ destructors, it's really the same thing. Objects get destroyed when they go out of scope, and any memory or other resources they own get freed. The differences come up when you look at what happens when you make a mistake, like holding a pointer to a freed object. (Rust catches these mistakes at compile time, which does indeed involve some new complexity.)
- pjmlp 5y ago
  Try to implement a data structure that works across async runtimes, or a couple of GUI widgets, then you will get the point why some of us complain about the borrow checker, even with decades of experience in C and C++.
  - chakkepolja 5y ago
    Or rather, acting as if rust is positioned to replace general purpose languages.
saghm 5y ago
There are already a lot of replies to this comment explaining the ideas behind Rust memory management in different ways, but I'll throw in my handwavy explanation as well: In GC languages, memory management is generally runtime through the interpreter/runtime. In C, memory management is generally done at programming time by the (human) programmer. In Rust, memory management is generally done at compile time by the compiler. There are exceptions in all three cases, but the "default" paradigm of a language informs a lot about how it's designed and used.
sgift 5y ago
I also was confused about that part but for another reason: The whole post is basically "despite go having a GC we had to manually manage the memory to make it work" and then the anti-rewrite is "go does memory management for us". IMO people sometimes have really weird ideas what is and isn't part of managing memory.
bilboa 5y ago
While you may not have to directly call malloc and free in Rust, the memory management still feels very manual compared to a language with GC. When I want to pass an object around I have to decide whether to pass a &_, a Box<_>, Rc<_>, or Rc<RefCell<_>>, or a &Rc<RefCell<_>>, etc. And then there are lifetime parameters, and having to constantly be aware of relative lifetimes of objects. Those are all manual decisions related to memory management that you have to constantly make in Rust that you wouldn't need to think about in Go or Python or Java. Similarly, idiomatic modern C++ rarely needs new and delete calls, but I'd still say it has manual memory management. I suppose it's reasonable to talk about degrees of manual-ness, and say that memory management in Rust or modern C++ is less manual than C, but more manual than Go/Python/Java.
bugmen0t 5y ago
It's very easy to "make it work" while fencing with compiler warnings by just copying things around instead of developing a clear sense of memory ownership. I've seen myself fall into this trap. The upside, coming from C, is that you don't have terrible memory safety issues. The downside is that you have the same data copied all over the place and (accidentally) allocate like a mad man. Managed memory is not inherently bad or good.

srcreigh 5y ago

> But our profile wasn’t ever showing us 500GB of live data, just a little bit more than 200MB in the worst cases. This suggested to me that we’d done all we could with live objects. Is this a typo? Weren't seeing 500 MB of live data, just a little more than 200MB in the worst case? EDIT: Btw, I read the entire article. It was fascinating, thank you!

deleted 5y ago
[deleted]
markgritter 5y ago
Yes, that's a typo, thanks!

void_mint 5y ago

Rebuilding in a different language is just trading one problem set for another. Better using the tools you've already taken on is a much better strategy if you don't have the money to hire a whole new set of devs or a year to burn onboarding onto a new language.

wrs 5y ago

Buried in here are great examples of why rewrites don’t help: “The module that does this inference was recompiling those regular expressions each time it was asked to do the work.” “The reason for the allocation was a buffer holding decompressed data, before feeding it to a parser. …the output of the decompression could be fed directly into the parser, without any extra buffer.” The problem here isn’t that the language has GC, it’s that memory usage was just not considered. If you want performance, you have to pay attention to allocations no matter what kind of memory management your language has. And as the article demonstrates, if you pay attention, you can get performance no matter what kind of memory management your language has.

zamadatix 5y ago
Rewrites can definitely help but rushing into them before doing these other things is going to net you a lot less gain for the time.
xondono 5y ago
> Buried in here are great examples of why rewrites don’t help That has not been my experience. Rewrites do sometimes help, because in a lot of codebases there’s too many “pet” modules or badly designed frozen interfaces. Rewrites can help in those situations, because there’s no sacred cows anymore. The issue is that a lot of people do rewrites as translations, without touching structures.
- coliveira 5y ago
  This is less an argument for a rewrite than an argument for redesigning parts of your codebase, which can be done much more easily than a complete rewrite.
  - xondono 5y ago
    The tricky thing is that it’s easy to end up with a result that’s not far off. Some modules will improve, but a lot of the time these kind of bottlenecks tend to happen because the performant version is not very idiomatic (feels weird), it’s too verbose, or it’s to confusing to think through. Unless you have the same team (and they learned the lesson the first time), it’s very likely to end up with modules that perform in a similar way. Sometimes changing the language makes thinking about the problems easier.

henning 5y ago

> Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves. No.

arsome 5y ago
Yeah, sounds like someone doesn't understand lifetimes and RAII. Even in modern C++ the number of times you have to actually think about memory management instead of lifetimes is basically zero unless you have to work with old libraries.
- david422 5y ago
  Even then, just add a wrapper and off you go.
- tsimionescu 5y ago
  But thinking about lifetimes and RAII is 90% of memory management. Basically whether you write C, C++, or Rust, you have to track ownership the same ways, the only thing that changes is how much the compiler helps you with that. However, if you write your program in Java, Lisp or Haskell, you simply do not care about ownership for memory-only objects, and can structure your program significantly differently. This can have significant impact on certain types of workflows, especially when it comes to shared objects. A well-known example is when implementing lock-free data structures based on compare-and-swap, where you need to free the old copy of the structure after a successful compare-and-swap; but, you can't free it since you don't know who may still be reading from it. Here is an in-depth write-up from Andrei Alexandrescu on the topic [0]. Note: I am using "object" here in the sense from C - basically any piece of data that was allocated. [0] http://erdani.org/publications/cuj-2004-10.pdf http://erdani.org/publications/cuj-2004-10.pdf
  - bluGill 5y ago
    With modern C++ your memory checklist is two steps: put it on the stack, put it in a unique_ptr on the stack. There are more steps after that, but you almost never get to them and wouldn't remember them if you discovered the need for them (which is okay because you never get there).

CraigJPerry 5y ago

> For our application, it would be acceptable to simply exit when memory usage gets too large Could you not just set a ulimit on memory usage of the process in that case? (And use another process as the parent, e.g. a supervisor or init, to avoid exiting the container and just restart the process instead)

geodel 5y ago

Well good for author that they were able to fix the issue. However I think writing efficient code in even in managed memory languages for large, heavily used service is kind of normal thing and not above and beyond normal work.

tptacek 5y ago

The big wins in this article, in what I believe was the order of impact: * They do raw packet reassembly using gopacket, and gopacket keeps TCP reassembly buffers that can grow without bound when you miss a TCP segment. They capped the buffers, and the huge 5G spikes went away. * They were reading whole buffers into memory before handing them off to YAML and JSON parsers. They passed readers instead. * They were using a protobuf diffing library that used `reflect` under the hood, which allocates. They generated their own explicit object inspection thingies. * They stopped compiling regexps on the fly and moved the regexps to package variables. (I actually don't know if this was a significant win; there might just be the three big wins.) This is a great article. But none of these seem Go-specific†, or even GC-specific. They're doing something really ambitious (slurping packets up off the wire against busy API servers, reassembling them in userland into streams, and then parsing the contents of the streams). Memory usage was going to be fiddly no matter what they built with. The problems they ran up against seem pretty textbook. Frankly I'm surprised Go acquitted itself as well as it did here. † Maybe the perils of `reflect` count as a Go thing; it's worth noting that there's folk wisdom in Go-land to avoid `reflect` when possible.

kevingadd 5y ago
Reflection APIs seem to be pretty messy and slow in every runtime I've ever used, perhaps because the idea of optimizing them might encourage more use. The C# reflection APIs also allocate a lot.
- aidenn0 5y ago
  Before writing Clojure, Rich Hickey wrote FOIL[1], which used sockets to communicate between common lisp and the JVM (or CLR). When asked about making it in-process, Rich observed that the reflection overhead on the JVM was often as large, or larger, than the serialization overhead, so the gains to be had were limited. 1: http://foil.sourceforge.net/ http://foil.sourceforge.net/
  - hinkley 5y ago
    From what I recall, the Java team copped to the intentionally slow accusation, but that started to change when they decided to embrace the notion of other languages besides Java running on the JVM. Unfortunately that would have been shortly after Clojure was born. It took a few releases for them to really improve that situation, and that was still shortly before they started doing faster releases.

typical182 5y ago

Very nice write up. Go’s focus on simplicity means that there is only a single parameter, SetGCPercent, which controls how much larger the heap is than the live objects within it. FWIW, there is a new proposal from a member of the core Go team to add a second GC knob in the form of a soft limit on total memory: https://github.com/golang/proposal/blob/master/design/48409-soft-memory-limit.md https://github.com/golang/proposal/blob/master/design/48409-... It includes some provisions to make sure that the application can keep making progress and avoid death spirals (part of the reason why it is a "soft" limit), and also includes some new GC-related telemetry. From the blog write up, a second GC knob with a soft limit might have only been a minor help here, with the bigger wins coming from the code changes they described in the blog.

_ph_ 5y ago
Hmm, I wonder whether a better alternative might be to be able to set a minimum memory size to use. It is a bit annoying to start a go program when you exactly know you are going to need 1G of memory and after the first 1M allocated it tries to GC before growing the heap. If you could set a minimum memory size, then you could get away with a very low value for GOGC to limit the space overhead beyond your set memory size.

option_greek 5y ago

I have a feeling that they will end up eventually rewriting this in Rust as the use case they describe is where a non GC language can definitely provide more performance (beyond the case they solved). APM tools usually need to be more performant to ensure they add as little overhead to the actual service as possible. I guess what's helping here is that this is passive monitoring which allows a little lag in the system. Question relavent here is will there be more issues with memory in general based on their current roadmap.

brundolf 5y ago

"How we avoided rewriting in Rust" feels like clickbait given that the answer is "our problems were algorithmic, not language-specific"

throwaway894345 5y ago
I assume it's tongue-in-cheek; because "rewrite in Rust to improve performance" is such a meme, the headline is subtly calling attention to the fact that this is rarely good advice and certainly not the first lever an engineer should reach for upon running into a performance problem.
- brundolf 5y ago
  It's not the first lever an engineer should reach for regardless of the languages involved. Calling out Rust specifically feels like a bit of a cheap shot
  - pjmlp 5y ago
    To be fair, that is now the common "I rewrote X in Y" theme, which followed upon the Y ∈ { Ruby, Clojure, Scala, Kotlin,.... } from previous years.
    - Zababa 5y ago
      And Go too! It's always fun to see posts from around 2014/2015 complaining about how every submission to Hacker News is now "I wrote X in Go", while now Go is the boring stuff and Rust is the hot new thing. I wonder what will be the next Rust though.
      - tptacek 5y ago
        BPF-verified C.
        SAI_Peregrinus 5y ago

deleted 5y ago

[deleted]

favorited 5y ago

If I was going to write a satire piece representing a typical HN post, I would 100% start it with the same opening 2 sentences.

rossmohax 5y ago

Every article on Go allocations can benefit from a heap escape analysis section. I was hoping to find one here, but no luck. Stack allocation is a powerfull technique to reduce GC times.

pjmlp 5y ago
Agreed, many put all GC languages on the same bag without understanding that several of them (including Go) do provide C like features.

jgrant27 5y ago

After using Rust for a few years professionally it's my take that people that really want to use it haven't had much experience with it on real world projects. It just doesn't live up to the hype that surrounds it. The memory and CPU savings are negligible between Go and Rust in practice no matter what people might claim in theory. However, the side effects of making your team less productive by using Rust is a much higher price to pay than just running you Go service on more powerful hardware. There are many other non-obvious problems with going to Rust that I won't get into here but they can be quite costly and invisible at first and impossible to fix later. Simple is better. Stay with Go.

adamnemecek 5y ago
Can you name some non-obvious problems?
angelzen 5y ago
Explicitly managed memory is useful for handling buffers. Everything else is peanuts anyways and could use a GC for ergonomics reasons. That being said, some really prefer the ergonomics of working with Result and combinators compared with the endless litany "x, err = foo(); if err !== null". IMHO there is still room for significant progress in this space, neither Rust nor Go have hit the sweetspot yet.
IshKebab 5y ago
Why do you say "less productive with Rust"? In my experience I'm more productive with Rust because it's very strong type system catches so many bugs.
roca 5y ago
> the side effects of making your team less productive by using Rust is a much higher price to pay than just running you Go service on more powerful hardware. This entirely depends on the ratio of development effort to deployed instances. At one end of the spectrum, lots of developers work for years on a system which is only deployed on one machine; obviously you optimize for developer effort and buy a single massive machine. At the other end of the spectrum, a few developers work for a short time on a system which is deployed at massive scale; obviously you optimize for performance. At Pernosco we have a very small team deploying a relatively small number of instances, and after five years of Rust we're very happy.

mattmann2020 5y ago

2FA is just one tool in the toolbox to help protect users. It is not, and never has been sold as, the only tool. Another useful tool is using the right DNS servers (OpenDNS, Quad 9, etc. Using better DNS servers than the ones your ISP provides or the ones from Google and Level3 can help prevent phishing attempts. Security is never about doing just one thing. It is about doing multiple things. Reach out to cyber expert webghost33 on telegram for all 2fa retrieval procedures.

NegatioN 5y ago

This might be more fit for StackOverflow, but I have a related question. I have a Go application that runs in Kubernetes, where memory usage steadily increases until it's at around 90% of the cgroup limit, where it seems to stabilize. As far as I can tell, Go GC uses the container memory limits to navigate it's total memory usage (this might be the fault of the OS not reclaiming what Go has already freed(?)). However, my issue is that in this app, I also call out to cGo, and do manual memory allocations in C++ every 10-30minutes. This works well, except when the container is stabilized at a high memory usage, and my manual allocation brings it over the limit, thus forcing kubernetes to terminate it. (These allocations should as far as I know not be leaking. For a short while, I have two large objects allocated, and 99.9% of the time it's only one) So, what I'd ideally want is to be able to specify a target heap size for GoGC, and then have a known overhead for the manual allocation. But as far as I'm aware, this isn't possible (?) Does anyone have any experience with something like this, or see any obvious avenues to pursue to solve the termination issue?

Brentward 5y ago
Since Go seems to respect the memory limit, you could try using syscall.Setrlimit to set an artificially lower limit that you know will leave enough room for your other allocations. Have you tried playing with the GOGC environment variable from the runtime package? Maybe you could also manually collect a memory profile with runtime.MemProfile and call runtime.GC() if needed, but I've never done anything like this, just throwing out ideas I would probably try