6 ms·

It's not uncommon for up to 1/3 of usage and therefore the bill on VMs in the cloud to be consumed by garbage collection. So if you can rewrite it without a ga

It's not uncommon for up to 1/3 of usage and therefore the bill on VMs in the cloud to be consumed by garbage collection. So if you can rewrite it without a garbage collector, you can save money. A great book on this topic is "The Beast Is Back" by jetbrains. Advocating C++ in that case (written in 2015). If GC makes you more productive, that's good, but at some point rewriting things without GC makes sense.

icebraining 7y ago
How much is the usage consumed by malloc/free or equivalent?
- ncmncm 7y ago
  malloc/free are not used in modern C++. The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.
  - steveklabnik 7y ago
    They're not directly invoked, but that's still what's called under the hood.
    - deleted 7y ago
      [deleted]
    - ncmncm 7y ago
      In Rust as in C++. But the fact that the actual calls don't appear in your source code means you don't incur any programmer cost relying on them. Yet, where runtime cost would be a problem (typically, affecting latency on a hot path) it can be avoided entirely.
  - lossolo 7y ago
    > malloc/free are not used in modern C++. I think using term "are abstracted away" is better choice here. You still allocate memory, dosen't matter if it's malloc/free, mmap/unmap or compiler is doing it for you. It cost time and space. Sometimes the cost is negligible but not always, depends on application.
  - jchw 7y ago
    > malloc/free are not used in modern C++. I don’t think that was their point. > The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero. This is at least slightly misleading, though. Obviously, memory comes from somewhere. You can amortize costs by not needing to allocate new pages often. Minimizing the amount of memory consumed by an application dynamically is the only way to absolutely reduce the cost. There’s lots of ways to do this, and plenty of C++ and Go software aim for “zero allocations.” However, there is still actually allocations in many softwares with “zero allocations” because they still use the stack. For deeply concurrent applications, the stack ends up being a lot of memory. If you reduce the amount of stack memory per fiber, it reduces memory usage initially, but then fibers are more likely to hit the guard page and allocate more stack. There’s strategies to reduce dynamic allocations pretty much all over (even in GC’d languages like Go.) The fact is, though, avoiding it is much akin to avoiding the GC. In Go, its actually identical to avoiding the GC. (As a note in post, I acknowledge that not every concurrent application uses fiber style concurrency, but I believe with minor adaptations this point still stands for many classes of applications. Fully avoiding OS allocations is possible, but it definitely isn’t the “default” for C++ apps.) —— This isn’t to say your point is not correct at least for some viewpoint, but it’s not actually that simple, which is absolutely worth noting.
  - vardump 7y ago
    > The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero. In that case you don't use std::string, std::vector... or any containers at all? Or anything else that allocates, directly or indirectly.
stcredzero 7y ago
So if you can rewrite it without a garbage collector, you can save money. Save money over what? It's usually a lot easier to optimize memory use than to rewrite code. Going by the 80/20 rule, 80% of the memory pressure on the GC will be created by 20% of the code. So amortize the price of the rewrite over the expected lifespan of the app. Then compare this to, let's say, the cost of optimizing 10% of the app to eliminate 40% of the memory pressure. Then, also factor in the likely rate of bugs introduced in a rewrite compared to the optimizing GC and the cost of incurring, finding, and fixing those bugs. Going by this analysis, I would suspect that in some environments where rapid iteration is key, progress is fast, and apps have short lifespans, it might be better to optimize GC instead of rewrite to eliminate it. I'd also expect that in other cases, it is better to rewrite to eliminate GC.
- ncmncm 7y ago
  This analysis assumes that not using GC costs something. However, in modern C++ code, as in Rust code, you can root around as long as you like and not find any code outside low-level, standard libraries that does any memory management. Avoiding GC costs exactly nothing in progress or in iteration time. So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.
  - stcredzero 7y ago
    Avoiding GC costs exactly nothing in progress or in iteration time. This might be true in new development. The specific context in this discussion was rewrites. So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X. Again, you're talking about new development. That's not going to fit everyone's situation.
    - ncmncm 7y ago
      OK, I get you. The problem is that whatever level of GC overhead you start with, or achieve, it will be non-zero, and its actual magnitude, including typically big cache-footprint knock-on effects that show up attributed, in perf results, to mainline processing, will be practically impossible to estimate reliably without comparing against a rewrite. So, instead, you generally have to say: we compared some similar(-ish) program Y that was rewritten and cut the number of server instances required to meet demand by 30%, 60%, or what-have-you. But, exactly for the reasons you cite, comparisons published are against performance under GC after that optimization has already been done, as much as was practical.
weberc2 7y ago
C++ still needs to allocate and free memory too and for all we know naive C++ might spend more time in memory management code than a GC language.
- ncmncm 7y ago
  There is a fair bit of C++ code in use. We have no need to guess. And, the answer is that real programs in obligate-GC languages spend overwhelmingly more time in GC than C++ or Rust. Much of this time is spent waiting on cache misses, which are hard to track to the responsible bit of code.
  - weberc2 7y ago
    Do you have evidence for this? Specifically C++ and Rust tend to be written for applications where tight control over memory is necessary and so any sample like you're describing is going to be biased by these carefully tuned C++/Rust applications. Even the standard libraries for C++ and Rust differ considerably in allocation behavior from Java or Python--these languages are conventionally designed to allocate differently, but that doesn't mean that the GC is the problem. Further, different programming paradigms allocate memory differently and the distribution of these paradigms across GC languages and non-GC languages (or whatever terms you like) are almost certainly varied. There are lots of confounding variables to control for, and until you control for them you're pretty much just guessing.
    - nicoburns 7y ago
      I think it's pretty obvious if you use these languages. A lot of things that require you to heap allocate in Java or Python (like classes!) have stack-allocated versions in rust/c++. Which means that code which avoids slow heap allocation is much nicer than equiavlent code in Java that must avoid high level abstractions. You're right that the GC isn't necessarily the issue. It's more the forced heap allocation which most GC languages come with.
      - weberc2 7y ago

icebraining 7y ago

How much is the usage consumed by malloc/free or equivalent?

ncmncm 7y ago
malloc/free are not used in modern C++. The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.
- steveklabnik 7y ago
  They're not directly invoked, but that's still what's called under the hood.
  - deleted 7y ago
    [deleted]
  - ncmncm 7y ago
    In Rust as in C++. But the fact that the actual calls don't appear in your source code means you don't incur any programmer cost relying on them. Yet, where runtime cost would be a problem (typically, affecting latency on a hot path) it can be avoided entirely.
- lossolo 7y ago
  > malloc/free are not used in modern C++. I think using term "are abstracted away" is better choice here. You still allocate memory, dosen't matter if it's malloc/free, mmap/unmap or compiler is doing it for you. It cost time and space. Sometimes the cost is negligible but not always, depends on application.
- jchw 7y ago
  > malloc/free are not used in modern C++. I don’t think that was their point. > The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero. This is at least slightly misleading, though. Obviously, memory comes from somewhere. You can amortize costs by not needing to allocate new pages often. Minimizing the amount of memory consumed by an application dynamically is the only way to absolutely reduce the cost. There’s lots of ways to do this, and plenty of C++ and Go software aim for “zero allocations.” However, there is still actually allocations in many softwares with “zero allocations” because they still use the stack. For deeply concurrent applications, the stack ends up being a lot of memory. If you reduce the amount of stack memory per fiber, it reduces memory usage initially, but then fibers are more likely to hit the guard page and allocate more stack. There’s strategies to reduce dynamic allocations pretty much all over (even in GC’d languages like Go.) The fact is, though, avoiding it is much akin to avoiding the GC. In Go, its actually identical to avoiding the GC. (As a note in post, I acknowledge that not every concurrent application uses fiber style concurrency, but I believe with minor adaptations this point still stands for many classes of applications. Fully avoiding OS allocations is possible, but it definitely isn’t the “default” for C++ apps.) —— This isn’t to say your point is not correct at least for some viewpoint, but it’s not actually that simple, which is absolutely worth noting.
- vardump 7y ago
  > The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero. In that case you don't use std::string, std::vector... or any containers at all? Or anything else that allocates, directly or indirectly.

stcredzero 7y ago

So if you can rewrite it without a garbage collector, you can save money. Save money over what? It's usually a lot easier to optimize memory use than to rewrite code. Going by the 80/20 rule, 80% of the memory pressure on the GC will be created by 20% of the code. So amortize the price of the rewrite over the expected lifespan of the app. Then compare this to, let's say, the cost of optimizing 10% of the app to eliminate 40% of the memory pressure. Then, also factor in the likely rate of bugs introduced in a rewrite compared to the optimizing GC and the cost of incurring, finding, and fixing those bugs. Going by this analysis, I would suspect that in some environments where rapid iteration is key, progress is fast, and apps have short lifespans, it might be better to optimize GC instead of rewrite to eliminate it. I'd also expect that in other cases, it is better to rewrite to eliminate GC.

ncmncm 7y ago
This analysis assumes that not using GC costs something. However, in modern C++ code, as in Rust code, you can root around as long as you like and not find any code outside low-level, standard libraries that does any memory management. Avoiding GC costs exactly nothing in progress or in iteration time. So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.
- stcredzero 7y ago
  Avoiding GC costs exactly nothing in progress or in iteration time. This might be true in new development. The specific context in this discussion was rewrites. So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X. Again, you're talking about new development. That's not going to fit everyone's situation.
  - ncmncm 7y ago
    OK, I get you. The problem is that whatever level of GC overhead you start with, or achieve, it will be non-zero, and its actual magnitude, including typically big cache-footprint knock-on effects that show up attributed, in perf results, to mainline processing, will be practically impossible to estimate reliably without comparing against a rewrite. So, instead, you generally have to say: we compared some similar(-ish) program Y that was rewritten and cut the number of server instances required to meet demand by 30%, 60%, or what-have-you. But, exactly for the reasons you cite, comparisons published are against performance under GC after that optimization has already been done, as much as was practical.

weberc2 7y ago

C++ still needs to allocate and free memory too and for all we know naive C++ might spend more time in memory management code than a GC language.

ncmncm 7y ago
There is a fair bit of C++ code in use. We have no need to guess. And, the answer is that real programs in obligate-GC languages spend overwhelmingly more time in GC than C++ or Rust. Much of this time is spent waiting on cache misses, which are hard to track to the responsible bit of code.
- weberc2 7y ago
  Do you have evidence for this? Specifically C++ and Rust tend to be written for applications where tight control over memory is necessary and so any sample like you're describing is going to be biased by these carefully tuned C++/Rust applications. Even the standard libraries for C++ and Rust differ considerably in allocation behavior from Java or Python--these languages are conventionally designed to allocate differently, but that doesn't mean that the GC is the problem. Further, different programming paradigms allocate memory differently and the distribution of these paradigms across GC languages and non-GC languages (or whatever terms you like) are almost certainly varied. There are lots of confounding variables to control for, and until you control for them you're pretty much just guessing.
  - nicoburns 7y ago
    I think it's pretty obvious if you use these languages. A lot of things that require you to heap allocate in Java or Python (like classes!) have stack-allocated versions in rust/c++. Which means that code which avoids slow heap allocation is much nicer than equiavlent code in Java that must avoid high level abstractions. You're right that the GC isn't necessarily the issue. It's more the forced heap allocation which most GC languages come with.
    - weberc2 7y ago