9 ms·

I thought the segmented stack was the key to having hundreds of thousands of userland threads. It seems pretty farfetched (though not impossible) that realloc'

• krenoten 13y ago
  Further down the thread there are several alternatives to segmented stacks brought up.
• qznc 13y ago
  I think Go is primarily designed for servers and servers are overwhelmingly 64bit these days. In a 64bit virtual address space, you can have millions of contiguous stacks each of gigabytes size.
  • vinkelhake 13y ago
    That's true, but the majority of those 64-bit servers only have a 48-bit address space to millions of gigabyte-sized stacks would still be a tight fit.
    • alexlarsson 13y ago
      Obviously it would be a "virtual" gigabyte-size stack. I.e. it would be mmapped, but all the threads would not actually be using all their stack. Physical memory is not allocated, but virtual addresses are.
      • dbaupp 13y ago
        One can only fit 2^18 == 262144 stacks with size 1GiB into the 48-bit x86-64 address space (i.e. 48 bits is the total virtual address size).
        • zokier 13y ago
          Isn't 1GiB stack quite huge? I would have excepted them to be like order of magnitude smaller.
          • noselasd 13y ago
            Yes, it's huge. C programs on linux normally defaults to a 8 or 10 MB stack per thread, and that is considered huge by many.
  • tomp 13y ago
    Not really. Assuming that page size is 4KB, and given that for most systems, the minimal amount of memory that can be committed is a page, you will exhaust all 4GB of your server's physical RAM with only 1 100 000 threads.
    • eliben 13y ago
      I don't think many real servers have 4 GB these days. The dev box under my desk has 32. Linux servers often have 1 TB of physical memory. So based on your calculation that's 256 million threads. Sounds enough :)
      • ithkuil 13y ago
        The whole point of a cheap concurrency like goroutines is the ability to avoid using callback based async IO (i.e. the go standard libraries perform select/epoll/... under the hood but your code behaves like if it was blocking on IO). You might have better use of your memory (caching, in memory databases) than wasting it for even a fraction of a million of routines that are just waiting for something to happen.
      • tomp 13y ago
        Yes, but if I buy a server with e.g. 128 GB of RAM, I would preferably spend it to cache 100GB of live data and searchable indices, and not worry when I will spawn too many threads so that I will start trashing the swap.
• kelnos 13y ago
  Not entirely, as I understand it. Segmented stacks is just one solution. Another is to start with a very small stack, and grow it contiguously (copying to a new location if necessary) when an overflow is imminent. You still get the benefit of having small stacks, which lets you make those 100ks of userland threads, and as long as they don't blow up their stack usage (same requirement as for segmented stacks), they'll be fine.
  • Roboprog 13y ago
    I played with the idea of something like this a while back, but never finished it: https://github.com/roboprog/buzzard/blob/master/test/src/main.c https://github.com/roboprog/buzzard/blob/master/test/src/mai... It performed reasonably well in a demo doing a bunch of string concatenation, but I never followed up on it. https://github.com/roboprog/mp_bench/blob/master/thd_frk.bzrt.c https://github.com/roboprog/mp_bench/blob/master/thd_frk.bzr...