5 ms·

Old: 2.4 GHz Pentium 4, 4 GB RAM, 32-bit FreeBSD 5.3. New: 3.0 GHz Core whatever, 12 GB RAM, 64-bit FreeBSD 7.1.

by rtm 18y ago

cperciva 18y ago
You were running FreeBSD 5.3 until recently? You know it reached its EoL at the end of October 2006, right? There have been lots of security issues over the past 2.5 years which weren't fixed in FreeBSD 5.3. And by "FreeBSD 7.1" you mean 7.1-RELEASE-p3, right? :-)
vaksel 18y ago
thanks
lsb 18y ago
So, if you've got all those in-memory data structures, there are lots of lists of pointers to objects. How does it compare on a 32-bit OS, with 32-bit pointers, vs a 64-bit with 64-bit pointers?
- cperciva 18y ago
  I'd guess that most of the memory is used by blocks of text which are much larger than the pointers which reference them, so probably the size_t expansion has a minimal impact on the total memory usage.
  - jwilliams 18y ago
    That's true, but 64 bit will affect other things as well - pointer size goes up, your stack will be bigger and the packing (alignment) of your structures changes. If your compiler is unfriendly, your structure size can change significantly -- afaik, GCC is pretty clever about structure packing, but I'd imagine it varies a lot depending on the architecture.
- jwilliams 18y ago
  You mean compare in terms of memory usage? It will certainly eat more memory for the same data structures. We've got a simple functional language here - being very unscientific, and looking at the key data structures, for my current workload, straight pointers are about 5-10% of the allocation. In practice I do see a 10-25% memory usage increase going from 32bit to 64bit (Linux). (Anecdotally -- I've read 20% is a rough rule of thumb for 64bit anyway). There is also an increase in data structure size due to data alignment, but this is going to vary quite a bit -- depends on the structure, the compiler -- and if you're doing dynamic memory allocation you may even find it makes no difference at all -- as malloc may have been delivering an oversize allocation anyway. So, it's certainly not a trivial increase... Especially as we're currently running on a 256mb slice (I sort of wish they had a supported 32bit option)... but it's not massive. However, for us, 64 bit still has a lot of advantages. For example, you can do more expansive memory mapping and the like. Not sure if that helps.
- notaddicted 18y ago
  Obviously the pointers take twice as much space but I don't think that is what you are asking. arc runs in MzScheme 352 (http://download.plt-scheme.org/mzscheme/v352.html http://download.plt-scheme.org/mzscheme/v352.html) the freeBSD binaries on this page are i386, if thats what the new server is using then it won't make a difference. The source is available as well, if pg compiled the source in 64 bit mode it would increase memory usage. But using a 64 bit OS doesn't necessitate using a 64 bit address space.
- deleted 18y ago
  [deleted]
- rtm 18y ago
  64-bit mzscheme uses about 50% more memory than 32-bit mzscheme to hold all of the Hacker News data.

Old: 2.4 GHz Pentium 4, 4 GB RAM, 32-bit FreeBSD 5.3. New: 3.0 GHz Core whatever, 12 GB RAM, 64-bit FreeBSD 7.1.

by rtm 18y ago

Old: 2.4 GHz Pentium 4, 4 GB RAM, 32-bit FreeBSD 5.3. New: 3.0 GHz Core whatever, 12 GB RAM, 64-bit FreeBSD 7.1.

You were running FreeBSD 5.3 until recently? You know it reached its EoL at the end of October 2006, right? There have been lots of security issues over the past 2.5 years which weren't fixed in FreeBSD 5.3. And by "FreeBSD 7.1" you mean 7.1-RELEASE-p3, right? :-)

vaksel 18y ago

thanks

lsb 18y ago

So, if you've got all those in-memory data structures, there are lots of lists of pointers to objects. How does it compare on a 32-bit OS, with 32-bit pointers, vs a 64-bit with 64-bit pointers?

cperciva 18y ago
I'd guess that most of the memory is used by blocks of text which are much larger than the pointers which reference them, so probably the size_t expansion has a minimal impact on the total memory usage.
- jwilliams 18y ago
  That's true, but 64 bit will affect other things as well - pointer size goes up, your stack will be bigger and the packing (alignment) of your structures changes. If your compiler is unfriendly, your structure size can change significantly -- afaik, GCC is pretty clever about structure packing, but I'd imagine it varies a lot depending on the architecture.
jwilliams 18y ago
You mean compare in terms of memory usage? It will certainly eat more memory for the same data structures. We've got a simple functional language here - being very unscientific, and looking at the key data structures, for my current workload, straight pointers are about 5-10% of the allocation. In practice I do see a 10-25% memory usage increase going from 32bit to 64bit (Linux). (Anecdotally -- I've read 20% is a rough rule of thumb for 64bit anyway). There is also an increase in data structure size due to data alignment, but this is going to vary quite a bit -- depends on the structure, the compiler -- and if you're doing dynamic memory allocation you may even find it makes no difference at all -- as malloc may have been delivering an oversize allocation anyway. So, it's certainly not a trivial increase... Especially as we're currently running on a 256mb slice (I sort of wish they had a supported 32bit option)... but it's not massive. However, for us, 64 bit still has a lot of advantages. For example, you can do more expansive memory mapping and the like. Not sure if that helps.
notaddicted 18y ago
Obviously the pointers take twice as much space but I don't think that is what you are asking. arc runs in MzScheme 352 (http://download.plt-scheme.org/mzscheme/v352.html http://download.plt-scheme.org/mzscheme/v352.html) the freeBSD binaries on this page are i386, if thats what the new server is using then it won't make a difference. The source is available as well, if pg compiled the source in 64 bit mode it would increase memory usage. But using a 64 bit OS doesn't necessitate using a 64 bit address space.
deleted 18y ago
[deleted]
rtm 18y ago
64-bit mzscheme uses about 50% more memory than 32-bit mzscheme to hold all of the Hacker News data.