5 ms·

The point of SRAM, especially at the L1/L2 level is having an extremely high BW and extremely low latency (a few clock cycles). So it is not really an option to

The point of SRAM, especially at the L1/L2 level is having an extremely high BW and extremely low latency (a few clock cycles). So it is not really an option to put them somewhere else (although L3 and as mentioned other lower level layers) can and are already being put into either separate chiplets in the same PCB w/extremely fast ring OR directly on top of the die (3D stacking).

chongli 3y ago
Yeah. The analogy for cache that I like to use is a table at the library. If you think about doing research (the old fashioned way) by looking through a library shelf by shelf and bringing books to your table to read through more closely. If you have a bigger table you can store more books which can speed your lookup times since you don’t need to get up and go back and forth to the shelves. But at some point making your table larger just defeats the purpose of the library itself. Your table becomes the new library, and you have to walk around on it and look up things in these piles of books. So you make a smaller table in the middle of the big table. Your fundamental limitation is how small you can make a memory cell, not how big you want to make a cache. That’s akin to making the books smaller print size so you can fit more on the same size table.
- markhahn 3y ago
  well, sorta, since caches are just sram+tag logic. you can parallelize tables, so that each remains fast, but it costs you power/heat. the decoder inherent to sram is what introduces the size-speed tradeoff.
  - chongli 3y ago
    I was ignoring the details on how SRAM works in favour of thinking about it physically. Most of those details just affect the average cell size at the end of the day. The other physical aspect we’re dealing with is propagation delay and physical distance. That’s where the library analogy really shines: if there’s a minimum size to a book and a minimum size of you (the person doing the research) this corresponds roughly to minimum cell sizes and minimum wire pitch, so you’re ultimately limited in the density you can fit within a given volume.
- randall 3y ago
  Really good analogy!
DeathArrow 3y ago
Is it possible to use big and fat CPU registers instead of cache? There might be no wasted clock cycles and no delay.
- saati 3y ago
  CPU registers are either SRAM or even larger flip-flops, they have the same problem.
- Dylan16807 3y ago
  Registers are quite expensive in space and power, because multiple at once have to be accessible in many places. If you add more registers, the cost per register increases rapidly, and you very quickly hit your limits. If you make registers wider, that's still very expensive, and you introduce extra steps to get to your data most of the time. So no, you can't do that in a reasonable way.
  - DeathArrow 3y ago
    Thank you!
- Nevermark 3y ago
  A compiler AND processor design amateur here. (Latter in school.) Once you have enough registers, having more mean lowers active utilization for any given instruction (bad use of space, vs. fast pipelined access to cached stack) or higher levels of parallel instruction dispatch (much greater complexity, and even greater inefficiency for branching misses). Then you have to update instruction sets, which could be impossible given how tightly they fit in current instruction sizes. Ergo, increasing register banks is a major architecture & platform change from hardware to software redesign, with heavy end user impact, and a fair chance of decreasing performance. In contrast, anything that improves caching performance is a big non-disruptive win.
  - als0 3y ago
    What about if you use register windows or special renaming of architectural registers to internal ones? https://en.wikipedia.org/wiki/Register_window https://en.wikipedia.org/wiki/Register_window

chongli 3y ago

Yeah. The analogy for cache that I like to use is a table at the library. If you think about doing research (the old fashioned way) by looking through a library shelf by shelf and bringing books to your table to read through more closely. If you have a bigger table you can store more books which can speed your lookup times since you don’t need to get up and go back and forth to the shelves. But at some point making your table larger just defeats the purpose of the library itself. Your table becomes the new library, and you have to walk around on it and look up things in these piles of books. So you make a smaller table in the middle of the big table. Your fundamental limitation is how small you can make a memory cell, not how big you want to make a cache. That’s akin to making the books smaller print size so you can fit more on the same size table.

markhahn 3y ago
well, sorta, since caches are just sram+tag logic. you can parallelize tables, so that each remains fast, but it costs you power/heat. the decoder inherent to sram is what introduces the size-speed tradeoff.
- chongli 3y ago
  I was ignoring the details on how SRAM works in favour of thinking about it physically. Most of those details just affect the average cell size at the end of the day. The other physical aspect we’re dealing with is propagation delay and physical distance. That’s where the library analogy really shines: if there’s a minimum size to a book and a minimum size of you (the person doing the research) this corresponds roughly to minimum cell sizes and minimum wire pitch, so you’re ultimately limited in the density you can fit within a given volume.
randall 3y ago
Really good analogy!

DeathArrow 3y ago

Is it possible to use big and fat CPU registers instead of cache? There might be no wasted clock cycles and no delay.

saati 3y ago
CPU registers are either SRAM or even larger flip-flops, they have the same problem.
Dylan16807 3y ago
Registers are quite expensive in space and power, because multiple at once have to be accessible in many places. If you add more registers, the cost per register increases rapidly, and you very quickly hit your limits. If you make registers wider, that's still very expensive, and you introduce extra steps to get to your data most of the time. So no, you can't do that in a reasonable way.
- DeathArrow 3y ago
  Thank you!
Nevermark 3y ago
A compiler AND processor design amateur here. (Latter in school.) Once you have enough registers, having more mean lowers active utilization for any given instruction (bad use of space, vs. fast pipelined access to cached stack) or higher levels of parallel instruction dispatch (much greater complexity, and even greater inefficiency for branching misses). Then you have to update instruction sets, which could be impossible given how tightly they fit in current instruction sizes. Ergo, increasing register banks is a major architecture & platform change from hardware to software redesign, with heavy end user impact, and a fair chance of decreasing performance. In contrast, anything that improves caching performance is a big non-disruptive win.
- als0 3y ago
  What about if you use register windows or special renaming of architectural registers to internal ones? https://en.wikipedia.org/wiki/Register_window https://en.wikipedia.org/wiki/Register_window