The current bottleneck for most current hardware is RAM capacity than memory bandwidth and last is FLOPS/TOPS. The coral has 8 MB of SRAM which uh, won't fit t

The current bottleneck for most current hardware is RAM capacity than memory bandwidth and last is FLOPS/TOPS. The coral has 8 MB of SRAM which uh, won't fit the 2GB+ that nearly any decent LLM require even after being quantized. LLMs are mostly memory and memory bandwidth limited right now.