7 ms·

It's a little arbitrary. Look at the graph on page 6, there's no steep gap in the spectrum there. 16 just about the balance point

• moi2388 9mo ago
  But there is a steep gap in the spectrum at 16 on page 7
  • yorwba 9mo ago
    That's the spectrum of LoRAs, which are LoW RAnk by design.
    • moi2388 9mo ago
      Yes. But from their paper: “In our analysis, we present compelling empirical evidence for the existence of universal subspaces within LoRA adapters across different modalities and tasks.” I also don’t understand what they write under figure 2, since resnet50 has 50 layers, not 31.