5 ms·

I think you've misunderstood. Megabyte scales better with context window length. I don't know if they're saying the training data / compute are any more efficie

I think you've misunderstood. Megabyte scales better with context window length. I don't know if they're saying the training data / compute are any more efficient.