6 ms·
Not quite. There can be non-deterministic race conditions, and strange head size and sequence length requirements.
by NLPaep 3y ago
Not quite. There can be non-deterministic race conditions, and strange head size and sequence length requirements.
- pama 3y agoYes. For a model within the limits of the head requirements, however, you wouldn’t be able to see a quality difference from regular attention. Non determinism is a performance price; regular transformers may also suffer from it depending on the implementation.