5 ms·
Why do you think it doesn't have understanding of semantics? I think that was one of the first things to fall to LLMs, as even early models interpreted the word
by PlasmaPower 6mo ago
Why do you think it doesn't have understanding of semantics? I think that was one of the first things to fall to LLMs, as even early models interpreted the word "crashed" differently in "I crashed my car" and "I crashed my computer", and were able to easily conquer the Winograd schema challenge.
- alpaca128 6mo ago> even early models interpreted the word "crashed" differently in "I crashed my car" and "I crashed my computer" That has nothing to do with semantical understanding beyond word co-occurrence. Those two phrases consistently appear in two completely different contexts with different meaning. That's how text embeddings can be created in an unsupervised way in the first place.
- esafak 6mo agoWhat do you mean? Semantics are determined by distribution. https://en.wikipedia.org/wiki/Distributional_semantics https://en.wikipedia.org/wiki/Distributional_semantics