8 ms·
I think they are talking about the word embeddings, where context is embedded into high geometric dimensions (one dimension might capture how 'feminine' a word
by qorrect 2y ago
I think they are talking about the word embeddings, where context is embedded into high geometric dimensions (one dimension might capture how 'feminine' a word is, or how 'blue' it is).
- cubefox 2y agoWhich word embeddings get their own dimension though, and which don't? ("feminine" and "blue" are words like any other)
- exe34 2y agomaybe it's like a PCA/Huffman deal, where the more regularly useful ones get to be the eigenvectors.
- willy_k 2y agoMy extremely naive understanding is that the more useful ones, which also tend to be structures of language like gender or color, get their own dimensions, and other embedding are represented with combinations. A weak illustration of this is this site[1], from an HN post a few months ago[2]. [1] https://neal.fun/infinite-craft/ https://neal.fun/infinite-craft/ [2] https://news.ycombinator.com/item?id=39205020 https://news.ycombinator.com/item?id=39205020
- exe34 2y agoooooh what if qualia is just embedding? some philosophers would get their toga in a twist!
- darby_nine 2y agoNot Wittgenstein :cool: