7 ms·

It's definitely not about semantics or language. As far as language is concerned similarity metrics are semantically vacuous and quantifying semantic similarity

by _t89y 3y ago

It's definitely not about semantics or language. As far as language is concerned similarity metrics are semantically vacuous and quantifying semantic similarity is a bogus enterprise.

blackbear_ 3y ago
Can you elaborate?
- _t89y 3y ago
  Modeling language in a latent space is useful for certain kinds of analyses and certain aspects of language. It has its place as an empirical tool. That place is not the nuts and bolts of language itself. There are more suitable formalisms for this than directional magnitudes and BPE tiktokens.
  - blackbear_ 3y ago
    What are those formalisms?
  - throwaway2562 3y ago
    Intuitively I agree with you, despite the unexpected ‘success’ of current approaches. What formalisms do you suggest?
    - _t89y 3y ago
      The Lambek calculus. Categorial grammars. Meanings are proofs. Not clusters of directional magnitudes in space.
      - jll29 3y ago
        Curiously, the upcoming third edition of Jurafsky and Martin [1], one of the two standard text books for NLP, places Context-Free Grammars, Combinatory Categorial Grammars, and logical meaning representations in its appendices on the companion Web site, no longer in the text book itself. Unthinkable only a few years ago. [1] https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/
        _t89y 3y ago
        That's a really interesting thing to point out. NLP doesn't even work on language anymore. If it was adjacent to information retrieval before it is now a subfield of information retrieval. As long as it's grounded in Firth Mode natural language understanding, as it's called, can't really be a semantics. I tried to create a Kaggle (TensorFlow Hub, TensorFlow Quantum) competition for motivating alternative formalisms but was unable to publish it because all Kaggle competitions must be evaluated with information retrieval metrics. Talk about a one-track mindset! Today work in NLP advances by ``leaderboards'' and dubious, language-specific evaluation datasets that the same authors stand to benefit from when their proprietary model is praised for doing well on the evaluation criteria they invented a few months back. It validates the price hike for access to their proprietary models. These formalisms that do work are at odds with Firth Mode, the preferred representation for Google (Stanford, OpenAI), so I guess we should be thankful they're still in the book. If you're interested in language, though, I'd suggest picking up a different book.

by _t89y 3y ago

It's definitely not about semantics or language. As far as language is concerned similarity metrics are semantically vacuous and quantifying semantic similarity is a bogus enterprise.

blackbear_ 3y ago

Can you elaborate?

_t89y 3y ago
Modeling language in a latent space is useful for certain kinds of analyses and certain aspects of language. It has its place as an empirical tool. That place is not the nuts and bolts of language itself. There are more suitable formalisms for this than directional magnitudes and BPE tiktokens.
- blackbear_ 3y ago
  What are those formalisms?
- throwaway2562 3y ago
  Intuitively I agree with you, despite the unexpected ‘success’ of current approaches. What formalisms do you suggest?
  - _t89y 3y ago
    The Lambek calculus. Categorial grammars. Meanings are proofs. Not clusters of directional magnitudes in space.
    - jll29 3y ago
      Curiously, the upcoming third edition of Jurafsky and Martin [1], one of the two standard text books for NLP, places Context-Free Grammars, Combinatory Categorial Grammars, and logical meaning representations in its appendices on the companion Web site, no longer in the text book itself. Unthinkable only a few years ago. [1] https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/
      - _t89y 3y ago
        That's a really interesting thing to point out. NLP doesn't even work on language anymore. If it was adjacent to information retrieval before it is now a subfield of information retrieval. As long as it's grounded in Firth Mode natural language understanding, as it's called, can't really be a semantics. I tried to create a Kaggle (TensorFlow Hub, TensorFlow Quantum) competition for motivating alternative formalisms but was unable to publish it because all Kaggle competitions must be evaluated with information retrieval metrics. Talk about a one-track mindset! Today work in NLP advances by ``leaderboards'' and dubious, language-specific evaluation datasets that the same authors stand to benefit from when their proprietary model is praised for doing well on the evaluation criteria they invented a few months back. It validates the price hike for access to their proprietary models. These formalisms that do work are at odds with Firth Mode, the preferred representation for Google (Stanford, OpenAI), so I guess we should be thankful they're still in the book. If you're interested in language, though, I'd suggest picking up a different book.