6 ms·

Agglutinative Language

59 points by cubecul 7y ago

sansnomme 7y ago
Turkish is probably strict enough to be used as a programming language. The only downside is that its vocabulary is utterly alien for most speakers of Latin/Anglo-Saxon languages aside from some borrowed words from French and Arabic.
- yabadabadoes 7y ago
  It's actually quite a bit easier to learn since it has few false friends with Latin languages. I often thought search engines written by English speakers focused on bags of words can't work very well in Turkish though?
  - rolleiflex 7y ago
    Not only search, but also autocorrect. Turkish autocorrect on iOS is a flaming disaster even after a decade. Here’s a real (if unlikely) word in Turkish and how this whole agglutination business works: https://twitter.com/languagecrawler/status/623858803868598272?s=21 https://twitter.com/languagecrawler/status/62385880386859827... I don’t blame Apple though - it might actually be just impossible to do Turkish autocorrect in the same way English autocorrect works, because the beginning of the word indicates the actual word but the end indicates everything else (direction, modifiers etc.). So it’s about as easy/hard as English to guess the beginning of the word, but impossible to guess the modifiers that get added because the moment the modifier sequence starts, every single letter starts to change the meaning, thus there are almost no incorrect paths. A correct Turkish autocorrect implementation would autocomplete the word root, but leave at the halfway-compete word at where the modifier suffixes start so that the user can complete the modifier sequence on his / her own.
    - bonoboTP 7y ago
      Seems like you're talking about autocompletion, not autocorrect. In autocorrect you have completed the word, hit space and then the software fixes your typos. In autocomplete you get a list of suggested words while typing and you can tap them if your intended word is shown.
      - rolleiflex 7y ago
        No - while auto completion is also broken, I’m talking about autocorrect. It’s a very common occurrence in Turkish iOS that something that you typed in correctly gets autocorrected to something else that makes no sense.
  - eindiran 7y ago
    All languages with synthetic morphology (both agglutinative languages, which glue chains of morphemes together, and fusional languages, which inflect morphemes) struggle with the language modelling techniques used for English. A big issue is that in synthetic languages 'words' are much more 'rare' (because there are more morpheme combinations per word). So if you're building something like a bag-of-words or an ngram model, your input data is likely to be very sparse which translates to poor modelling of the language itself/what words speakers would judge as grammatical. With agglutinative languages like Turkish, a technique that has been used with considerable success is just considering each morpheme a distinct token, but it has many of the same problems as word-level tokenization. I was looking at a paper recently that claimed to have found a good way to do smoothing so that unseen ngrams could be assigned a non-zero probability in a way that conformed to the rules of the language, but we'll have to see if that can work in practice.
- deleted 7y ago
  [deleted]
monkeycantype 7y ago
I was just reading this yesterday after the term came up in a Japanese grammar book.
Bootwizard 7y ago
Can someone here explain this in an easier to understand way? This was a bit too dense for my understanding...
- sansnomme 7y ago
  Simply put: a lot of grammar is based on appending to words. E.g. the Turkish word for book is Kitab (shared by a bunch of other middle eastern languages too). My Book is Kitabim. Your book is Kitabsin. (Note last example is vastly simplified, a proper Turkish speaker should correct it) It allows for a lot of really short sentences; here's a nonsensical example: His book is on fire - kitabı yanıyor. The word endings is sufficient to provide context and meaning. If you find Turkish to be too difficult to learn, try Malay. It's also agglutinative and used by ~300 million people (Malay and Indonesian are for all practical purposes the same language).
  - stevenpetryk 7y ago
    These are actually mostly wrong - Book = kitap, not kitab - My book = kitabım, not kitabim - Your book = kitabın, not kitabsin
    - sansnomme 7y ago
      The i is due to using English keyboard, it's a bit tricky to type the proper i without autocompletion.
- bonoboTP 7y ago
  Agglutinative just means you glue (the -glu- refers to this) pieces (suffixes) at the end of words to express lots of things. This exists in English as well, but in restricted forms. For example blue+ish, quick+ly, blue+ness, look+ed. In an agglutinative language, this is how most of the things are expressed. For example a totally normal Hungarian word is: szolgáltatásaiért = szolgá+l+tat+ás+a+i+ért = for his/her/its services. Szolga means servant, from Slavic origin. Szolgál is a verb meaning to serve. Szolgáltat means to provide service. Szolgáltatás means service (as in "goods and services", "internet service", etc.). Szolgáltatása means his/her/its service. Szolgáltatásai means his/her/its services. Szolgáltatásaiért means "for his/her/its services".
beefman 7y ago
More broadly, synthetic languages are like statically-typed programming languages, whereas analytic languages[1] are like dynamically-typed programming languages. Also, intransitive verbs[2] are like thunks. [1] https://en.wikipedia.org/wiki/Analytic_language https://en.wikipedia.org/wiki/Analytic_language [2] https://en.wikipedia.org/wiki/Intransitive_verb https://en.wikipedia.org/wiki/Intransitive_verb
- deleted 7y ago
  [deleted]
bonoboTP 7y ago
I often wonder how much of a head start the isolating nature of English gave for computing. It allowed ignoring a lot of inflectional and agglutinative complexity. Concretely I mean it's very easy to generate text using sentence templates. Just plug in words and it works out. "The $process_name has completed running." "Like $username's comment" "Ban $username". Relatedly, I think focusing NLP efforts on English masks a lot of interesting phenomena, because English text already comes in a reasonably tokenized, chunked up and pre-digested, easy to handle form. For example speech recognition systems started out with closed vocabularies, with larger and larger numbers of words, and even in their toy forms you could recognize some proper English sentences. To do that in Hungarian for example, the "upfront costs" to a "somewhat usable" system are much higher, because closed vocabulary doesn't get you anywhere. (Similarly, learning basic English is very easy, you can build 100% correct sentences on day 1, you learn "I", "you", "see" and "hear" and can say "I see" and "You see" and "I see you" and "I hear Peter" which are all 100% correct. In Hungarian these are "nézek", "nézel", "nézlek", "hallom Pétert" requiring learning several suffixes and vowel harmony and definite/indefinite conjugation. The learning curve till your first 100% correct 3-5 word sentences is just steeper.) I don't mean it's impossible to handle agglutinative languages in NLP, I just mean the "minimum viable model" is much simpler and attainable for English, which on the one hand was able to kickstart and propel the early research phases and on the other hand perhaps fueled a bit too much optimism. English can seem very well structured and it can tempt one to think of language in a very symbolic, within-the-box, rule-based way. In terms of syntax trees, sets of valid sentences etc, instead of "fuzzy probabilistic mess" that it really is. Surely, the syntax tree, generative grammar approach (Chomsky and others) gave us a lot of computer science, but this kind of "clean" and pure symbolic parsing doesn't seem to drive today's NLP progress. In summary, I wonder how linguistics and especially computational linguistics and NLP would have evolved in a non-Anglo culture, e.g. Slavic or Hungarian.
- jerf 7y ago
  Being able to encode it reasonably in 5 bits and comfortably in 6 (adding case and a few last nice symbols) was helpful too.
- eindiran 7y ago
  Certainly English's morphosyntactic simplicity helped out NLP; your phrase "minimum viable model" hits the nail on the head. But increasingly over the last 5-10 years, I think there is a lot of progress on techniques for handling morphological complexity. Some of the unsupervised tokenization methods that first saw use for English (eg Goldsmith's work) now sees play for agglutinative languages: see here for example[0]. So its not clear to me if NLP in a non-Anglo culture would just use the same techniques (arriving at practical achievements a decade later) or if there would be fundamentally different techniques that are totally unobvious to me now. Re your point on language being a "[f]uzzy probabilistic mess" -- language is absolutely NOT a fuzzy probabilistic mess and its a damn shame that NLP based its success on black-box models, because it means no one bothers realizing that language isn't a mess at all. See Jelinek's law of speech recognizer accuracy [1]. Simply because we get results using messy black box models doesn't mean that's how things work under-the-hood. [0] https://www.researchgate.net/publication/221013038_Unsupervised_Tokenization_for_Machine_Translation https://www.researchgate.net/publication/221013038_Unsupervi... [1] https://en.wikipedia.org/wiki/Frederick_Jelinek https://en.wikipedia.org/wiki/Frederick_Jelinek
foobar_ 7y ago
Forth is probably the only agglutinative language in a way.
romwell 7y ago
Sumerian, an agglutinative language, is an important plot point in a famous cyberpunk novel, Snow Crash by Neal Stephenson[1] (which also popularized the word "avatar" as we use it today). If you find the concept interesting, you will enjoy reading the novel. [1]https://en.wikipedia.org/wiki/Snow_Crash https://en.wikipedia.org/wiki/Snow_Crash

59 points by cubecul 7y ago

Turkish is probably strict enough to be used as a programming language. The only downside is that its vocabulary is utterly alien for most speakers of Latin/Anglo-Saxon languages aside from some borrowed words from French and Arabic.

yabadabadoes 7y ago
It's actually quite a bit easier to learn since it has few false friends with Latin languages. I often thought search engines written by English speakers focused on bags of words can't work very well in Turkish though?
- rolleiflex 7y ago
  Not only search, but also autocorrect. Turkish autocorrect on iOS is a flaming disaster even after a decade. Here’s a real (if unlikely) word in Turkish and how this whole agglutination business works: https://twitter.com/languagecrawler/status/623858803868598272?s=21 https://twitter.com/languagecrawler/status/62385880386859827... I don’t blame Apple though - it might actually be just impossible to do Turkish autocorrect in the same way English autocorrect works, because the beginning of the word indicates the actual word but the end indicates everything else (direction, modifiers etc.). So it’s about as easy/hard as English to guess the beginning of the word, but impossible to guess the modifiers that get added because the moment the modifier sequence starts, every single letter starts to change the meaning, thus there are almost no incorrect paths. A correct Turkish autocorrect implementation would autocomplete the word root, but leave at the halfway-compete word at where the modifier suffixes start so that the user can complete the modifier sequence on his / her own.
  - bonoboTP 7y ago
    Seems like you're talking about autocompletion, not autocorrect. In autocorrect you have completed the word, hit space and then the software fixes your typos. In autocomplete you get a list of suggested words while typing and you can tap them if your intended word is shown.
    - rolleiflex 7y ago
      No - while auto completion is also broken, I’m talking about autocorrect. It’s a very common occurrence in Turkish iOS that something that you typed in correctly gets autocorrected to something else that makes no sense.
- eindiran 7y ago
  All languages with synthetic morphology (both agglutinative languages, which glue chains of morphemes together, and fusional languages, which inflect morphemes) struggle with the language modelling techniques used for English. A big issue is that in synthetic languages 'words' are much more 'rare' (because there are more morpheme combinations per word). So if you're building something like a bag-of-words or an ngram model, your input data is likely to be very sparse which translates to poor modelling of the language itself/what words speakers would judge as grammatical. With agglutinative languages like Turkish, a technique that has been used with considerable success is just considering each morpheme a distinct token, but it has many of the same problems as word-level tokenization. I was looking at a paper recently that claimed to have found a good way to do smoothing so that unseen ngrams could be assigned a non-zero probability in a way that conformed to the rules of the language, but we'll have to see if that can work in practice.
deleted 7y ago
[deleted]

monkeycantype 7y ago

I was just reading this yesterday after the term came up in a Japanese grammar book.

Bootwizard 7y ago

Can someone here explain this in an easier to understand way? This was a bit too dense for my understanding...

sansnomme 7y ago
Simply put: a lot of grammar is based on appending to words. E.g. the Turkish word for book is Kitab (shared by a bunch of other middle eastern languages too). My Book is Kitabim. Your book is Kitabsin. (Note last example is vastly simplified, a proper Turkish speaker should correct it) It allows for a lot of really short sentences; here's a nonsensical example: His book is on fire - kitabı yanıyor. The word endings is sufficient to provide context and meaning. If you find Turkish to be too difficult to learn, try Malay. It's also agglutinative and used by ~300 million people (Malay and Indonesian are for all practical purposes the same language).
- stevenpetryk 7y ago
  These are actually mostly wrong - Book = kitap, not kitab - My book = kitabım, not kitabim - Your book = kitabın, not kitabsin
  - sansnomme 7y ago
    The i is due to using English keyboard, it's a bit tricky to type the proper i without autocompletion.
bonoboTP 7y ago
Agglutinative just means you glue (the -glu- refers to this) pieces (suffixes) at the end of words to express lots of things. This exists in English as well, but in restricted forms. For example blue+ish, quick+ly, blue+ness, look+ed. In an agglutinative language, this is how most of the things are expressed. For example a totally normal Hungarian word is: szolgáltatásaiért = szolgá+l+tat+ás+a+i+ért = for his/her/its services. Szolga means servant, from Slavic origin. Szolgál is a verb meaning to serve. Szolgáltat means to provide service. Szolgáltatás means service (as in "goods and services", "internet service", etc.). Szolgáltatása means his/her/its service. Szolgáltatásai means his/her/its services. Szolgáltatásaiért means "for his/her/its services".

beefman 7y ago

More broadly, synthetic languages are like statically-typed programming languages, whereas analytic languages[1] are like dynamically-typed programming languages. Also, intransitive verbs[2] are like thunks. [1] https://en.wikipedia.org/wiki/Analytic_language https://en.wikipedia.org/wiki/Analytic_language [2] https://en.wikipedia.org/wiki/Intransitive_verb https://en.wikipedia.org/wiki/Intransitive_verb

deleted 7y ago
[deleted]

bonoboTP 7y ago

I often wonder how much of a head start the isolating nature of English gave for computing. It allowed ignoring a lot of inflectional and agglutinative complexity. Concretely I mean it's very easy to generate text using sentence templates. Just plug in words and it works out. "The $process_name has completed running." "Like $username's comment" "Ban $username". Relatedly, I think focusing NLP efforts on English masks a lot of interesting phenomena, because English text already comes in a reasonably tokenized, chunked up and pre-digested, easy to handle form. For example speech recognition systems started out with closed vocabularies, with larger and larger numbers of words, and even in their toy forms you could recognize some proper English sentences. To do that in Hungarian for example, the "upfront costs" to a "somewhat usable" system are much higher, because closed vocabulary doesn't get you anywhere. (Similarly, learning basic English is very easy, you can build 100% correct sentences on day 1, you learn "I", "you", "see" and "hear" and can say "I see" and "You see" and "I see you" and "I hear Peter" which are all 100% correct. In Hungarian these are "nézek", "nézel", "nézlek", "hallom Pétert" requiring learning several suffixes and vowel harmony and definite/indefinite conjugation. The learning curve till your first 100% correct 3-5 word sentences is just steeper.) I don't mean it's impossible to handle agglutinative languages in NLP, I just mean the "minimum viable model" is much simpler and attainable for English, which on the one hand was able to kickstart and propel the early research phases and on the other hand perhaps fueled a bit too much optimism. English can seem very well structured and it can tempt one to think of language in a very symbolic, within-the-box, rule-based way. In terms of syntax trees, sets of valid sentences etc, instead of "fuzzy probabilistic mess" that it really is. Surely, the syntax tree, generative grammar approach (Chomsky and others) gave us a lot of computer science, but this kind of "clean" and pure symbolic parsing doesn't seem to drive today's NLP progress. In summary, I wonder how linguistics and especially computational linguistics and NLP would have evolved in a non-Anglo culture, e.g. Slavic or Hungarian.

jerf 7y ago
Being able to encode it reasonably in 5 bits and comfortably in 6 (adding case and a few last nice symbols) was helpful too.
eindiran 7y ago
Certainly English's morphosyntactic simplicity helped out NLP; your phrase "minimum viable model" hits the nail on the head. But increasingly over the last 5-10 years, I think there is a lot of progress on techniques for handling morphological complexity. Some of the unsupervised tokenization methods that first saw use for English (eg Goldsmith's work) now sees play for agglutinative languages: see here for example[0]. So its not clear to me if NLP in a non-Anglo culture would just use the same techniques (arriving at practical achievements a decade later) or if there would be fundamentally different techniques that are totally unobvious to me now. Re your point on language being a "[f]uzzy probabilistic mess" -- language is absolutely NOT a fuzzy probabilistic mess and its a damn shame that NLP based its success on black-box models, because it means no one bothers realizing that language isn't a mess at all. See Jelinek's law of speech recognizer accuracy [1]. Simply because we get results using messy black box models doesn't mean that's how things work under-the-hood. [0] https://www.researchgate.net/publication/221013038_Unsupervised_Tokenization_for_Machine_Translation https://www.researchgate.net/publication/221013038_Unsupervi... [1] https://en.wikipedia.org/wiki/Frederick_Jelinek https://en.wikipedia.org/wiki/Frederick_Jelinek

foobar_ 7y ago

Forth is probably the only agglutinative language in a way.

romwell 7y ago

Sumerian, an agglutinative language, is an important plot point in a famous cyberpunk novel, Snow Crash by Neal Stephenson[1] (which also popularized the word "avatar" as we use it today). If you find the concept interesting, you will enjoy reading the novel. [1]https://en.wikipedia.org/wiki/Snow_Crash https://en.wikipedia.org/wiki/Snow_Crash