7 ms·
It can emulate a bad English speaker if prompted to do that.
by xcv123 2y ago
It can emulate a bad English speaker if prompted to do that.
- otabdeveloper4 2y agoYes, there's enough explicitly tagged bad English in the training dataset to make a valid average approximation.
- xcv123 2y agoNo. Not explicitly tagged. They are initially trained on vast amounts of data which are not tagged. You fundamentally misunderstand how this works. The LLMs learn the various grammars and "accents" implicitly. They automatically differentiate these grammars. Sounds like you still have this idea that LLMs are a giant Markov chain. They are not Markov chains. They are deep neural networks with hundreds of layers and they automatically model relations at extremely deep levels of abstraction.
- otabdeveloper4 2y agoThe context is the explicit tagging in this case. You don't need to understand language to detect English-as-a-second language speakers. (Indeed Markov chains will happily solve this problem for you.) > they automatically model relations No, they do not model anything at all. If you follow the tech bubble turtles all the way down you find a maximum likelihood logistic approximation. I know, I know - then you'll do a sleight of hand and claim that all intelligence and modeling is also just maximum likelihood, even thought it's patently and obviously untrue.
- xcv123 2y agoIt's literally a model. Large Language Model (LLM). Hundreds of layers with a trillion weights and you think "nothing is modelled" there. The comments on this site are ridiculous. Studies have traced individual "neurons" in LLMs that represent specific concepts. It's not even debatable at this point.