5 ms·
LLMs reflect the data in the corpus, and that corpus is essentially in American Business English, which has long been the lingua franca of the Internet. I bet
by kjellsbells 16d ago
LLMs reflect the data in the corpus, and that corpus is essentially in American Business English, which has long been the lingua franca of the Internet.
I bet a large %age of the HN readership are consuming the tech stack in their second or third language. Germans blogging in English or Indians commenting in English. We even have a broadly shared tech culture with norms, in-jokes, taboos, etc. LLMs reflect that.
If we want AI to do better, we should pay attention to making sure the non-English (and non-US English) internet thrives. Personally I´d love to see LLMs doing the needful and writing in Indian English, or dinner-party-argument French, instead of the sanitized corporate pablum tone it uses in English.
Or perhaps more controversially, we let LLMs chew up the English internet and deliberately build an offline culture in local languages. Keep your AI slop and your em-dashes, I'm gonna publish a zine in Italian.
- ChickeNES 16d agoIt's refreshing seeing someone actually engage and propose interesting ideas instead of just bemoaning the status quo/future! :)
- anal_reactor 15d agoIt's actually funny because Italy, France and Germany used to be much more linguistically diverse but then public education happened and it all collapsed to the standard language. Dialects and minority languages are being pretty much eradicated.
- orbital-decay 15d agoThe corpus is already massively multilingual, it's really large, and has nothing to do with it. Models are very specifically engineered to respond in a default register when asked in English. There were models trained differently, for example Claude 3 Opus was trained to imitate extremely diverse writing styles and sociolects on command and without a command, and was able to follow writing examples much better than any current model. But it still had poor variety in its vocabulary and ideas, being repetitive regardless of the chosen style. This is an entirely separate issue (mode collapse).