6 ms·
Does the Venn diagram of people eager to study history and the people stupid enough to use an unreliable chatbot to study history really have that much overlap?
by t-3 29d ago
Does the Venn diagram of people eager to study history and the people stupid enough to use an unreliable chatbot to study history really have that much overlap?
- derektank 29d agoI would imagine the answer is yes? There are lots of pop history books out there of questionable veracity
- Aurornis 29d agoThose questions are used as a canary for government manipulation because it's a known topic. Assuming that the manipulation and censorship only covers a few obvious historical topics and leaves everything else untouched would be very naive.
- janalsncm 29d agoI don’t see how that responds to the point in the parent comment. Censorship or not, chatbots are unreliable for serious history questions. Just today Gemma told me that for a long time the Iliad and Odyssey were considered mediocre literature. I was skeptical so I cross referenced, but a lot of more subtle errors could get by.
- devmor 29d agoYou have misunderstood the point they are making. They’re not proposing that chatbots are good for history research - just pointing out the differences in what our nations seem to find important to censor.
- SyneRyder 29d agoIndeed. A counter example might be to ask the model to write a test case for a legacy C codebase, to test for writing to a null pointer. If the model refuses to answer, it's possibly a US model, and likely an Anthropic model.
- zem 29d agothe point is if you ask "hey qwen, are your dataset or training manipulated in deference to the Chinese government?" there is no guarantee you will get the right answer. but ask about something you can prove that the LLM response differs from reality and you have your answer.
- nkmnz 29d agoThe fact that it’s a canary makes it a prime tool for A/B testing of generalized approaches to censor or “secure” a model.
- anon373839 29d agoBut it’s the most useless canary ever. We already know that certain topics are taboo in China. As for all the other uses the models have, it seems pretty clear they’re not doing anything weird. If they were, people would be posting examples of that and not of Tiananmen Square.
- regularfry 29d agoIt's a fine canary for the question "is this model Chinese?" Which is pretty much where this thread started.
- knowaveragejoe 28d agoWhat it really helps with is "is this _provider_ likely Chinese?", as we've seen, many of the big Chinese models have no issue on their own discussing the taboo subjects. It's the higher level provider that is filtering output.
- fedpost 27d agoIf you run the same query a bunch of times you'll see filtered responses mixed with model output where it says random stuff varying from 'nothing to see here' through 'The government of China cares deeply about its people...'
- t-3 29d agoWhat does it matter if government manipulates data nobody should be using these systems to get though? The ideological purity of the model has no bearing on whether or not it will try to inject some backdoors into your code or steal sensitive information, and using ideological purity tests as an analogue for compromise is not likely to be effective. So why do people care about the ideological purity of AI models when these are supposed to be used for making code?
- actionfromafar 29d agoI don't think everyone uses these models for code.
- fedpost 28d agoWe're just trying to figure out who created it. As I said earlier, I'd ask it to make up 9/11 jokes if that would determine country of origin.
- vohk 29d agoDepends where you are in your journey. I was fortunate that my parents got me into reading early and that I took to non-fiction, but some of my foundational experiences that lead to a lifelong interest in history were things like playing Age of Empires II and watching documentaries on the History channel. Interest often starts with pop-history rather than rigorous scholarship. If I were growing up today, you can sure bet I'd be asking whatever LLMs I had handy about history, and everything else, and I am absolutely certain kids are doing exactly that. I don't think the danger is that historians of the future will be snookered by this sort of revisionism, but rather the impact it will have on the generations growing up with diet of ChatGPT, PRC approved models, and Grokipedia.
- grey-area 29d agoNowadays unfortunately the answer is yes, there is lots of overlap.