16 ms·

Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

• deleted 2y ago
  [deleted]
• 3abiton 2y ago
  If you were a book recommender system, what other books are similar to yours? I've read plenty of science/maths light read non-fiction, so I want to compare the reading experience before I jump into the book.
• bertman 2y ago
  Synopsis from the project's "instructor guide: >This is not a computer science course, nor even an information science course—though naturally it could be used in such programs. >Our aim is not to teach students the mechanics of how large language models work, nor even the best ways of using them in various technical capacities. >We view this as a course in the humanities, because it is a course about what it means to be human in a world where LLMs are becoming ubiquitous, and it is a course about how to live and thrive in such a world.
  • deleted 2y ago
    [deleted]
• deleted 2y ago
  [deleted]
• ArchitectAnon 2y ago
  [flagged]
  • deleted 2y ago
    [deleted]
• deleted 2y ago
  [deleted]
• ssivark 2y ago
  Kudos; feels very timely! I feel that one underappreciated nuance is why we cannot use human examinations to judge AI. I haven't seen this satisfactorily spelt out anywhere, so I recently wrote a Twitter thread [1], including an example with running -vs- biking. It might be worth making sure your students understand this. Happy to expand on any aspects if you seek. [1] : https://x.com/ergodicthought/status/1887774722706063606 https://x.com/ergodicthought/status/1887774722706063606
  • TeMPOraL 2y ago
    Perhaps it's no longer being spelled out because it's getting outdated? In your thread you argue we can't assume AI models generalize the same way we do (which is technically true except maybe not in the limit), but you seem to be worried about the extent of generalization ability (like learning to run vs. bike example, in terms of generalizing from either to climbing stairs). Thing is, people made these objections a lot until the last year or two - this is what we're now calling a narrow AI problem. A "hot dog or not?" classifier ins't going to generalize into open-ended visual classifier of arbitrary images; a sentiment analysis bot isn't going to generalize into an universal translator; a code completion model isn't going to be giving good personal advice while speaking in pirate poetry. Specialized models fundamentally couldn't do that. But we went past that very rapidly, and for the past half a year or so, we've already seen models excelling at every single task listed above simultaneously. Same architecture, same basic training approach, few extra modalities, ever growing capabilities. Between that and both successes and failures being eerily similar to how humans succeed or fail at these tasks, it's understandable that people are perhaps no longer convinced this class of models can't generalize in a similar way to how humans do.
    • ssivark 2y ago
      > But we went past that very rapidly, and for the past half a year or so, we've already seen models excelling at every single task listed above simultaneously. Same architecture, same basic training approach, few extra modalities, ever growing capabilities. With due deference to the title of the top-level post, I'm tempted to call bullshit unless your claim can be justified. Just because a single model can do a handful of things you've listed doesn't mean that its capabilities are not "jagged"; you've just cherry-picked a few things it can do among the countless things it cannot yet. If AI really were so good at every single task, then (for example) it wouldn't matter much how you prompt it. PS: I really do want to debate this further and understand your perspective, so I will reach out for continuing discussion.
• ThouYS 2y ago
  they're quite useful for being "bullshit machines"
  • aqueueaqueue 2y ago
    I read a bit and the book is more nuanced/fair/unbiased than the site url suggests.
• KronisLV 2y ago
  This is a pretty admirable goal! I'm saying this unironically, but I wish there were courses on looking at information critically and more in how to have a healthy and safe life in the modern day world (including things like data security, how to deal with social media etc.) that would be taught to everyone in schools/colleges/universities. In my country, there are still public announcements about not trusting random people calling you, never giving your bank details to strangers (every bank homepage says that, that the employees will never ask for that stuff) and people regularly get scammed anyways, the only thing sort of saving them is that scamming is only scalable so far... until you throw automation in the mix, in addition to just plainly spreading misinformation about any topic, or even just allowing people to be confidently incorrect and eliminating the need for them to even think that much (e.g. students just asking ChatGPT to do their homework). Any step at least in the direction of educating people feels like a good thing. That said, I don't hate LLMs or anything, I use them for development more or less daily (lovely for boilerplate in your average enterprise Java codebase, for example) and recently saw this project, which made me happy: https://sites.google.com/view/eurollm/home https://sites.google.com/view/eurollm/home
  • deleted 2y ago
    [deleted]
  • XorNot 2y ago
    Most scam prevention fails though because the world is full of exceptions. Like it's mind-blowing to me that charities still call you and ask for credit card details over the phone and this is like...a legitimate way to go about things. Or that any government agency calls you and doesn't just leave a verifiable number to call the operator back on.
    • KronisLV 2y ago
      > Like it's mind-blowing to me that charities still call you and ask for credit card details over the phone and this is like...a legitimate way to go about things. > Or that any government agency calls you and doesn't just leave a verifiable number to call the operator back on. That's rather unfortunate! I wonder if in those cases it'd be better to tell them that you'll get in contact through e-mail or something, because then at least it's you going to their actual homepage, looking up contact details and communicating through that. In my country, we also have a bunch of governmental e-services, one of which is a web based communication platform with most institutionns (translated description, because they haven't bothered to translate it themselves, and also sometimes block connections from outside the country): > An e-address, or official electronic address, is a personalized mailbox on the Latvija.gov.lv portal for unified and secure communication with state and local government institutions. The e-address system organizes secure, efficient and high-quality e-communication and e-document circulation between state institutions and private individuals, ensuring data confidentiality and protection of personal data from unauthorized access, unlawful processing or disclosure, accidental loss, alteration or destruction. An e-address is not e-mail, but its use is similar. Communication in an e-address is confidential, and the data is guaranteed to be available only to you and the institution you contacted. The main purpose of an e-address is to replace registered paper letters with electronic ones in cases where a state administration institution needs to send information and documents to a specific resident or entrepreneur. Citizens and entrepreneurs can also contact more than 3,000 institutions at any time and from any location via E-address. These include not only state and local government institutions, such as the Food and Veterinary Service, the State Labor Inspectorate, the Competition Council, etc., but also judicial institutions, sworn bailiffs and insolvency administrators, as well as private individuals to whom state administration tasks have been delegated. That seems like a pretty good common sense idea for organizing trusted 2 way communication.
• aidos 2y ago
  This is amazing! I was speaking to a friend the other day who works in a team that influences government policy. One of the younger members of the team had been tasked with generating a report on a specific subject. They came back with a document filled with “facts”, including specific numbers they’d pulled from a LLM. Obviously it was inaccurate and unreliable. As someone who uses LLMs on a daily basis to help me build software, I was blown away that someone would misuse them like this. It’s easy to forget that devs have a much better understanding of how these things work, can review and fix the inaccuracies in the output and tend to be a sceptical bunch in general. We’re headed into a time where a lot of people are going to implicitly trust the output from these devices and the world is going to be swamped with a huge quantity of subtly inaccurate content.
  • aqueueaqueue 2y ago
    I made the same sort of mistake with the internet being young back in 93! Having a machine do it for you can easily turn into brain switch off.
    • hunter-gatherer 2y ago
      I keep telling everyone that the only reason I'm paid well to do "smart person stuff" is not because I'm smart, but because I've steadily watched everyone around me get more stupid over my life as a result of turning their brain switch off. I agree a course like this needs to exist, as I've seen people rely on chatGPT for a lot of information. Just yesterday I demonstrated with some neighbors about how easily it could spew bullshit if you sinply ask it leading questions. A good example is "Why does the flu inpact men worse than women"/"Why foes the flu impact women worse than men". You'll get affirmative answers for both.
      • directevolve 2y ago
        If men are more likely to die from flu if infected, and women more likely to be infected, an affirmative answer to both questions could be reasonable. When you take into account uncertainty about the goals, knowledge and cognitive capacity of the person asking the question, it's not obvious to me how the AI ought to react to an underspecified question like this. Edit: When I plug this into a temporary chat on o3-mini, it gives plausible biochemical and behavioral mechanisms that might explain a gender difference in outcomes. Notably, the mechanisms it proposes are the same for both versions of the question, and the framing is consistent. Specifically, for the "men worse than women" and "women worse than men" questions, it proposes hormone differences, X-linked immune regulatory genes, and medical care-seeking differences that all point toward men having worse outcomes than women. It describes these factors in both versions of the question, and in both versions, describes them as explaining why men have worse outcomes than women. It doesn't specifically contradict the "women have worse outcomes than men" framing. But it reasons consistently with the idea that men have worse outcomes than women either way the question is posed.
• aqueueaqueue 2y ago
  Thanks for making this, for making it gratis, and making it interesting to read and pedagogical.
• joshdavham 2y ago
  Do you feel that you may be being a bit provocative by calling LLM's 'bullshit machines'? I understand the frustration as I've been bullshitted by these models just as much as the next programmer, but surely with recent advancements in RAG and reasoning, they're not just 'bullshit machines' at this point, are they?
  • anon373839 2y ago
    “Bullshit” actually means something: > In philosophy and psychology of cognition, the term "bullshit" is sometimes used to specifically refer to statements produced without particular concern for truth, clarity, or meaning, distinguishing "bullshit" from a deliberate, manipulative lie intended to subvert the truth. https://en.m.wikipedia.org/wiki/Bullshit https://en.m.wikipedia.org/wiki/Bullshit It’s really an ideal term to describe what LLMs do.
    • HPsquared 2y ago
      I prefer "waffle" https://en.m.wikipedia.org/wiki/Waffle_(speech) https://en.m.wikipedia.org/wiki/Waffle_(speech) "Waffle machines" is even kind of funny.
      • deleted 2y ago
        [deleted]
      • m0llusk 2y ago
        Makes me imagine coming up to the hindquarters of a bull with a waffle machine on an extension cord.
      • fzzzy 2y ago
        Waffle machines is way better. Love it. Thanks.
      • burch45 2y ago
        Waffle means something very different though in the U.S., to “flip-flop” on a position. Not hold it for any fundamental reasons. But I don’t think you can say that an LLM holds a position whatsoever. Also, https://en.m.wikipedia.org/wiki/On_Bullshit https://en.m.wikipedia.org/wiki/On_Bullshit the essay originally popularizing the definition of bullshit considered here. Note the references to LLM output at he bottom.
• picafrost 2y ago
  I think a great number of working professionals need a course like this too. I am already tired of ChatGPT being cited by the less experienced as an invisible expert in the room during technical discussions.
  • fujinghg 2y ago
    I'm at the state of thinking that I am quite happy to let them screw themselves with it. I am very good at clearing up disasters and getting paid a hell of a lot for it as the deciding factor isn't your ability to use an LLM but to know what the hell you are doing. We have had quite a few disasters due to inexperienced and experienced people throwing stuff into an LLM and assuming it has any veracity or authority over what comes out. I tried warning at first and reinforcing validation but I was poo pooed as a spoilsport luddite with basically a faith argument. Not my fucking funeral!
  • deleted 2y ago
    [deleted]
  • deleted 2y ago
    [deleted]
  • iamacyborg 2y ago
    This stuff is so frustrating, I have colleagues who sent long, clearly AI generated documents who don’t seem to understand that if they can’t be arsed to write something, why should I bother reading it?
  • jaimebuelta 2y ago
    Write well is think well. A big part of the writing process is being forced to structure your thoughts and ideas, and I am worried that we focus too much on the end result without understanding the process that lead to good outcomes.
• Karrot_Kream 2y ago
  Great stuff! LLMs, social media, the information landscape has changed so much in the past decade. We need good pedagogical resources on how to think of these tools, both their benefits and their downsides.
  • deleted 2y ago
    [deleted]
• mkarliner 2y ago
  I wish I'd written this. Excellent. Everyone should read this
• Angostura 2y ago
  I just wanted to thank you. I have only looked at the first two lessons so far, but this is an extraordinary piece of work, in its message’s clarity, accessibility and the quality of analysis. I will certainly be spreading it far and wide and it is making me rethink my own writing. Impressed with the Shorthand publishing system too. I hadn’t come across it previously
  • ctbergstrom 2y ago
    Thank you, and as a non-designer, I've been quite impressed with Shorthand in the short time I've been using it.
• padolsey 2y ago
  Is there a way to download and read this as a document instead of web pages? They're hard to navigate.
  • deleted 2y ago
    [deleted]
  • monomial 2y ago
    I agree, I just want normal text instead of all the images and scrolling. The content seems great but it's a bit unreadable as is.
  • 63stack 2y ago
    Was thinking the same, the image slide-ins are broken in firefox and unreadable (white text on white background)
    • ctbergstrom 2y ago
      I'm surprised at the firefox problems; I did almost all the development in firefox. I know it's not your job to fix any of this, but if you are so inclined I'd be grateful for an email me with screenshots or descriptions of where things break.
  • klabb3 2y ago
    Was hoping HN would pick this up. Scroll is completely broken on Firefox (iOS), flickering vscroll. Very common with journalistic expose-style articles. For the love of everything, please stop scrolljacking. Layout, images, go nuts. CSS is powerful these days, use it.
  • ctbergstrom 2y ago
    Many comments about this, so I'll address them here. We talked extensively with the 18-20 year olds who make up our target demographic and this "scrollytelling" style is their strong preference over the "wall of text" that I and most of my generation prefer. What your comments make clear is that we need to develop a parallel version that is more less plain text for people who are using a range of devices, for people who have the same reading preferences that I do, etc. Right now we're entirely self-funded and doing this on spare time but it's clear to me that an alternative version with a very clean CSS layout is the way to go, possibly with a pdf option as well. I don't want to let versions proliferate too extensively, simply because this is very much a living document. Technologies are changing so fast in this area that many of the examples will seem dated in a year and — while we've tried to be forward-thinknig about this — some of principles may even need revision.
• teknopaul 2y ago
  LLMs pattern match, they say something that sounds good at this point but with no notion of correct. copilot is like pair programming with a loud pushy intern that has seen you write stuff before didn't understand it, but keeps suggesting what to do anyway. some medium sized chunks of code can be delegated but everyline it writes needs careful review. Crazy tech, but companies are just wring to be trying to use LLMs as any kind of source of truth. Even Google is blind enough to think that aí could be used for search results, which are memes they are soo bad. And they won't get better. They just become more convincing
  • teknopaul 2y ago
    Not important once has copilot ever suggested a correction, found a bug, noticed a typo, prompted for a better solution, which is what any human pair programmer would do. It's a tool. But thinking ng it's like a "copilot" marketing as such is fundamentally missing the point. It won't get better untill people recognise what it _can't_ do as much as what it appears it can do.
  • kristopolous 2y ago
    I've had quite a bit of success but my technique is to explain the technology and libraries I'm going to use, think through the problem, stub out function names, how they'll interact, and then llm saves me the typing. I'll also use openrouter with sessions so I can take one context and use it around a variety of invocation tools without losing the attention. It hasn't done anything I don't know how to do - fails if I ask it to do that. But it does save me lots of typing and thinking of minutia It's not magic, it's still just a program running on a computer - a decent abstraction tool. I'm sure it will be ruined in time like every new paradigm when the next generation feels a need to complicate this new tidy little world.
  • deleted 2y ago
    [deleted]
• abmmgb 2y ago
  Your site looks cool! Nice topic! Some of them just try to predict the most likely next word. With reasoning and pause for thought they are becoming more capable. Most likely there is a big element of hype but the way you use them can make them really useful and accelerate your work. I recommend the CoIntelligent book for newbie like myself.
  • deleted 2y ago
    [deleted]
• K0balt 2y ago
  There is a bit of very important content missing from the explanation of the autocomplete analogy. The combination of encoding / tokenization of meanings and ideas, related concepts, and mapping these relationships in vector space makes LLMs not so much glorified text prediction engines as browsers/oracles of the sum total of cultural-linguistic knowledge as captured in the training corpus. Understanding how the implicit and explicit linguistic, memetic, and cultural context is integrated into the idea/concept/text prediction engine helps to show how LLMs produce such convincing output and why they often can bring useful information to the table. More importantly, understanding this holistically can help people to predict where the output that LLMs can generate will -not- be particularly useful or even may be wildly misleading.
  • deleted 2y ago
    [deleted]
  • lazide 2y ago
    It’s also why they can produce such hard to identify bullshit and harmful output. I’ve had some really convincing, yet fundamentally flawed, code output that if I hadn’t done about a million code reviews before I might have just used. And been totally screwed later. Near as I can tell, that the bullshit is so much more convincing with them is a huge detriment that society really won’t learn to appreciate until it’s gotten really bad. As I noted in another thread, it allows people to get much further into the ‘fake it until you make it’ hole than they otherwise would. That 90% of the time it’s fine is what actually makes it all worse.
    • dave4420 2y ago
      The uncanny valley of competence.
    • K0balt 2y ago
      This is the big pain point to be sure. Subtly wrong but mostly excellent results.
  • Earw0rm 2y ago
    What they capture is not knowledge, it's word relationships. And that can indeed be powerful, useful and valuable. They're a tool I'm grateful to have in my armoury. I can use it as a torch to shine light into areas of human knowledge which would otherwise be prohibitively difficult to access. But they're information retrieval machines, not knowledge engines.
• deleted 2y ago
  [deleted]
• emseetech 2y ago
  Fantastic work. Quick suggestion: a link at the bottom of the page to the next and previous lesson would help with navigation a ton.
  • ctbergstrom 2y ago
    Absolutely. Great point. I just finished updating accordingly. My design options are a bit limited so I went with a simple link to the next lesson.
    • threecheese 2y ago
      Looks like you pushed this midway through my read; I was pleasantly surprised to suddenly find breadcrumbs at the end and didn’t need to keep two tabs open. Great work, and I mean in total - this is well written and understandable to the layman.
      • ctbergstrom 2y ago
        Yep, I probably did. I really appreciate all of the feedback people are providing!
• neuronic 2y ago
  > Moreover, a hallucination is a pathology. It's something that happens when systems are not working properly. > When an LLM fabricates a falsehood, that is not a malfunction at all. The machine is doing exactly what it has been designed to do: guess, and sound confident while doing it. > When LLMs get things wrong they aren't hallucinating. They are bullshitting. Very important distinction and again, shows the marketing bias to make these systems seem different than they are.
  • silvestrov 2y ago
    LLMs are always bullshitting, even when they get things right, as they simply do not have any concept of truthfulness.
    • looofooo0 2y ago
      But you can combine them with something producing truth such as a theorem prover.
    • sgt101 2y ago
      They don't have any concept of falsehood either, so this is very different from a human making things up with the knowledge that they may be wrong.
      • tmnvdb 2y ago
        I think the first part of that statement requires more evidence or argumentation, especially since models have shown the ability to practice deception. (you are right that they don't _always_ know what they know)
      • remich 2y ago
        But sometimes when humans make things up they also don't have the knowledge they may be wrong. It's like the reference to "known unknowns" and "unknown unknowns". Or Dunning-Kruger personified. Basically you have three categories: (1) Liars know something is false and have an intent to deceive (LLMs don't do this) (2) Bullshitters may not know/care whether something is false, but they are aware they don't know (3) Bullshitters may not know something is false, because they don't know all the things they don't know Do LLMs fit better in (2) or (3)? Or both?
• einrealist 2y ago
  This website is so important! Now ask yourself why AI companies don't want to be regulated or scrutinized. So many companies (users and providers) jump on the AI hype train because of FOMO. The end result might be just as destructive as this mythical "AGI". Edit: I am not saying to not use the technology. I am just on the side of caution and constant validation. The technology has to serve society. But I fear this hype (and ideology) has it the other way around. Musk isn't destroying the US government for no reason...
  • vladms 2y ago
    My impression is that companies in most of the fields do not like to be regulated or scrutinized, so nothing new there. While observing some people using LLMs, I realized that for a lot of people it really makes a huge difference in time saved. For me the difference is not significant, but I am generally solving complex problems, not writing nicely formatted reports where words and not numbers are relevant, so YMMV.
    • rwmj 2y ago
      Is it good for one person (the writer) to save time, only for lots of other people (the readers) to have to do extra work to understand if the work is correct or hallucinated?
      • tmnvdb 2y ago
        Is it good for one person (the writer) to ask a loaded question just to save some time on making their reasoning explicit, ony for lots of other people (the readers) to have to do extra work to understand what the argument is?
      • csa 2y ago
        > Is it good for one person (the writer) to save time, only for lots of other people (the readers) to have to do extra work to understand if the work is correct or hallucinated? This holds true whether an LLM/AI is used or not — see substantial portions of Fox News editorial content as an example (often kernels of truth with wildly speculative or creatively interpretive baggage). In your example, a responsible writer who uses AI will check all content produced in order to ensure that it meets their standards. Will there be irresponsible writers? Sure. There already are. AI makes it easier for them to be irresponsible, but that doesn’t really change the equation from the reader’s perspective. I use AI daily in my work. I describe it as “AI augmentation”, but sometimes the AI is doing a lot of the time-consuming stuff. The time saved on relatively routine scut work is insane, and the quality of the end product (AI with my inputs and edits) is really good and consistent.
• tialaramex 2y ago
  This course mentions the famous Apple advertisement. Unfortunately it slightly oversells and while I'm sure that's not because this fragment was written by an LLM it is exactly the sort of over-simplification which leads to LLMs generating wild bullshit when they interpolate this "fact" with other "facts" they've been fed, and we ought to strive to do better when writing for humans. "Describe how prior to 1984, there was no such thing as a graphical user interface, visual desktop, an intuitive menu system, or mouse-based navigation." Apple were offering a mass market product which had these features so that's important - but there had been "such a thing" for quite some time before that. Douglas Engelbart's "Mother Of All Demos" in 1968 -- Sixteen years earlier shows all the features you mentioned. https://en.wikipedia.org/wiki/The_Mother_of_All_Demos https://en.wikipedia.org/wiki/The_Mother_of_All_Demos Unfortunately the demo is very long for a modern audience, so unlike "Watch a Superbowl ad" it's a hard sell to show the entire demo, but do go watch for yourself.
  • rightbyte 2y ago
    I like the distinction between "teletypes" and the new fancy "glass teletypes".
  • leoc 2y ago
    Jimmy Carter installed a Xerox Alto in the White House in 1978! https://www.ourmidland.com/news/article/Check-out-the-first-computer-in-the-White-House-8361290.php https://www.ourmidland.com/news/article/Check-out-the-first-... Never mind the Xerox Star, or Apple shipping the Lisa in 1983 ...
  • deleted 2y ago
    [deleted]
  • deleted 2y ago
    [deleted]
  • ctbergstrom 2y ago
    You're right of course. In the original drafts I had a long section on this, including some of the history of the GUI, the development of the mouse, etc. It was way too much for the main text when the point is just to set up a metaphor for students who have seen a Mac 128. That said, we can and should do better in the instructor guide. Thanks for the reminder. I'll add some context there.
• Almondsetat 2y ago
  I'm sorry, but this website is awful. Not only does it have an illogical structure (table of contents at the end? no "next lesson" button? gigantic images that fill the entire screen?), but the aesthetic of the entire thing is off. It tries to be sleek and modern with scrolling animations, but they are janky and rigid and the images are rectangles put in front of a bad gradient. Not to mention the video interviews are badly produced (clipping audio, interviewer doesn't have a dedicated microphone) and it's not even clear why they're there. Please, take inspiration from actual e-learning platforms.
  • Kevcmk 2y ago
    I would like to read this but the jerkiness of needing to scroll 1 page per paragraph renders this unusable
  • heikkilevanto 2y ago
    I agree. I tried the first chapter with the Reader Mode in FireFox, and the whole long scroll hell collapsed to about one screenful of text. I have a feeling it skipped some text, but the result was a quick read that got the main points through. I wish the whole thing was available in a plain text format, preferably in one longer document.
  • allenu 2y ago
    Totally agree here. I visited the page and scrolled through it to see what it was all about and saw a bunch of pull quotes and couldn't work out what I was looking at. It just looks like a light magazine article or brochure until you either click on the hamburger menu to see a full table of contents or scroll to the very bottom to see the table of contents there. This really needs some design improvements if they want people to read through to the actual lessons. Most people are going to drop off after scrolling through that landing page.
• deleted 2y ago
  [deleted]
• deleted 2y ago
  [deleted]
• sgt101 2y ago
  I wish the title wasn't so aggressively anti-tech though. The problem is that I would like to push this course at work, but doing so would be suicidal in career terms because I would be seen as negative and disruptive. So the good message here is likely to miss the mark where it may be most needed.
  • stefantalpalaru 2y ago
    [dead]
  • beepbooptheory 2y ago
    Really? I am curious how this could be disruptive in any meaningful sense. Whose feelings could possibly be hurt? It just feels like it would be getting offended from a course on libraries because the course talks about how sometimes the book is checked out.
    • mpbart 2y ago
      Any executive who is fully bought in on the AI hype could see someone in their org recommending this as working against their interest and take action accordingly.
      • sgt101 2y ago
        Yes. This is the issue. "not on board", "anti-innovation", "not a team player", "disruptive", "unhelpful", "negative". bye bye bye bye.... I see a lot of devs and IC's taking the attitude that "facts are facts" and then getting shocked by a) other people manipulating information to get their way and b) being fired for stating facts that are contrary to received wisdom without any regards to politics.
    • hcs 2y ago
      > I just feels like it would be getting offended from a course on libraries because the course talks about how sometimes the book is checked out. If it was called "Are libraries bullshit?" it is easy to imagine defensiveness in response. There's some narrow sense in which "bullshit" is a technical term, but it's still a mild obscenity in many cultures.
• lm28469 2y ago
  > We were promised hyper-intelligent computer systems that would usher in an era of unparalleled prosperity and innovation. Automated factories were supposed to deliver us from work, yet we work as much if not more than before for a lesser part of the profit cake. We can't keep falling for the capitalist trick over and over...
  • dailykoder 2y ago
    We were also told that smart home will make our life easier and save time to have more available for the "important" things in life. But reality is that people just spend more time with their smart home things and are even shouting at their computers even though the computers just do what they were told...
• tmnvdb 2y ago
  Hmm, it seems that the author takes very clear (and sometimes cynical) positions on some controversial questions. For example, "They don't have the capacity to think through problems logically." is an hotly debated claim, and I think with the advent of reasoning models this has at least become something one should not state in entry level material, which would hopefully reflect common understanding rather than the authors personal opinion in an ongoing discussion. There are more claims like this about what language models can't do "because they just predict the next token". This line of reasoning, while superficially plausible, holds a lot of assumptions that have been questioned. The heavy lifting here is done by the word "just" - if you can correctly predict the next token in every situation (including novel challenges), does that not require an excellent world model - somehow explicitly reflected in the weights? This is not a settled question but the last few years of LLM success have been completely on the side of those who think that token prediction is quite general. The material also makes several comparisons to human intelligence, and while it is obvious that humans are different from language models we do not really understand the emergence of all the things that are claimed to be "impossible" for the machine to have in humans (consciousness, morality, etc), it just so happens we are all human so we all agree we have it. Furthermore, it is not clear to me that something can only be called 'intelligent' if it perfectly mimics humans in every way. This is maybe just human bias to our own experience and risks a "submarines can't swim" debate which is really about language. Many of these philosophical objections have been questioned by people in the field and more importantly by the rapid progress of the models in tasks they were supposed to be incapable of performing according to philosophical objectors. The last few years, every time somebody claims models "can't do X" a new model is released and lo and behold, X is now easy and solved. (If you read a 6 month old paper of impossible benchmarks, expect 75% to be already solved). In fact, benchmark satuation is a problem now. In other words, the goalposts are having trouble keeping up, despite moving at high speed. I don't think you are doing the general public any service by simply claiming that it is a lot of hype and marketing, these models are really advancing rapidly and nobody really knows where it will end. The philosophical objections seem to be rather weak and are in rapid retreat with every new model, on the other hand the argument in favor of further progress is just "we had progress so far by scaling, if we keep scaling surely we will have more progress" (induction). This is not a strong guarantee of further progress. The claim that the labs are 'marketing geniuses" for realising language models as chat instead of autocomplete (which they "really" are according to the text - what does that mean?) also seems a bit silly given the obvious utility of the models is already much higher than 'autocomplete'. This seems to be another instance of the common bias that a model that "just" predicts the next token is not allowed to be as succesful as it clearly is in all kinds of tasks. I don't think a lot of these opinions are particularly well founded and they probably should not be presented in entry level material as if they are facts. Edit: just to add a positive note, I do think it is extremely useful to educate people on the reliability problem, which is surely going to lead to lots of problems in the wrong hands.
• aucisson_masque 2y ago
  Wow, that's really interesting! I didn't have the time to read all the pages, but I definitely will. It helps to bring one's expectations about AI back down to earth.
• qwGafv 2y ago
  That's an interesting Altman quote on the site. LLMs cannot be compared to electricity and the Internet. People wanted those. LLMs were an impressive parlor trick at first but disappointing later. Many stopped using them altogether. Now there is a president who fuels the hype, shakes down rich countries for "AI" investments. The Saudi prince who lost money on Twitter is in for the new grift and praises Musk on Tucker Carlson. The grift-oriented economy might continue with the bailout of Bitcoin whales through the "sovereign wealth fund" scheme. That is how the "economy" works. No houses will be built and nothing of value will be created.
  • tmnvdb 2y ago
    People have stopped using LLMs? I wasn't aware of that. Can you share a source for that?
    • TheOtherHobbes 2y ago
      I know a lot of people who went through the "Oh, wow - wait a minute..." cycle. Including me. They're approximately useful in some contexts. But those contexts are limited. And if there are facts or code involved, both require manual confirmation. They're ideal for bullshit jobs - low-stakes corporate makework, such as mediocre ad copy and generic reports that no one is ever going to read.
      • tmnvdb 2y ago
        > And if there are facts or code involved, both require manual confirmation. The hidden assumption here seems to be that the model needs to be perfect before it has utility.
        • TeMPOraL 2y ago
          Also hidden assumption, or perhaps lack of clear perception of reality, that most jobs on the market are strongly dependent on factual correctness. Also assumption that this is any different than human relationship with empirical truth is.
• dailykoder 2y ago
  God says... bradytrophic indical sacella unlittered surveying mis-tilled brachymetropic oxime masa hypermilitant litas acarotoxic thrust O retranslating awee tetraodont prevoidance cabbala Linker pervertedness absurdest coruscates aminopyrine spitting ledgers Fremontia upthrown simplifiers acomous statutable Ampelopsis
• FergusArgyll 2y ago
  The sys prompt given to run the turing test (from https://arxiv.org/pdf/2405.08007 https://arxiv.org/pdf/2405.08007) actually works well. I'm honestly not sure I'd be able to tell (unless I test it adversarially e.g ignore the prompt, write a poem etc)
  • parliament32 2y ago
    Curiously, this is the same point we were at in the 60s: https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA
• hirenj 2y ago
  This is a great resource, thanks. We (myself, a bioinformatician, and my co-cordinators, clinicians) are currently designing a course to hopefully arm medical students with the required basic knowledge they need to navigate the changing world of medicine in light of the ML and LLM advances. Our goal is to not only demystify medical ML, but also give them a sense of the possibilities with these technologies, and maybe illustrate pathways for adoption, in the safest way possible. Already in the process of putting this course together, it is scary how much stuff is being tried out right now, and is being treated like a magic box with correct answers.
  • sabas123 2y ago
    > currently designing a course to hopefully arm medical students with the required basic knowledge they need to navigate the changing world of medicine in light of the ML and LLM advances Could you share what you think would be some key basic points what they should learn? Personally I see this landscape changing so insanely much that I don't even know what to prepare for.
    • hirenj 2y ago
      Absolutely agree that this is a fast-moving area, so we're not aiming to teach them specific details for anything. Instead, our goals are to demystify the ML and AI approaches, so that the students understand that rather than being oracles, these technologies are the result of a process. We will explain the data landscape in medicine - what is available, good, bad and potentially useful, and then spend a lot of time going through examples of what people are doing right now, and what their experiences are. This includes things like ethics and data protection of patients. Hopefully that's enough for them to approach new technologies as they are presented to them, knowing enough to ask about how it was put together. In an ideal world, we will inspire the students to think about engaging with these developments and be part of the solution in making it safe and effective. This is the first time we're going to try running this course, so we'll find out very quickly if this is useful for students or not.
• TeMPOraL 2y ago
  "The LLMs have no ground truth" claim (around chapter 2) that's core to the "bullshit machines" argument is itself wrong. Of course LLMs have ground truth. What do the authors think here, that the text in training corpus is random? Hint: it isn't. Real conversations are anything but random. There's a lot of information hidden in "statistical ordering of the words", because the distribution is not arbitrary. Statistical ground truth isn't any worse than explicitly given one. In fact, fundamentally, there only ever is statistical certainty. Realizing it is a pretty profound insight, and you'd think it should be table stakes at least in STEM, but sonehow it fails to spread as much as it should.
  • ImPleadThe5th 2y ago
    So if I ask ChatGpt about bears and in the middle of explaining their diet it tells me something about how much they like porridge and in the middle of habitat it tells me they live in a quaint cabin in the woods, that's ... True? Statistically we certainly have a lot of words about 3 bears and their love for porridge. That doesn't mean it's true, it just means it's statistically significant. If I asked someone a scientific question about bears and they told me about Goldilocks, id think it was bullshit.
    • tmnvdb 2y ago
      Having a ground truth doesn't mean it does not make huge and glaring mistakes.
    • vaidhy 2y ago
      If that were the case, then you are right.. However, the current crop of LLMs seem to be good at understanding the context. A scientific data point about bears is unlikely to have Goldilocks in there (unless talking about evolution of life and Goldilocks zone). You can argue that there is meaning hidden in words that is not captured by words themselves in a given context - psychic knowledge as opposed to reasoned out knowledge. That is a philosophical debate.
      • TeMPOraL 2y ago
• ohthehugemanate 2y ago
  I just want to say: I've been publicly calling them "bullshit machines" since the first big media wave. I am incredibly pleased that this mental model helps other people, too. And that the specific term sees broader use is also nice. Also, neener neener neener I called them bullshit machines BEFORE it was cool. /humor Seriously though the humanities have a lot to chew on with LLMs, and are incredibly important to how we live and work with them. Who knew that epistemology would become front page news, and the sexiest topic for VC?
• abzolv 2y ago
  Your scroll-to-death user interface made me close the window before the end of the second page. Did you ask an LLM to recommend the most user-friendly UI to you?
  • ctbergstrom 2y ago
    We asked our target audience, 19-year-olds. They had a strong preference for this style. I know....
    • hennell 2y ago
      Aside from some of the long gaps between text I didn't think it was so bad. And I wholeheartedly approve of a process that checks the preferences of the target audience even if it's not what I (or they) would pick. However I can't imagine anyone tests well with the video content. The discussion on teachers using AI generated slides (lesson 2) was really interesting, but it had to fight my desire to stop that awful audio. Clearly the sound recording didn't go well and you have what you have, but at least edit it so the three talkers are at some sort of consistent volume. I was raising and lowering trying to make out what was said from one speaker then being deafened by the next. (To combat the poor sound, and make it more accessible, could be worth looking at adding subtitles. A fun opportunity to play with AI subtitling systems maybe ;) )
    • ptx 2y ago
      Maybe as a concession to us older folks you could make the pagedown key instantly flip to the next page (without changing the position of the current page relative to the viewport)? Then the site could be used like a PDF slide deck in "fit page" mode, which would be a lot better.
  • spudlyo 2y ago
    I also could not get past my outrage about this, how about a link to the content in a format suitable for the old and cranky? Even raw text would be better than this.
• gsky 2y ago
  People said the world wouldn't need more than 10 computers. Newyork times ridiculed space industry and etc.. AI is gonna disrupt all industries
  • _heimdall 2y ago
    Any new technology given this much attention and money will disrupt, that's not really a question. The question is whether we're going to be better off for it, and if people want all that change in the first place.
    • JKCalhoun 2y ago
      It could be argued that the adoption of the automobile was a bad move. You and I debating either its efficacy or social good though is irrelevant if it marches on regardless.
      • _heimdall 2y ago
        I didn't explain my point there, let me try with the automobile example. There's a key difference in the how the impact of the automobile happened - consumers got to choose to buy them and the impact was driven in large part by market demand. The fact that LLMs are going to have a big impact seems obvious because a comparatively few number of people are making a huge deal out of them, both with attention and money. LLMs will be big but that says nothing of their usefulness or even consumer demand, it says more about how the industry is being financed.
        • JKCalhoun 2y ago
          I'm not too worried about Big Corporation trying to push a rope. In the end it really is only going to succeed if "we" want it — find value in it.
          • _heimdall 2y ago
            > only going to succeed if "we" want it — find value in it. I'll be pleasantly surprised if that's how it turns out. At least so far market dynamics haven't really been much of a driver for LLMs. Those with the money think its the next big thing and are pouring cash both into the LLMs themselves and any product that slaps a "powered by AI" sticker on the box. That's not to say people aren't also actively choosing to use LLMs, but in my opinion the market demand doesn't account for the massive amount of hype and funding, or the pervasiveness of LLMs being added to so many products.
• dzogchen 2y ago
  This will not age well. Yesterday my bullshit machine wrote a linker argument parser to hook a C++ library up in a Rust build config. Oh it also wrote tests for it. https://chatgpt.com/share/67a89e5f-b5b4-8011-9782-472d469cc259 https://chatgpt.com/share/67a89e5f-b5b4-8011-9782-472d469cc2...
  • tmnvdb 2y ago
    [flagged]
    • bayindirh 2y ago
      Allowing a parrot to iterate on given examples and generate a similar one with the information baked in their weights does not invalidate "Stochastic Parrot" take. On the contrary, it proves it. LLMs are statistical machines. The catch is you feed it hundreds of terabytes of valid information, so it asymptotically generates valid information as a result of this statistical bias. Even yet, they can hallucinate so badly. I mean, the same OpenAI model claimed that I'm a footballer, a goal keeper in fact. Stochastic parrot, yes. On LSD, very yes.
      • tmnvdb 2y ago
        It's clearly true that the LLMS are 'stochastic parrots', but for all we know that might be the key to intelligence. It is in itself not a deep observation any more than calling your fellow humans 'microbial meatbags'. Saying that LLMs are stochastic machines does not establish an upper bound for success.
        • fatata123 2y ago
          We are not stochastic parrots. Old components of our brain help “ground” our thoughts and allow things like doubt or a gut feeling to develop which means we can question ourselves in ways an LLM cannot.
        • bayindirh 2y ago
          The thing is, this assumption of LLMs might be intelligent lies in the assumption is intelligence is enabled solely by the brain. However, as the science improves, we understand more and more that brain is just part of a much bigger network, and its size or surface roughness might not be the only thing determines the level of intelligence. Also, all living things have processes which allows constant input from their surroundings and they also have closed feedback loops which constantly change and tweak things. Call these hormones, emotions or self-reflection or whatnot. We the scientists love to play god with the information we have at hand, yet we constantly humbled by the nature by experiencing the shallowness of what we know. Because of that I, as a CS Ph.D., am not so keen on to jump to that bandwagon which claims that we invented silicon brains. They are arguably useful automatons built on dubious data obtained in ethical gray areas. We're just starting to see what we did, and we have a long way to go. So, a living parrot might be more intelligent than these stochastic parrots. I'll stay on the cautious critics wagon for now.
• d4rkp4ttern 2y ago
  The format is very interesting. Can you speak to the tech stack behind how you made it ?
  • magicalhippo 2y ago
    I'll agree on interesting, however I found it very difficult to follow. Reading the landing page, I expected a link to get started. Took me some time to really register the links at the very bottom. After finding my way into the lectures I got distracted by all the scrolling. Fortunately Reader Mode fixed that. However a few lectures in and I notice there's several videos I've missed... However since I'm starting to approach the get-off-my-lawn age, I guess I'm not the target audience.
  • ctbergstrom 2y ago
    After talking to an awful lot of 18-20 year olds (our target audience) we decided we wanted to go with a "scrollytellying" style. I'm not a designer and I've done worked in that style before. After looking into a range of platforms — Vev and Closeread for Quarto deserve particular mention — I felt that Shorthand (https://shorthand.com/ https://shorthand.com/) was the best option for rapid development given my lack of experience in this whole process. In general I've been very pleased. You don't have the fine scale control you do on a platform like Vev, but for someone like me that is probably a good thing because it keeps me from mucking around quite as much as I otherwise would with design decisions that I don't really understand. The price is a bit steep for a self-funded operation and we're constrained a bit by the need to use their starter tier, but I feel like we are definitely getting our money's worth and customer support from Shorthand has been exemplary.
• fancyfredbot 2y ago
  I have just read one section of this, "The AI scientist'. It was fantastic. They don't fall into the trap of unfalsifiable arguments about parrots. Instead they have pointed out positive uses of AI in science, examples which are obviously harmful, and examples which are simply a waste of time. Refreshingly objective and more than I expected from what I saw as an inflammatory title.
• jcgrillo 2y ago
  The cost of inference seems like a major barrier in making these things work commercially. If you put one in the user facing flow you'll need an extraordinary amount of compute that scales extremely poorly in number of users, given that each query takes O(nm^2) where m is the number of model weights (many) and n is the number of tokens in the query. So it seems clear if each user session implies multiple LLM queries, those user sessions had better be exceptionally valuable. This seems like a completely different business with very different constraints and margins. The problem is it doesn't seem like the value added is all that great. So how do you justify the ruinous cost? I would like to know more about how people actually intend on using these things profitably than to hear about how they're going to wake up and become "intelligent". Does anyone have any success stories actually using this thing? It's hard to find any discussion of this amidst all the bullshit hype, and it's really the only question--can you make money with this or not? And I don't mean raising billions in venture capital, I mean can you integrate these things profitably and sustainably into a website?
  • tmnvdb 2y ago
    The costs are a problem. We don't have hard evidence that this will be solved, but with algorithmic efficiency and raw compute costs both changing rapidly, the cost per token has gone down by about a factor of 10 per year for the last 3 years, i.e. 1000x over 3 years.
    • jcgrillo 2y ago
      As far as I can tell, those charts are merely describing the price per token that LLM hosting companies are charging, not what running the model actually costs. The distinction is important for two reasons: 1. These companies are heavily subsidized by huge amounts of venture investment 2. If I'm integrating this technology into my web product there's absolutely no way I'll be adding a 3rd party company as a dependency. This is all way too new and bubbly to trust any of the current offerings will still exist in O(years). Are there any similar studies showing not sticker price but actual compute/performance decrease?
      • 2y ago
• roland35 2y ago
  I like the term "bullshit" over "hallucination". These AI machines have no perceptions, no concept of truth, so indeed they are just spewing out words with no regard for truth. And unfortunately the cost of spreading bullshit has gone down to almost 0.
  • uh_uh 2y ago
    So why are they SOTA translators? Would you consider old translation software bullshit generators? Because LLMs can do their job, and more.
    • krupan 2y ago
      Those who sell these things as almighty powerful AI, so powerful and dangerous that it should be regulated (so my competition can't keep up with me), are responsible for this overreaction. Yes, not everything LLMs produce is bullshit but enough is, contrary to how they were marketed, that people are reacting this way. In other words, you don't counter hyperbolic and downright false marketing with subtleties.
  • tim333 2y ago
    I dislike the term bullshit because it's use regarding ChatGPT does not match the dictionary version "stupid or untrue talk or writing; nonsense". If your sentence "LLMs output is bullshit" is wrong you may be better off changing the sentence than rewriting the dictionary to fit your sentence. I mean you can redefine words if you like, like how young people use sick and bad to mean much the opposite of what they did, which is fine as a fashion statement, but in trying to reason about LLMs it muddies the reasoning. Which of course is often why academics do it - see Hobbes, Calvin 1993 https://www.reddit.com/r/calvinandhobbes/comments/1300k80/academia_here_i_come/ https://www.reddit.com/r/calvinandhobbes/comments/1300k80/ac...
    • somebehemoth 2y ago
      I think a hallucinated sentence embedded in a paragraph of truth fits the definition: stupid and nonsense. A bullshitter can be right sometimes or even most of the time. They are still a bullshitter.
• cobertos 2y ago
  Read the whole course. There's a great amount of sourced case studies of people using AI in here with societal pushback which I find interesting to pull from. Disappointed though that the answer is so nuanced. There aren't hard and fast rules to when/when not to use AI but a set of 18 or so proposed principles that should guide are usage. And defense for those principles. The principles are at the bottom of each chapter. Also learned about the Eliza Effect as a term and that I found the passage in Ch14 by Ted Chiang to be really insightful, from a general social perspective. > When someone says “I’m sorry” to you, it doesn’t matter that other people have said sorry in the past; it doesn’t matter that “I’m sorry” is a string of text that is statistically unremarkable. If someone is being sincere, their apology is valuable and meaningful, even though apologies have previously been uttered.
• Johanx64 2y ago
  Somebody made a website to express their opinion - wherein their opinion can be surmised by reading the domain name. Text is scaled to 300% to indicate just how important and authoritative they think their opinion is. And it talks down to you in a "here comes the expert" style, with an atrocious aimed-at-preschoolers presentation. No thank you.
  • FrustratedMonky 2y ago
    That's too harsh. Some people do like big bullet list of points. Some people need to be spoon fed. Don't blame the spoon for being a spoon.
  • myaccountonhn 2y ago
    Two university professors in data science and computational biology are not just “somebody”.
    • Johanx64 2y ago
      People that find professors and similar in high esteem usually haven't spent enough time in or around academia and academics and for that reason still maintain some innocent aura of mystique and prestige around it in their mind. Quite literally anyone with a bit of persistence could become a professor (and this by far isn't even a top university either). They are quite literally just a somebody with an opinion just like anybody else. An opinion barked down to you with 300% scaled fonts and preschooler illustrations.
      • myaccountonhn 2y ago
        I have been around a lot of academics and been in academia. While we shouldn't trust academics just because they are academics, these people specialize in relevant fields and also back their claims with citations throughout the course. They are not just postulating.
    • sabas123 2y ago
      Being a "University Professor" means jack shit unless precisely in their (sub)-field. The authors are experts in biology, and evolution of information representation/communication, and about misinformation. I'll gladly defer to their expert opinion on those topics, but IMO to use such an authoritative voice when they are not experts in actual AI systems. Judging the massive progress in the field of AI, how can anyone even remotely state what these systems inherently are, when they are still so new and ever-evolving?
• EncomLab 2y ago
  Too little is made of the distinction between silicon substrate, fixed threshold, voltage moderated brittle networks of solid-state switches and protein substrate, variable threshold, chemically moderated plastic networks of biological switches. To be clear, neither possesses any magical "woo" outside of physics that gives one or the other some secret magical properties - but these are not arbitrary meaningless distinctions in the way they are often discussed.
• 6stringmerc 2y ago
  Very much looking forward to reviewing the material - so relevant, especially coming from the English language pedagogy side of things. My biggest concern is about the narrowing of language in order to basically eliminate "edge cases" in the name of streamlining efficiency. As in, if the "AI" does not understand the request, it will say so, and that's the end of the inquiry. What does not fit within the AI parameters will simply be, well, ignored. The English language is so malleable (as I recognized highly in jail with "On Grip!" being the new hotness) that limiting its ability is a completely do-able intentional or unintentional "societal management" outcome. Fortunately as a musician, LLMs produce absolutely shit music and if they ever make something decent, they will be sued into oblivion for training it using copyrighted material.
• shaggie76 2y ago
  I was thinking how this article claims that people crave the authenticity of live music and that bullshit-generators will never be able to supply that. At first, I saw this as a reason for optimism, but then I got to thinking about evidence that people may not necessarily want authenticity after all. Organic produce was the first metaphor the came to mind: it's probably more healthy for you even if it isn't as pretty, but many people aren't willing to pay a premium it and I suspect economics isn't often the reason. Is that a straw-man for live music? I don't know that it is because plenty of people are content to listen to recorded music -- sure, they might enjoy going to a live concert but they'll still listen to the radio on the drive to work. Then I got to thinking about something more crass: while breast implants and other cosmetic body surgery may be as much for the benefit of the subject self-image I imagine there are plenty of people that find it very attractive despite what is often obviously fake. So do we crave authenticity? I think I do but I'm not sure if that's a safe generalization to make.
  • bsenftner 2y ago
    Of course we do, but also realize that the threshold for "being authentic" is flimsy at best for many, and an Everest to climb for others. We want more skeptics in generalized society with their own personal Everest they require of their thought leaders. This variability of acceptance for integrity is a weakness in our civilization, strategically grown by attacking educational institutions, and currently being exploited to great success by Orwellian long players.
  • tmnvdb 2y ago
    Craving "authenticity" is somewhere very high up the hierarchy of needs, i.e. a luxury. That a lot of people do not care much for it is not a sign of moral failing but of having bigger fish to fry.
    • woodruffw 2y ago
      I think it'd fall somewhere close to a "social" need, i.e. right smack in the middle of Maslow's hierarchy.
  • 2y ago
• patja 2y ago
  I'm uncomfortable with the use of profanity as a core element of this campaign branding, especially given that it seems to be an educational outreach effort. While this seems targeted at college age and above, I think it would be highly relevant content for a teen audience as well. While I swear in privacy on occasion I think it has no place in the classroom. I really don't care for it in the workplace either but I grant that private enterprises can have their own culture. It is frustrating because I agree with most of the content and the need for informed debate on the topic. It is a bit like my reaction to reading Cory Doctorow: I agree with his politics but really dislike the hamfisted way he packages his advocacy in the form of action adventures. As if the merits of his arguments need to be packaged in cotton candy to be consumed, and there is an undercurrent of self-promotion and personal branding that feels suss. Probably all a "me" problem with associations built up over time from seeing snake oil being packaged using a similar playbook. If you have to sell your message by dressing it up with scroll effects and provocative offensive language you've already lost me.
  • woodruffw 2y ago
    I assume that the use of the word "bullshit" on this site is at least in part informed by "On Bullshit,"[1] which is a pretty common undergraduate reading. [1]: https://en.wikipedia.org/wiki/On_Bullshit https://en.wikipedia.org/wiki/On_Bullshit
  • deleted 2y ago
    [deleted]
  • ctbergstrom 2y ago
    This is something we've given serious consideration, having taught a course called "Calling Bullshit" (http://callingbullshit.org http://callingbullshit.org) for almost a decade and having authored a book by the same name that gets downranked on various Amazon features because of its title. But the bullshit is a term of art here, after the seminal 1986 article "On Bullshit" by Princeton philosopher Harry Frankfurt (later published as a little book). We strongly feel that it is exactly the right term for what LLMs are doing, and we make the case for that in lesson 2 of the course. (https://thebullshitmachines.com/lesson-2-the-nature-of-bullshit/index.html https://thebullshitmachines.com/lesson-2-the-nature-of-bulls...) We're also concerned about accessibility for high school teachers etc., and thinking about what to do in that direction. I'm curious: do you find "bs" to be any less offensive?
• nmca 2y ago
  (while I work at OAI, the opinion below is strictly my own) I feel like the current version is fairly hazardous to students and might leave them worse off. If I offer help to nontechnical friends, I focus on: - look at rate of change, not current point - reliability substantially lags possibility, by maybe two years. - adversarial settings remain largely unsolved if you get enough shots, trends there are unclear - ignore the parrot people, they have an appalling track record prediction-wise - autocorrect argument is typically (massively) overstated because RL exists - doomers are probably wrong but those who belittle their claims typically understand less than the doomers do
  • dimgl 2y ago
    What are “parrot people”? And what do you mean by “doomers are probably wrong?”
    • moozilla 2y ago
      OP is likely referring to people who call LLMs "stochastic parrots" (https://en.wikipedia.org/wiki/Stochastic_parrot https://en.wikipedia.org/wiki/Stochastic_parrot), and by "doomers" (not boomers) they likely mean AI safetyists like Eliezer Yudkowsky or Pause AI (https://pauseai.info/ https://pauseai.info/).
  • layoric 2y ago
    How does this help the students with their use of these tools in the now, to not be left worse off? Most of the points you list seem like defending against criticism rather than helping address the harm.
    • habinero 2y ago
      Agree. It's also a virtue to point out the emperor has no clothes and the tailor peddling them is a bullshit artist. This is no different than the crypto people who insisted the blockchain would soon be revolutionary and used for everything, when in reality the only real use case for a blockchain is cryptocoins, and the only real use case for cryptocoins is crime. The only really good use case for LLMs is spam, because it's the only use case for generating a lot of human-like speech without meaning.
• megaloblasto 2y ago
  > After talking to literally hundreds of educators, employers, researchers, and policymakers, we have spent the last eight months developing the course on large language models (LLMs) that we think every college freshman needs to take. Did you consider consulting with any college freshman or even college students? I know you're supposed to be guiding their education, but I think it's also good to check in and see what they care about learning.
  • ctbergstrom 2y ago
    Yes—I should have stressed that. More than anything, we've talked at great lengths about LLMs with over a thousand undergraduate students who we have taught in our courses since ChapGPT 3.5 launched in Nov 22.
• aucisson_masque 2y ago
  > LLMs are not capable of reflecting on and reporting about how or why they do what they do. i get the why, but about the how: deepseek has shown it's able to explain it's reasoning, how it reach a conclusion. That's quite a far stretch from the llm being only able to estimate statistically what's the right word to put after another one.
  • CamperBob2 2y ago
    Their arguments are just one special-pleading fallacy after another. "But... but... but... it's different when we do it!" No, it's not different when we do it. The takeaway here isn't that the AI algorithms are so special and magical, it's that our brains are not. It takes some nerve (literally) for humans to throw around labels like "bullshit machine." The only advantage we really have is long-term memory. I'm sure that will be addressed soon enough. Someone will figure out the ANN analogue of memory consolidation during sleep, and that'll be the tipping point.
    • daveguy 2y ago
      > The only advantage we really have is long-term memory. We also have short term memory that is associative to our long term memory. So, STM is about 7 items. But those 7 items are also pointers to LTM. Oh, and the ability to learn new things.
      • CamperBob2 2y ago
        I think of the context as short-term memory. If there were a way to update the weights based on the context, such that the outcomes of future queries would be influenced, things would get interesting in a hurry.
• ndstephens 2y ago
  Really enjoying this. Thank you for the great work. I'm currently on Lesson 11 and noticed a couple typos (missing words). I haven't found anywhere on the site itself where I could send feedback to report such a thing (maybe I missed it). Hopefully you aren't offended if I post them here. I think the easiest way to point them out is to just have you search for the partial line of text while on Lesson 11 and you'll see the spots. "No one is going to motivated by a robotic..." (missing the word "be") "People who are given a possible solution to a problem tend to less creative at..." (again missing the word "be")
  • ctbergstrom 2y ago
    Thank you very much — fixed!
• layer8 2y ago
  Background: https://thebullshitmachines.com/about-us/ https://thebullshitmachines.com/about-us/
• foobiekr 2y ago
  It is _very_ unfortunate there is no PDF version.
• timewizard 2y ago
  It's not a "ChatGPT world." You can thrive just fine by entirely ignoring it and all the snake oil vendors living in it. 4 years and all they have to show for it is absurdly powerful video cards, sub 90% accuracy where it matters, and the only application is "chat bot." It's a fad. Wake up everyone.
  • SpicyLemonZest 2y ago
    You can entirely ignore it only in the sense that sticking your head in the sand is an option. A small but growing fraction of the text you read and images you see were generated by an LLM, and since 2023 at the latest they've been good enough that you cannot reliably tell which fraction it is.
• BugsJustFindMe 2y ago
  > Large language models are both powerful tools, and mindless—even dangerous—bullshit machines. Yes, but so is the average person. Do you cover the similarities/parallels to common human behavior patterns in your course? It stands out to me as a major blind spot in a lot of the discourse about LLMs, down to willing acceptance of humans intuiting what someone else meant when the words themselves were fundamentally ambiguous, which is extremely akin to a hallucination chosen from the listener's language model.
  • justonceokay 2y ago
    Yes but people have culpability and responsibility. In places of power or influence, this culpability can lead to being fired, legal action, disbarment, loss of money, etc. so there is a real pressure to be coherent and aligned with reality.
    • BugsJustFindMe 2y ago
      You say this but I have a ton of experience that "can lead" and "there is" come with a gigantic pile of caveats to the extent that they appear to be more false than true. Or they're technically true but practically meaningless. The world has been utterly awash in mass-perpetuated misinformation for at least all of recorded history without any real ability to stem the onslaught. This is not a modern problem just because a modern technology also exhibits it. You should look up the percentage of Americans who believe in ghosts sometime. About as many people believe in ghosts as don't, so no matter which side you land on, the other side is enormous. One of the sides must be wrong, I won't claim which, though only the belief side fails a falsifiability check. Where's our accountability to believing and spreading ideas based in reality again? The believers believe because they learned about it from someone. It didn't happen spontaneously on its own. It's all just been memes the entire time.
      • evklein 2y ago
        This is a sidestep imo. People _can_ be held accountable, though they will not always be. Machines add a layer of complexity - money is lost or a life is lost because AI made the call, who bears the burden? Machines _can't_ be held accountable.
• dczx 2y ago
  I'm offended.
• idunnoman1222 2y ago
  My 80s copy of the encyclopedia Britannica was riddled with errors, perhaps we will survive this post truth
  • therein 2y ago
    The situation is closer to if we had 10,000 variants of Encyclopedia Britannica in 80s that all looked like distinct bodies of work, riddled with different errors while looking like they were written from scratch.
    • idunnoman1222 2y ago
      What is the difference to the end user? Our situation is better than it was yesterday not worse. If we had a genie who could appear out of nowhere and tell us the truth TM at any point that would kind of ruin the adventure . Who reads the output of a book or Wikipedia or a website or an AI and thinks oh good now I know the core truth of this thing and I never need to update this knowledge ever again case closed
• jvanderbot 2y ago
  > We (the authors of this website) have at times sought insight into the inner workings of an LLM by asking it “why did you just do that?” > But the LLM can’t tell us. It’s not a person. It doesn’t have the metacognitive abilities necessary to reflect on its past actions and report the motivations underlying them*. > With no clue why it did whatever it just did, the LLM is forced to guess wildly at a plausible explanation, like the ill-fated Leonard Shelby in Christopher Nolan's film Memento. > And we, gullible humans that we are, often believe its bullshit. --- I am almost convinced that we ourselves are a narrator riding along inside an animal's mind, trying desperately to put together explanations for our actions, mostly just to convince others, just as though an LLM were running on our own senses trying to portray some deep semblance of consciousness. I don't think we'll find a super smart AI, we'll just realize we were not very sophisticated all along. The power of speech for information and culture transfer, writing, inspiration, and coordination is just awe inspiring, evolutionary speaking, so once we could talk we had to, because the "better" talker almost always won. It's an arms race.
  • ysofunny 2y ago
    I am completely convinced that what we call consciousness is as you say. this means that it really exists in retrospect (20ms ? i recall some neuroscience articles from the 00s). nonetheless its whole reason for existing (retrospectively) is planning the future
  • I-M-S 2y ago
    Related, since the advent of LLMs I've become acutely aware how any argument of a considerable length with another person quickly starts meandering and how topics change seemingly of no one's volition - almost as if our own internal token limit has been exceeded.
    • eqqn 2y ago
      Tangentially related, I became aware how my (in)ability to reason 3 intertwined different programming languages across different files can be conveniently called as my own "context window". (the example here is HTML/CSS/JS where LLM greatly exceeds my own capacity).
• paulcole 2y ago
  > Others say they are nothing but bullshit machines. Simply ignore anyone who says this and go about your business. Doubly so if they bring up the environmental impact of AI.
• akomtu 2y ago
  "How to thrive in a machine world?" should be the rhetorical question.
• deleted 2y ago
  [deleted]
• codpiece 2y ago
  I am nearly 60, and am excited to take your course! I congratulate you in finding a compelling way to teach Humanities that is relevant to today's society. Bravo.
• aaplok 2y ago
  Really well done. It is really a challenge for students to navigate their way around the AI landscape. I am definitely considering sharing that with my students. Have you noticed a difference in how your students approach LLMs after taking your course? A possible issue I see is that it is preaching to the choir; a student who is enclined to use LLMs for everything is less likely to engage with the material in the first place. If you allow feedback, I was interested in lesson 10 on writing, as an educator who tries to teach my science/IT/maths students the importance of being able to communicate. I would suggest to include a paragraph to explain why being able to write without LLMs is just as important in scientific disciplines, where precision and accuracy are more essential than creativity and personalisation.
  • ctbergstrom 2y ago
    This is an excellent point about scientific writing. We'll add something to that effect. We have not taught this course from the web-based materials yet, but it distills much of the two-week unit that we covered in our "Calling Bullshit" course this past autumn. We find that our students are generally very interested to better understand the LLMs that they are using — and almost every one of them does, to vary degree. (Of course there may be some selection bias in that the 180 students who sign up to take a course on data reasoning may be more curious and more skeptical than the average.)
• pjs_ 2y ago
  Not sure why everyone rates this. It’s full of very confidently made statements like “the AI has no ground truth” (obviously it does, it has ingested every paper ever), it “can’t reason logically” which seems like a stretch if you ever read the CoT of a frontier reasoning model and “can’t explain how they arrived at conclusions” where - I mean just try it yourself with o1, go as deep as you like asking how it arrived at a conclusion and see if a human can do any better. In fact the most annoying thing about this article is that it is a string of very confidently made, black and white statements, offered with no supporting evidence, and some of which I think are actually wrong… i.e. it suffers from the same kind of unsubstantiated self confidence that we complain about with the weaker models
  • bwfan123 2y ago
    the machine is fooling you with a mimicry of reasoning. and you are falling for it.
    • ZephyrBlu 2y ago
      What is reasoning if not a chain of logically consistent thoughts?
      • bwfan123 2y ago
        fair, but "logically consistent thoughts" is a subject of deep investigation starting from the early euclidean geometry to the modern godel's theorems. ie, that logically consistent thinking starts from symbolization, axioms, proof procedures, world models. otherwise, you end up with persuasive words.
        • ZephyrBlu 2y ago
          You just ruled out 99% of humans from having reasoning capabilities. The beautiful thing about reasoning models is that there is no need to overcomplicate it with all the things you've mentioned, you can literally read the model's reasoning and decide for yourself if it's bullshit or not.
• bwfan123 2y ago
  Fantastic course and website, thank you professors for this valuable contribution. I would ask for exercises and practices attached to the course that one could do to penetrate the BS - this would be invaluable as we increasingly get bombarded by ai-generated media. Indeed what is learning and education except critical thinking skills ? An un-intended consequence of LLMs is that they have triggered this conversation.
• svilen_dobrev 2y ago
  very well done. Thank you. btw, typo in lesson 11: "(2) understand how an LMM can help them" .. instead of LLM IMO, "understanding" is the notion most endangered and the one which loss is with most catastrophic consequences, from all the possibly affected ones. It has never been very favored, but now is even worse than ever - it is thinned and abandoned en-masse. Check brothers Strugatsky's Snail_on_the_Slope [0] , there's very stringent monologue of Peretz on the topic, ~~ page 11 [1] [0] https://en.wikipedia.org/wiki/Snail_on_the_Slope https://en.wikipedia.org/wiki/Snail_on_the_Slope [1] https://strugacki.ru/book_19/768.html https://strugacki.ru/book_19/768.html
• zaptheimpaler 2y ago
  I like it. It's pretty basic but it is very good for a broad audience and covered things many people don't understand. I liked that you mentioned not to anthropomorphize the model. We would greatly benefit from 50+ year old policymakers and more taking the course even more than 19 year old freshmen.
• dzonga 2y ago
  thank you for presenting this in digestible format. fortunately unfortunately, people who know llm's shill bullshit are the ones selling llm's while they wouldn't feed llm's to their kids or eat llm's.
• raincom 2y ago
  Since you are on Frankfurtian bullshit, why don't you consider Late G.A. Cohen's take on intellectual bullshit (bullshit perpetuated in the academia), as the latter's notion of bullshit is linked to knowledge, unlike the bullshit we hear from salesmen, showmen, etc.
  • ctbergstrom 2y ago
    We've discussed Cohen's work in our book _Calling Bullshit_, but the type of bullshit the Cohen focuses on — unclarifiable unclarity, particularly in academic writing — is not what LLMs produce so it strikes us as far less relevant to this course than Frankfurt's notion.
• fergal_reid 2y ago
  I think the authors misunderstand what's actually going on. I think this is the crux: >They are vastly more powerful than what you get on an iPhone, but the principle is similar. This analogy is bad. It is true that the _training objective_ of LLMs during pretraining might be next token prediction, but that doesn't mean that 'your phone's autocomplete' is a good analogy, because systems can develop far beyond what their training objective might suggest. Literally humans, optimized to spread their genes, have developed much higher level faculties than you might naively guess from the simplicity of the optimisation objective. If the behavior of top LLMs didn't convince you of this, they clearly develop much more powerful internal representations than an autocomplete does, are much more capable etc. I would point to papers like Othello-gpt, or lines of work on mechanistic interpretability, by Anthropic, and others, as very compelling evidence. I think that, contrary to the authors, using words like 'understand' and 'think' for these systems is much more helpful than to conceptualise them as autocomplete. The irony is that many people are autocompleting from the training objective to the limits of the system; or from generally being right by calling BS on AI, to concluding it's right to call BS here.
• amelius 2y ago
  What is scientific about this? Saying that LLMs are just guessing the next tokens therefore they are parrots, that doesn't bring anything to the table. You might as well say that humans guess the next key they type on their keyboards. Both are probably true, at some level, but you don't gain any insight from it. You also didn't say anything about the probability distribution of the guesswork, so you didn't say if the guessing was smart guessing or dumb guessing. To be honest, I think this view is just a way of sticking your head in the sand about the upcoming technological revolution.
• s2radhak 2y ago
  Fascinating. The article repeatedly makes the claim that “LLMs work by predicting likely next words in a string of text”. Yet there’s the seemingly contradictory implication that we don’t know how LLMs work (ie we don’t know their secret sauce). How does one reconcile this? They’re either fancy autocompletes, or magic autocompletes (in which case the magic qualifier seems more important in understanding what they are than the autocomplete part).
  • habinero 2y ago
    We...do know how they work?
    • Workaccount2 2y ago
      We know how they work in that we built the framework, we don't know how they work in that we cannot decode what is "grown" on that framework during training. If we completely knew how they worked we could go inside an explain exactly why every token generated was generated. Right now that is not possible to do, as the paths the tokens take through the layers tend to be outright nonsensical when observed.
    • abecedarius 2y ago
      We know how they're trained. We know the architecture in broad strokes (amounting to a few bits out of billions, albeit important bits). Some researchers try to understand the workings and have very very far to go.
  • Terr_ 2y ago
    This occurs because of ambiguous language which conflates the LLM algorithm with the training-data and the derived weights. The mysterious part involves whatever patterns might naturally exist within bazillions of human documents, and what partial/compressed patterns might exist within the weights the LLM generates (on training) and then later uses. Analogy: We built a probe that travels to an alien planet, mines out crystal deposits, and projects light through those fragments to show unexpected pictures of the planet's past. We know exactly how our part of the machine works, and we know the chemical composition of the crystals, but...
• llm_trw 2y ago
  I thought I'd give this the benefit of the doubt. It's trying its hardest to be put in the "this is bullshit" pile. Five lines of content spread through five pages through the magic of parallax scrolling. Examples wandering around without going anywhere and confidently repeating talking points that were debunked 3 years ago. Please release a textbook that can be read instead of whatever this is.
• deleted 2y ago
  [deleted]
• blobbers 2y ago
  Haha, just read this title and I can’t help but agree that this is necessary because… they are bullshit machines. They’re just better at coding than most bullshitters.
• deleted 2y ago
  [deleted]
• dyauspitr 2y ago
  I’m all for courses like this but they become outdated very quickly. This area is moving fast
• pama 2y ago
  I wonder if the authors can explain the aparent inconsistency between what we now know about R1 and their statement “They don’t engage in logical reasoning” from the first lesson. My simple-minded view of logical reasoning by LLMs is that the hard question (say a math puzzle) has a verifiable answer that is hard to produce and is easy to verify, yet within the realm of knowledge of humans or the LLM itself, so the “thought” stream allows the LLM to increase its confidence by a self-discovered process that resembles human reasoning, before starting to write the answer stream. Much of the thought process that these LLMs use looks like conventional reasoning and logic, or more generally higher level algorithms to gain confidence in an answer, and other parts are not possible for humans to understand (yet?) despite the best efforts by DeepSeek. When combined with tools for the boring parts, these “reasoning” approaches can start to resemble human research processes as per the Deep Research by OpenAI.
  • CJefferson 2y ago
    As an example, I have asked tools like deepseek to solve fairly simple Sudoku puzzles, and while they output a bunch of stuff that looks like logical reasoning, no system has yet produced a correct answer. When solving combinatorics puzzles, deepseek will again produce stuff that looks convincing, but often makes incorrect logical steps and ends up with wrong answers.
    • pama 2y ago
      Here is o3-mini on a simple sudoku. In general the puzzle can be hard to explore combinatorially even with modern SAT solvers, so I picked one marked as “easy”. It looks to me like it solved it but I didnt confirm beyond a quick visual inspection. https://chatgpt.com/share/67aa1bcc-eb44-8007-807f-0a49900ad6bb https://chatgpt.com/share/67aa1bcc-eb44-8007-807f-0a49900ad6...
      • hennell 2y ago
        And thus we have the AI problems in a nutshell. You think it can reason because it can describe the process in well written language. Anyone who can state the below reasoning clearly "understands" the problem: > For example, in the top‐left 3×3 block (rows 1–3, columns 1–3) the givens are 7, 5, 9, 3, and 4 so the missing digits {1,2,6,8} must appear in the three blank cells. (Later, other intersections force, say, one cell to be 1 or 6, etc.) It's good logic. Clearly it "knows" if it can break the problem down like this. Of course if we stretch ourselves slightly to actually check beyond a quick visual inspection you'd quickly see it actually put a second 4 in that first box despite "knowing" it shouldn't. In fact several of the boxes have duplicate numbers, despite the clear reasoning aboving. Does the reasoning just not get used in the solving part? Or maybe a machine built to regurgitate plausible text, can also regurgitate plausible reasoning?
• s2radhak 2y ago
  “ When we write, we share the way that we think. When we read, we get a glimpse of another mind. But when an LLM is the author, there is no mind there for a reader to glimpse.” — I dunno, I feel like reading is more a glimpse into how I think than how the author thinks…a generated story can be just as moving as one from a human, I think.
• quacked 2y ago
  There seems to be a lot of coping in the anti-AI department about the utility of the current LLMs. "I'm not impressed". You're not impressed that 98% of knowledge workers can 10-100x their own work output for free? As a very intelligent infrastructure engineer told me: AI isn't going to take your job, but someone using AI is.
  • deleted 2y ago
    [deleted]
• theincredulousk 2y ago
  reductionist view can be applied to what we call "thinking" and "intelligence" too. When I'm asked a question, my brain is also just picking a suitable sequence of words based on my experience (training). Talking is what we consider part of thinking, something only "intelligent" creatures can do. Just feels like a lot of coping from people that don't want to let go of our concept of "intelligence superiority" or w/e you want to call it. The end game of this will be them wild-eyed in front of a string-crossed cork board, claiming they've found the one thing human brains can do that AI can't, so it's not thinking it's just x,y,z.
• owenversteeg 2y ago
  No comments on the course, but the title (in classic HN fashion) - how I survive in a ChatGPT world: it’s simple. I consume as little recent content as possible and I aim for the bulk of that to be “raw” information: various forms of data and the minimum set of news from wire services. Every time I dare stick my head out further I drown in a deluge of generated sewage. Blogs are dead, social media is dead, forums are dead, the media is dead, text is dead, video is dead and photos are dead. The only unpoisoned wells left in the land are old books and the elderly.
• iamnotsure 2y ago
  a picture is worth a thousand cords
• azakai 2y ago
  Overall this is very good, but I have one specific note: Lesson 6 says "LLMs aren't conscious." I think I get what you're saying there - they are not conscious in the same way that humans are - but "consciousness" is a highly-debated term without a precise definition, and correspondingly philosophers have no consensus on whether machines in general are capable of it. Here is one of my favorite resources on that, the Stanford Encyclopedia of Philosophy's page on the Chinese Room Argument: https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/ Things that appear conscious, or that appear to understand a language, are very hard to distinguish from things that actually are those respective things. Again, I think I get the intended point - some people interact with ChatGPT and "feel" there is another person on the other side, someone that experiences the world like them. There isn't. That is good to point out. But that doesn't mean machines in general and LLMs specifically can't be conscious in some other manner, just like insects aren't conscious like us, but might be in their own way. Overall I think the general claim "LLMs aren't conscious" is debatable on a philosophical level, so I'd suggest either defining things more concretely or leaving it out.
  • planb 2y ago
    Philosophy aside - how can an LLM be conscious without a memory or manifestation in the real world? It is a function that, given an input, returns an output and stops existing afterwards. You wouldn't argue that f(x)=x^2 is conscious? I would maybe accept debates about whether for example ChatGPT (the whole system that stores old conversations and sends the history along with the current user entry) is conscious - but just the model? Isn't that like saying the human brain (just the organ lying on a table) is conscious?
    • Borealid 2y ago
      There's a great exploration of this concept in Permutation City, a science fiction novel by Greg Egan. In the book, a deterministic human brain is simulated (perfectly) in random-access order. This thought experiment addresses all three of your arguments. I don't see why something that doesn't exist some of the time inherently couldn't be conscious. Saying that something's output is a function of its inputs also doesn't seem to preclude consciousness. Some humans don't have persistent memory, and all humans (so far) don't exist for 99.99999999% of history. I'm not trying to claim a particular definition of consciousness, but I find the counterarguments you're presenting uncompelling.
• misterflibble 2y ago
  Thank you @ctbergstrom for this valuable and most importantly, objective, course. I'm bookmarking this and sharing it with everyone.
• eqqn 2y ago
  It is a good read. Surprising amount of Parrot defenders in the comments, probably missed "LESSON 6 : No, They Aren't Doing That".
• watwut 2y ago
  I have to scroll or click a lot to get to the actual content. I would not bothered to read after three slogans if people did not praised it so much in the comments - and I am still unsure whether they like the content or the sundown picture.
• bjourne 2y ago
  What I find frightening is how many are willing to take LLM output at face value. An argument is won or lost not on its merits, but by whether the LLM say so. It was bad enough when people took whatever was written on Wikipedia at face value, trusting an LLM that may have hardcoded biases and is munging whatever data it comes across is so much worse.
  • AlienRobot 2y ago
    I've seen someone use an LLM to summarize a paper to post it on reddit for people who haven't read the paper. Papers have abstracts...
    • MetaWhirledPeas 2y ago
      Sounds fun, if only to compare it to the abstract.
      • AlienRobot 2y ago
        You know, these days I think the abstracts are generated by LLMs too. And the paper. Or at least it uses something like Grammarly. If things keep going this ways typos are going to be a sign of academic integrity.
        • lithocarpus 2y ago
          A proper LLM will include realistic rates of typos eventually. ;)
          • owl_vision 2y ago
            Amen, Eliza wins. The humans' mistakes of irrational response still boggles my mind.
          • AlienRobot 2y ago
            Darn.
• superbatfish 2y ago
  The author makes this assertion about LLMs rather casually: >They don’t engage in logical reasoning. This is still a hotly debated question, but at this point the burden of proof is on the detractors. (To put it mildly, the famous "stochastic parrot" paper has not aged well.) The claim above is certainly not something that should be stated as fact to a naive audience (i.e. the authors' intended audience in this case). Simply asserting it as they have done -- without acknowledging that many experts disagree -- undermines the authors' credibility to those who are less naive.
  • cristiancavalli 2y ago
    Disagree — proponents of this point still have yet to prove reasoning and other studies agree about “reasoning” being potentially fake/simulated: https://the-decoder.com/apple-ai-researchers-question-openais-claims-about-o1s-reasoning-capabilities/ https://the-decoder.com/apple-ai-researchers-question-openai... Just claiming a capability does not make it true and we have 0 “proof” of original reasoning that can be proved coming from these models. Especially given the potential cheating in current SOTA benchmarks
    • ninetyninenine 2y ago
      It’s stupid. You can prove that LLMs can reason by simply giving it a novel problem where no data exists and having it solve that problem. LLMs CAN reason. Whether it can’t reason is not provable. To prove that you have to give the LLM every possible prompt that it has no data for and effectively show it never reasons and gets it wrong all the time. Not only is the proof impossible but it’s already been falsified as we have demonstrable examples of LLMs reasoning. Literally I invite people to post prompts and correct answers to ChatGPT where it is trivially impossible for that prompt to exist in the data. Every one of those examples falsifies the claim that LLMs can’t reason. Saying LLMs can’t reason is an overarching claim similar to the claim that humans and LLMs always reason. Humans and LLMs don’t always reason. But they can reason.
      • Miraste 2y ago
• almosthere 2y ago
  I cashed a check the other day with my name but it had the wrong address (at least an address my bank is not aware of). I asked google real quick - panicking it wouldn't go through. Google's AI came up and immediately told me to get the check re-issused, go through all this crazy hassle. First result after that, and all the results basically said - you'll be fine. The check is cashed, and went through just fine. They only care about the name. LLMs are bullshit machines for sure. That doesn't mean they have no value, but they can be wrong.
• charlesjamesfuf 2y ago
  [flagged]
• ShellaStevenj 2y ago
  [dead]
• FrankUmah 2y ago
  [dead]
• AniCrystalgu 2y ago
  [flagged]
• SofiaColleano 2y ago
  [dead]
• cyanydeez 2y ago
  Isn't the political climate already lumping people into bullshit vs bull.
• ssssvd 2y ago
  Here's a bridging argument b/w the OPs and the commenters. Anglo-Saxon thought (utilitarianism, behaviorism, pragmatism) treats truth as probability. If an LLM outputs the right tokens in the right order, that’s thinking. If it predicts true statements better than humans, that’s knowledge. The Turing Test? Behaviorist by design. Bayesian inference? A formalization of Anglo empiricism. Continental philosophy rejects this. Heidegger: no Dasein, no being. Sartre: no self-awareness, no thought. Derrida: no deconstruction, no meaning. The German Idealists would outright laugh. So in the Anglo tradition, LLMs are already "thinking." In the French/German view, they’re an epistemic trick — a probabilistic mirror, not a mind. It’s not what LLMs are, it’s how your epistemic tradition defines “thinking.” And that’s probably why the EU is so "lagging behind" in the AI race — no amount of quacking makes an LLM a duck to a Continental. It’s still a parrot. Where you land in this debate is easy to test: Are you comfortable with the statement, "Truth is just what’s most probable given what we already know"?
  • ssssvd 2y ago
    The hilarious outcome? Americans eventually build something they consider "smarter" than themselves — French philosophers agree, but only because it lets them place themselves one step higher.
• anukulrm 2y ago
  Bookmarked and will be sharing this with my friends. Separately, glad to see you, @ctbergstrom and (now Prof) Jevin West still collaborating :). You gave me the opportunity to spend a few days as an undergraduate in your lab 15yrs ago when Jevin was a grad student. It was during the H5N1 scare and I remember you being called up. I was just star struck then and honestly overwhelmed. I think I helped fix some perl code for the eigenfactor project, iirc. Will never forget you and Prof Ben Kerr. Anyways, just want to say thanks after all these years.
• appleorchard46 2y ago
  I think this takes too much of a sort of top-down approach. Many broad statements are made about what LLMs are and aren't - some positive, some negative, some well-substantiated, some less so - when there needs to be a greater focus on fundamental knowledge of their inner workings and (im)practical uses. The title being a question implies it will teach you to answer the question yourself. But it feels more like you're expected to enter with the belief that they're oracles, and this is here to convince you they're bullshit machines. I don't care which one it is! It doesn't matter if we call what they do logical reasoning or not, because either way it doesn't help to give a full understanding of their actual capabilities. Much of the language surrounding this course makes me think the intention is a grounded view. If that is the case though, this misses the mark. Rather than educate on the reality of the situation, I fear the use of it would only exacerbate the principles-first approach seen all too often in discussion around AI. More accurate principles than most perhaps, but principles nonetheless.
• satiric 2y ago
  Apologies in advance for nitpicking a small detail. In lesson 9, you say "The CEO of the Perplexity.ai search platform argues that AI has made blue links obsolete. "But without sources you are driving blind." Isn't the entire point of perplexity that it gives you the sources?