8 ms·
I work at an intersection of tech, applied research, and science. Something I’ve noticed in collaboration that does occur is an increased confidence in people
by Frost1x 6d ago
I work at an intersection of tech, applied research, and science.
Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction.
I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument.
It’s occurring, a lot more. People are treating their LLMs in collaboration as a source of truth and using then to focus on their specific path or goals they think or have bias towards going down, vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable.
It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day.
- varjag 6d agoYup, the arrogance when barging into unfamiliar domain that was previously reserved mainly to physics grads seems to have spread everywhere. I also had people opining on my expert area via their LLMs and the problem is they can't even ask the model the right question, let alone evaluate the nuance of their answer.
- alcomatt 6d agolike a prosthetic dunning-kruger....
- conductr 6d agoAs an unfamiliar, I find it interesting how you specifically called out physics grads in the pre-LLM era. Is that just snark or is there some widespread truth / stereotype of that behavior?
- dhbradshaw 6d agoIt might be a reference to this: https://xkcd.com/793/ https://xkcd.com/793/
- jltsiren 5d agoI prefer the SMBC version: https://www.smbc-comics.com/comic/2012-03-21 https://www.smbc-comics.com/comic/2012-03-21
- harvey9 6d agoI suspect the person you replied to only happened to see that from physics grads. I have seen it in all sorts of people and I sometimes did it myself when I was young (I am not a physics grad).
- francoisdevlin 6d agoNot the parent, but it's half joke half truth. Anyone who's done a lot of "first principles" work falls into this trap a bit, it's an issue across stem fields. Most grow out of it, or at least learn to qualify their statements for the audience.
- acomjean 6d agoIts snark/stereotype, but there is some truth. But my take, is physics sort of took over a lot of chemistry, biology.. and to them its the answer to all questions, because in the end it just molecules. There is some truth to that but..
- skinfaxi 6d agoObligatory xkcd, everything is math. https://xkcd.com/435/ https://xkcd.com/435/
- torginus 6d agoIf you are very smart, and/or have an incredibly high tolerance for pain/repetition, you can solve amazingly complex issues, with awful code. Both of these statements are true for physicists
- odyssey7 6d agoI hypothesize that with more people on a daily basis encountering information and claims that require those skills, there will be a time in the future when those skills are much more enhanced.
- gowld 5d agoCurrently, the leader of USA national AI policy (Trump admin and OpenAI) is a history Bachelors with no technical training or advanced training of any kind, Dean Ball.
- buellerbueller 5d ago>arrogance when barging into unfamiliar domain describes nearly every tech attempt at "disruption"
- GlibMonkeyDeath 5d agoThere is a reason this exists: https://xkcd.com/793/ https://xkcd.com/793/ I sadly share this as a physicist. To be fair, sometimes this arrogance can be useful. I worked at the boundary between domains my entire career where there aren't "experts" per se. But yeah, barging into an established domain without some humility generally leads to embarrassment - especially Nobel winners (e.g. https://www.newsweek.com/nobel-prize-winner-who-doesnt-believe-climate-crisis-has-speech-canceled-1815020 https://www.newsweek.com/nobel-prize-winner-who-doesnt-belie...)
- tty456 5d agoHilarious. Do they preface the conversation with "Well, Claude said..."?
- _1 5d agoWith my younger coworkers it's their defence when I push back on them.
- inanutshellus 6d agoInstant armchair pundit: just add tokens!
- 21asdffdsa12 6d agoWe are going back to philosopher conversations. Just a few guys sitting on the stairs, with chalk, the street as their whiteboard, the jammer keeping the LLM-zombies away. After 2000 years, after the loudness makes right post-modern-pre-llm drivel of the frankfurter school- its a philosopher renaissance as resistance.
- 1euehe 6d ago[dead]
- deleted 6d ago[deleted]
- prepend 6d ago> It’s making me want to be a lot less collaborative with such individuals. I have the same conclusion and am getting a growing list of mental blacklisted people who I just sort of ignore or greatly discount their efforts. I suppose I always did this if people were time wasters but what used to be a tiny list of “idiots” is now getting much, much bigger. Hopefully it doesn’t grow to everyone I work with like an AI Nothing.
- acedTrex 5d agoyep, this is how it goes, you just ignore the llm brained people.
- frank00001 6d ago[dead]
- zzzeek 6d agoit's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect. I'd be curious to know specific examples of LLM-generated advice that goes against the advice of experts and does not consider tradeoffs. I've not had this experience myself. If I did have this experience, someone spouting off obvious LLM points that contradict my expert opinion on something, I'd be headed right over to gemini/claude/whatever to see where that's coming from. Not any differently than if someone cited a google result that contradicted my own experience.
- huurtehoog 6d agoLLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong. Anything more than using them to just pointing me to media and literature is usually a waste of time.
- app134 5d agoThat's interesting. Lately I have found the exact opposite to be true; the frontier models are quite disagreeable and require some convincing
- huurtehoog 5d agoInverted bias doesn't change much my view in practical terms
- giva 5d agoI find quite useful to write down my understanding/opinion and ask an LLM to find errors and write objections to it.
- phyzome 6d agoYeah, I've given up on arguing with heavy LLM users. They're just a meat proxy, and I'm arguing with the LLM. No thanks.
- simianwords 6d agoI think it is amazing and it will democratise information. A lot of times the concept ends up being simple with a lot of jargon exists as a gatekeeping method. Well of course the incumbents wouldn't like that rando from outside gets to understand and speak about things they worked on for many years. Sure he doesn't get it right 100% but he's in the correct direction.
- alcomatt 6d agoI very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.
- bbmatryoshka 5d agoyou can say the same of google, or libraries, information technology is the technology of moving and rearranging information, not of creating it
- oblio 6d agoAwesome, now balance that out will education levels dropping like a rock, due to various factor but also due to LLMs, and things will be balanced out by also democratizing stupidity, until we live in Idiocracy.
- kaffekaka 6d agoPartially true, but only partially. It also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.
- simianwords 5d ago> also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say. Well agree to disagree but my load bearing claim is that a lot of "effort" that the original people put are mostly useless. If you can get information without having to do that effort, you haven't lost anything. For example an archivist finds an 18th century spoon or something and he went through a lot of bureaucratic hoops and logistics to get it. After getting hold of the spoon, you and him are on the same epistemic level because the logistics added ~0 to the truth.
- torginus 6d ago> I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, with conviction. Oh my god, physicist code is evolving!
- fidotron 6d agoIt's a tough call but I'm not sure if physicist code or electronic engineer code is more frightening.
- 01100011 6d agoIdk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs. Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.
- noir_lord 5d agoIt's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool. I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal. Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier. It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end. Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).
- EGreg 6d agoI try to do the opposite. However, I do battle test ideas against LLMs a lot, as well as humans. The experts have very limited access and availability: https://magarshak.com/blog/why-im-confident-in-my-views/ https://magarshak.com/blog/why-im-confident-in-my-views/ The LLMs are trained to often be more pessimistic off the bat than the humans. But you can wear them down with arguments and they change their mind. Here is an example where I have pushed my ideas in areas where I am not an expert, and generated papers to submit to conferences in order to get them peer reviewed BY experts: https://magarshak.com/papers.html https://magarshak.com/papers.html This is exactly what you’re talking about, except done very carefully.
- deadbabe 6d agoAt some point, LLMs will be a far better source of truth, and then this distaste of people “outside their domains” is really just going to be a sort of snobbery from people who have had experience in a domain for a long time, (but they still have the same level of knowledge and insight as a person who just used an LLM to research). These people are basically nascent “human supremacists”, who believe that only raw human insight and output has value, and is even superior, than equivalent output looked up and synthesized via LLM.
- lins1909 6d agoNot trying to be rude, but is this a serious comment, or are you being sarcastic? "Human supremacists", haha. Also, why do you think someone who used an LLM to research would have the same level of knowledge and insight as someone who's been in the domain for a long time? That doesn't seem to hold up to any level of critical thinking.
- surajrmal 5d agoIf I talk to a professor about their field of expertise, does that empower me to talk with other professors in that field on an equal level?
- oniony 6d ago"I also hear a lot more questioning people in their domains and challenging opinions, then hearing what I can only imagine are fragmented pieces of conversations they had with an LLM thinking through some argument. Then there’s silence when you discuss shortcomings, then they come back later with their memorized fragments of what you said, combined with memorized fragments of the LLM response to the argument." We hired this guy recently.
- ambicapter 6d agoMy condolences :(
- acedTrex 5d agoNip it in the bud quickly
- fapjacks 5d agoThe worst thing about AI is that it's camouflage for stupid people. And it often gets combined with utter confidence in some position because they're too stupid to know the machine is confidently wrong.
- ticulatedspline 6d agoIsn't this basically the WebMD effect migrating to other domains? feels like the same lament doctors have had for ages after anyone could google their symptoms then self-diagnose. While AI is driving it the foundational cause feels like people having easy access to data/opinion that they trust but don't fully comprehend (or have bias towards). People do this in meat-space too, will confidently regurgitate garbage if they were told it from a person they trust as an expert.
- 1euehe 5d agoLet’s be honest. It’s always been present - e.g when I see people talking about finance on here with completely mangled fundamentals. But now it’s amplified like crazy. We are going into a world where the geniuses will have to be isolated and protected from brain rot.
- illiac786 6d agoI think there will be a rebalancing, and it might painful first for some. Confidence in LLM for highly complex tasks, very lots of context, most of all when this context isn’t in a single place or simply isn’t digitalised, will go down. But right now, it’s still the shiny new toy
- LogicFailsMe 6d agoI use state of the art frontier coding agents daily. While the fact they can code at all is a huge achievement, it's hard for to me to buy in to the looming AGI/2027 collapse scenario when they remain stymied by the simplest out of sample tasks and sabotage their own work frequently. But the blooming ignorance of everyone addicted to them is striking. These things are tools, used correctly they can do some amazing things, but going Tim the Toolman with them goes poorly.
- LogicFailsMe 6d agoThen there are the megacorporations of AI whose highly paid tech support folk happily copypasta your reported issues into Claude or Codex and paste its "solution" into an email as if you couldn't do that yourself and they don't even bother to check if the "solution" even works.
- hgoel 6d agoI also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI. Sure, they use it a lot, but despite the stereotypes against physicists in some of the replies to you, they have shown a good ability to catch themselves before leaning too hard on what an LLM tells them about things they are not experts in, and I haven't had any interactions where I felt like I was just arguing with a meat proxy. Maybe it's because in the environment I'm in; it's relatively easy to just ask someone who is an expert in the topic for their advice. My feeling is that this kind of overconfidence outside of one's domain is largely a thing for people with little "physical reality" experience. When all you deal with is the very flexible digital world, it becomes very easy to ignore how deep the knowledge and intuition goes in things that are directly constrained by reality. I consider myself to have been in this category too (CE background), though I have been making efforts to improve.
- noir_lord 5d ago> I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI. It's similar to the pattern I've noticed with experienced programmers I know and respect, the really good ones do seem to use AI but they are deeply sceptical about its claimed capabilities so they check/verify/assess what it's good for and use it for that, the tier down is less questioning and just accepts whatever it generates as gospel and it seems to degenerate the further you go down. It seems like as soon as the AI states something confidently and is wrong in the domain in which the user is an expert, it loses a lot of credibility and people become much more wary of it in general but that requires you to be able to see that it is wrong otherwise it becomes a "bullshit baffles brains" generator. It's useful to remember those times you've seen it do that when you are asking it something you aren't an expert in and then verify it's answer a different way.
- surajrmal 5d agoI saw this happen a bit earlier this year, but most everyone I work with has learned that it was foolish. The feedback that they are doing something wrong needs to be explicit and strong. We're all going through a learning curve and establishing cultural norms is important at this time.
- chunky1994 5d agoBut isn't there value to refuting what Claude is saying if it deserves to be refuted? The way I see it, if I was repeatedly getting sophisticated but subtly wrong arguments about my work that require me to understand why said argument is wrong, that's essentially drilling down to specifics of what precisely needs to be true to solve the problems I am aiming to solve. There is enormous value to this precision, isn't there?
- Matumio 5d agoYes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said. Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.
- chunky1994 5d agoYes it is annoying, however I find that it gives me a much better grasp on the things I am explaining to a "proxy" Claude person. Basically, I would never have stumbled upon the arguments that I'm being presented with as I would never have prompted Claude in the specific way that a proxy Claude person would. I find a lot of value in understanding how they have misunderstood things and clarifying those concepts.
- pastel8739 5d agoWhat do you suggest these people do instead? I’ve been frustrated by this recently: I pivoted to a new subfield and am working on stuff that I would love to dive deep into and really learn what is going on so I can speak intelligently about the tradeoffs etc. But that would take a long time, and I have tasks that I should get done. So I have found myself working with a pretty vague understanding that, when pressed by coworkers, quickly finds its limits. Then I go back and try to deepen my understanding enough to cover those limits. But because I’m not working with each detail of the problem, implementing line by line with time to think about what’s happening, there just isn’t time for me to learn this unfamiliar topic. But I would love to, and I would enjoy the work much more if I could. So what am I supposed to do?
- vasco 5d agoRead some books on the subject and within a year in this new domain you'll be much better. Did you forget your whole life before the last 3 years?
- pastel8739 5d agoYes, certainly, in a year I’ll be better. But in the meantime…? 3 years ago it would be understandable for me to take my time with the tasks I’m doing because they are complex. Now it would be unreasonable for me to spend enough time to understand the nuances.
- freedomben 5d agoI'm not GP, but I've been experiencing this too. All my coworkers are in the same boat though, so we don't really have anyone with the deep understanding. Our approach (which has been working pretty well, we've solved a lot of problems and learned some along the way) has been to use AI as an informer who also points us to sources (like documentation, github issues, etc) where we can then read up on something the AI has pointed us at. We try to time box things a bit to prevent going down the rabbit hole, so we don't always get to exhaust our curiosity, but we're continually gaining that knowledge and pressing forward. Most recent example is tuning the kernel settings and our app on our prod machines to behave better for WebRTC packet handling/forwarding. AI for things like ffmpeg has been a god send when nobody on the team is an ffmpeg expert. I've also had success just asking claude to write documentation on a system or subsystem or module, etc, and reading that. I then sometimes have it turn that into an svg diagram or something visual that often helps understand things. That domain-specific knowledge is hard to gain though, so not a silver bullet by any stretch.
- palmotea 5d ago> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day. There are also the guys who are just blatantly meat proxies for Claude. You send them a message, and you get back a response that's all Claude (and disorganized and not really making sense to boot). I've got one on an affiliated team and I basically don't want to collaborate with him at all anymore. But you know, "AI is the future of work," and all that. Those guys get a pat on the head by higher-ups and probably think they're doing what they're supposed to.
- goalieca 5d ago> back a response that's all Claude (and disorganized and not really making sense to boot You’re absolutely right! They trusted but did not verify. Thats a load bearing point — they assumed intention from your question.
- NichoPaolucci 5d agoWow, oddly similar thing happened to me. Anecdotally, I was working with a non-developer. She was building a reporting dashboard with Claude. I pointed out that the system she built was using the filesystem to store one line notes about the report, and I said "I would probably tell the agent to use a database for that, but it's not the end of the world if it works". And she replied "Claude said the developer's right, a database would be a better fit..." So, a non-developer, who has no understanding of the underlying systems they are working with, fact-checked a developer with years of experience. I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify. This is why collaboration exists. I trust my colleagues to give me insight+advice on things that I do not know. I don't ask AI to vet their decisions, because at some level there are judgement calls, and I WANT to trust them because it makes my life easier. Treating AI as if the output is factual is just a misuse of the tools, I think. It's significantly more effective when given expert direction and the output can be verified by... an expert.
- otoburb 5d ago>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify. This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.
- tempacct2cmmnt 5d agoDoctor: I don’t mind the questions. I mind the assertions of fact.
- roarcher 5d agoI would love to be able to trust that a doctor knows more about my problem than I do, but every time I see one they clearly haven't even read the information I just told the nurse 15 minutes ago.
- jcmontx 5d agoThis resonates a lot with me. I'm the CTO of a software factory-ish company. I get daily emails from clients running our decisions through LLMs and asking for ridiculous stuff that would multiply the implementation cost for them, just because Claude said so. No, your 3 users app doesn't need to be SOC2 compliant. No, of course we don't have triple zone redundancy while you're paying a 50 USD AWS bill. Most of the time, these are just concerns triggered by chatting by an over-zealous LLM; in the worst cases, they are demands. I'm usually able to clear concerns and disarticulate the LLM by explaining to customers how much more expensive everything would be if we did things like that. But it feels so frustrating, it's like you have to prove yourself everytime and justify every decision. These interactions have made me question my future in the industry, if I'm willing to keep dealing with these situations. I'm trying to foster patience in my life to cope with this.
- svachalek 5d agoUgh. To be fair to the LLMs they're trained on industry blogs, and if you spent too much time reading them you'd also think every electronic stamp album needs to scale to a billion users with millisecond regional failover.
- knuppar 5d ago> It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day. I echo the sentiment. It's frustrating to try to collaborate with a Claude/ChatGPT meat proxy. You can write a very thoughtful two-line question over 5 minutes and get a 5 page design doc back in a minute.
- away0g 5d agoI remember when google on your phone was new. The common complaints was, look at me while im talking, dont just fact check me. or something along those lines. Its interesting to see the same shaped problem come up again. we didnt really figure out a solution to this, the world just kind of absorbed it as normal. how long before this is absorbed.
- xnx 5d agoProbably more to it, but that's a pretty weak example. There are a lot of reasons text files might be better suited storing data vs. the overhead of a database.
- baxtr 5d ago>It’s making me want to be a lot less collaborative with such individuals. I don’t want to sit around and refute Claude text outputs all day. at least this behavior reveals now much faster people you don’t want to work with.
- alienbaby 5d ago"vs just opening discussing things, considering tradeoffs from experts multiple disciplines weigh in on and then taking an approach that everyone finds most agreeable." To be honest I struggle to do this with engineers as is, even without AI being a factor. Everyone likes to claim that their way is the right way to do things and fights for it, instead of stepping back and looking at all the options and gracefully acknowledging were things could be done differently for better results.