5 ms·
I found this principle particularly interesting: Human oversight: The use of AI must always remain under human control. Its functioning and outputs must be
by singiamtel 10mo ago
I found this principle particularly interesting:
Human oversight: The use of AI must always remain under human control. Its functioning and outputs must be consistently and critically assessed and validated by a human.
- conartist6 10mo agoIt's still just a platitude. Being somewhat critical is still giving some implicit trust. If you didn't give it any trust at all, you wouldn't use it at all! So they endorse trusting it is my read, exactly the opposite of what they appear to say! It's funny how many official policies leave me thinking that it's a corporate cover-your-ass policy and if they really meant it they would have found a much stronger and plainer way to say it
- miningape 10mo agoI think you're more reading what you want to read out of that - but that's the problem, it's too ambiguous to be useful
- hgomersall 10mo agoThat doesn't follow. Say you write a proof for a something I request, I can then check that proof. That doesn't mean I don't derive any value from being given the proof. A lack of trust does not imply no use.
- MaybiusStrip 10mo ago"You can use AI but you are responsible for and must validate its output" is a completely reasonable and coherent policy. I'm sure they stated exactly what they intended to.
- geokon 10mo agoIf you have a program that looks at CCTV footage and IDs animals that go by.. is a human supposed to validate every single output? How about if it's thousands of hours of footage? I think parent comment is right. It's just a platitude for administrators to cover their backs and it doesn't hold to actual usecases
- pu_pe 10mo agoI don't see it so bleakly. Using your analogy, it would simply mean that if the program underperforms compared to humans and starts making a large amount of errors, the human who set up the pipeline will be held accountable. If the program is responsible for a critical task (ie the animal will be shot depending on the classification) then yes, a human should validate every output or be held accountable in case of a mistake.
- mattkrause 10mo agoExactly. If some dogs chew up an important component, the CERN dog-catcher won't avoid responsibility just by saying "Well, the computer said there weren't any dogs inside the fence, so I believed it." Instead, they should be taking proactive steps: testing and evaluating the AI, adding manual patrols, etc.
- conartist6 10mo agoI take an interest in plane crashes and human factors in digital systems. We understand that there's a very human aspect of complacency that is often read about in reports of true disasters, well after that complacency has crept deep into an organization. When you put something on autopilot, you also massively accelerate your process of becoming complacent about it -- which is normal, it is the process of building trust. When that trust is earned but not deserved, problems develop. Often the system affected by complacency drifts. Nobody is looking closely enough to notice the problems until they become proto-disasters. When the human finally is put back in control, it may be to discover that the equilibrium of the system is approaching catastrophe too rapidly for humans to catch up on the situation and intercede appropriately. It is for this reason that many aircraft accidents occur in the seconds and minutes following an autopilot cutoff. Similarly, every Tesla that ever slammed into the back of an ambulance on the back of the road was a) driven by an AI, b) that the driver had learned to trust, and c) the driver - though theoretically responsible - had become complacent.
- pu_pe 10mo agoSure, but not every application has dramatic consequences such as plane or car crashes. I mean, we are talking about theoretical physics here.
- SiempreViernes 10mo ago> So they endorse trusting it is my read, exactly the opposite of what they appear to say! They endorse limited trust, not exactly a foreign concept to anyone who's taken a closer look at an older loaf of bread before cutting a slice to eat.
- Sharlin 10mo agoInteresting in what sense? Isn't it just stating something plainly obvious?
- jacquesm 10mo agoIt is, but unfortunately the fact that to you - and me - it is obvious does not mean it is obvious to everybody.
- Sharlin 10mo agoQuite. One would hope, though, that it would be clear to prestigious scientific research organizations in particular, just like everything else related to source criticism and proper academic conduct.
- SiempreViernes 10mo agoDid you forget the entire DOGE episode where every government worker in the US had to send an weekly email to an LLM to justify their existence?
- Sharlin 10mo agoI'd hold CERN to a slightly higher standard than DOGE when it comes to what's considered plainly obvious.
- SiempreViernes 10mo agoSure, but the way you maintain this standard is by codifying rules that are distinct from the "lower" practices you find elsewhere. In other words, because of the huge DOGE clusterfuck demonstrated how horrible practices people will actually enact, you need to put this into the principles.
- piokoch 10mo agoOddly enough nowadays CERN is very much like a big corpo, yes they do science, but there is a huge overhead of corpo-like people who running CERN as an enterprise that should bring "income".
- xtiansimon 10mo agoWhere is “human oversight” in an automated workflow? I noticed the quote didn’t say “inputs”. And with testing and other services, I guess human oversight can be reduced to _looking at the dials_ for the green and red lights?
- SiempreViernes 10mo agoSomeone's inputs is someone else's outputs, I don't think you have spotted an interesting gap. Certainly just looking at the dials will do for monitoring functioning, but falls well short of validating the system performance.
- monkeydust 10mo agoThe real interesting thing is how does that principle interplay with their pillars and goals i.e. if the goal is to "optimize workflow and resource usage" then having a human in the loop at all points might limit or fully erode this ambition. Obviously it not that black and white, certain tasks could be fully autonomous where others require human validation and you could be net positive - but - this challenge is not exclusive to CERN that's for sure.
- deleted 10mo ago[deleted]
- contrarian1234 10mo agoDo they hold the CERN Roomba to the same standard? If it cleans the same section of carpet twice is someone going to have to do a review?