10 ms·
The main point raised in the article is that these bots may void attorney client privileges. But the real danger with these IMO is that they're turning casual
by burningion 4mo ago
The main point raised in the article is that these bots may void attorney client privileges.
But the real danger with these IMO is that they're turning casual conversations into a permanent record, and one that will be completely discoverable in court, should the company get into trouble later.
- coffeebeqn 4mo agoPlus they are super inaccurate. Gemini gets one of its three bullet subtly or very majorly wrong almost every time. Just a few weeks ago Gemini said we’re rolling out our payment setup in Russia. You know the place where we have 20+ sanctions packages on? We were talking about France in the meeting.
- pjc50 4mo agoGiven how financial services can impose silent inexplicable lifetime bans for using the wrong words in the "what is this transaction for" field, I'm wondering at what point the AI automatically reports people for sanctions violation based on its mishearing.
- operation_moose 4mo agoWe've found they're surprisingly good if everyone on the call is using a decent headset. The problems start when using conference room audio or someone is on their laptop mic. If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. We just went through a round of 100+ (non-sensitive) VoC interviews and they really cut down the workload of compiling all of the feedback. If the audio was a little shaky though, we pretty much had to throw away the transcripts and do them from scratch like we used to.
- user_7832 4mo ago> If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Imo this is the single biggest flaw of LLMs. They're great at a lot of things, but knowing when they're wrong (or don't have enough information to actually work on) is a critical flaw. IMO there's nothing structural about why they shouldn't be able to spot this and correct themselves - I suspect it's a training issue. But presumably bots that infer context/fill in the dots rank better on what people like... at the cost of accuracy.
- r_lee 4mo agoI don't think it's a training issue, it's simply that there's no inherent "I don't know" in the transformer architecture unless it's really like something completely unknown, otherwise the nearest neighbor will be chosen and that will be whatever sounds similar or is relevant, even if it might cause a problem
- aspenmartin 4mo agoNot inherent in transformer architecture, we do try to ingrain a sense of uncertainty but it’s difficult not only technically but also philosophically/culturally. How confident do you want the model to be in its answer to “why did Rome fall”? Lots of tools in our toolbelts to do better uncertainty calibration but it trades off against other capabilities and actually can be rather frustrating to interact with in agentic contexts since it will constantly need input from you or otherwise be indecisive and overly cautious. It’s not technically a limitation of transformer architecture but it is more challenging to deal with than other architectures/statistical paradigms. Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant. But evals reward guessing at this point, and it’s very very hard to evaluate the calibration in these open ended contexts. But we’re slowly getting there, just not nearly as fast as other capabilities.
- fluoridation 4mo ago>How confident do you want the model to be in its answer to “why did Rome fall”? The confidence level can be any, as long as it's reported accurately often enough. "This is my conjecture, but", "I'm not completely sure, but", and "most historians agree that" are all perfectly valid ways to start a sentence, which LLMs never use. They state mathematical truth, general consensus, hotly debated stances, and total fabrication, with the exact same assertiveness.
- Terr_ 4mo ago> > Like you can maintain a belief state and generate conditional on this and train to ensure belief state is stable and performant > ways to start a sentence, which LLMs never use A huge part of the problem is we've invented a document-generator setup which exploits human cognitive illusions, and even the smartest person can't constantly override the instinctive brain-bits that "sees" fictional entities and infers the intent of a mind. That makes it weirdly-hard to discuss the setup's shortfalls or how to improve it. To wit: The machine does not possess any kind of confidence about how Rome fell. Or even whether Rome fell. It has "confidence" about which word/token will next in a "typical" document given the document-so-far has text like "How did Rome fall?" It may be straightforward to burn money training the system so that its "typical" story never has a computer-character with confident words about Roman history, but that's just papering over the underlying problem. TLDR: We can't fix the thinking-habits or beliefs inside the mind of an entity that doesn't actually exist. Changing the story-generator to contain a tee-totaling Dracula dispensing life-advice doesn't mean we "cured the disease of vampirism."
- steveBK123 4mo ago> The problems start when using conference room audio RTO problems
- inetknght 4mo agoTheir quality for different language accents also significantly varies. Got a team with Indian, Chinese, Texan, British, and Australian? Your A.I.-powered translation tool is going to get 80% of your conversation wrong.
- FireBeyond 4mo agoMy workflow uses krisp.ai for taking a transcript, and then I have a dedicated project in Claude. I feed it the transcript and ask for it to give me a summary in a specific format I define, with good front matter, etc., and it needs to spit that out in a way that I auto-import into Obsidian. But key in my prompt is asking 1) for it to flag any low confidence or context-nonsensical statements in the transcript, with the timestamp, so then I can listen to the original audio and either clarify, correct, or say "I couldn't understand that either, here's my best guess and mark it low confidence", then 2) which I see as critical: Claude also is told to create a "context" document that it maintains based on my answers, so it starts to gather ASR things like "transcript commonly hears A B and C as variants of name X", who is who, internal product and project names and context info on them. 3) Claude is told specifically to read this prior to summarizing the transcript, and to consult it as it is doing so, and to ask me on anything it's not confident on. What is then starting to get quite powerful for me is moving from full text search of my meeting notes in Obsidian (I'm a PM in a lot of meetings), but I can point Cowork to the Obsidian notes folder (because they're all Markdown) and start doing rich "querying" of it. "When did [stakeholder] first mention [feature] as a release blocker?" and it can point to the meeting. My system works well, and I've done a bit to fine tune the automation and friction reduction, and it's a bit easier to manage because I'm not generally creating summaries for broader consumption but as my second brain (I have a separate prompt that utilizes some of that "knowledge" to build those). One thing I've found helpful with this is moving the summarization itself into something with "context/memory". Krisp is capable of generating summaries but can't/doesn't review prior transcripts. Its role is just "give me the transcript as you heard it".
- Kushalzen 4mo agoFor the "when did X first mention Y" queries, does it surface things you'd forgotten about that turned out to matter? Or mostly just confirm what you half-remembered? Trying to figure out whether the value of the loop is rediscovery or just precise lookup.
- FireBeyond 4mo agoI think it's both, for me. I struggle with recall, so it helps me remember. I'm also using it to push to Todoist, etc. I need the memory jog, and then it helps me confirm things as they come back to me, to ensure I remember correctly. (ADHD, and across many projects as a PM).
- deleted 4mo ago[deleted]
- insane_dreamer 4mo agoRecent example: - the person said 8 to 10 - LLM transcribed as 18 Granted, the person had a foreign accent and didn't enunciate very clearly. But I knew they meant 8-10 if for no other reason than 18 didn't make sense given the context. But the AI isn't smart enough, and then 18 goes into the record.
- CGMthrowaway 4mo ago> headset Half- vs. full duplex. Headphones is all you really need, though of course a directional mic and/or one closer to your mouth will yield a clearer audio recording as well.
- cyanydeez 4mo ago"This technology works as long as you're not a pleb"
- laughing_man 4mo ago>If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Isn't that what people do?
- Lammy 4mo agoFor in-person conversations to keep the conversation flowing, sure, but any good transcription will say [unintelligible] when the scribe couldn't tell despite being able to listen over it again and again. Nixon tapes for example: https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintelligible%22&r=us&sh=10bBvH0fgFIYk3qhfvLO2Q https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintel...
- PeterStuer 4mo agoI have done many transcriptions of messy meeting recordings with thic euro-english accents, and a local Whisper large handled them near perfect.
- elzbardico 4mo agoVerbatim transcriptions are usually very good. Because even the ocasional "can/can't" replacement is usually obvious within the context of the full conversation. But the summarization feature is where the most ridiculous errors and omissions happens.
- therealdrag0 4mo agoOk zoom the default summary template is often lacking and incoherent, but switching it to the lengthy one “Discussion” works great. I think the default only works for single topic meetings where is rare.
- breppp 4mo agoThat's presumably great for legal exposure because it increases deniability
- buzer 4mo agoI wonder what kind of GDPR implications that has given the requirements around the accuracy of personal data.
- watwut 4mo agoBasically, it will be harder to hide illegal and unethical stuff companies routinely engage in.
- nz 4mo agoNo, that would be a strict improvement. The AI note-takers can easily "mishear" or "misreport" non-existent illegal and unethical things. It also seems to easily mess up numbers (which is big problem, because a lot of decisions hinge on precise numbers -- imagine inflating an inventory by an order of magnitude, and then imagine having to pay a tariff on something that never existed). I have a friend who works at a large-ish company that imports and manufactures things (in one of the clerical/quantitative professions). A few years back, they had the IT department go on a kind of "inquisition", wherein they forced employees to disable the summarization function that came with MS Teams, and threatened to fire them if they did not. The resistance to this demand was surprising -- most people are clueless about the cost of their own convenience. Worst of all, people would zone out of meetings, because the AI was producing summaries, which they would then never read. The effect of the technology was that it made meetings infinitely more expensive, because the supposed benefit of meetings was nullified by complacency, _and_ it made the meetings a liability (incorrectly summarized meetings, that could be used in the discovery process, sure, but could also be sold by MSFT as a kind of market-research-data to competitors in the space). Nothing illegal has to happen in these meetings at all, for this tech to cause an infinity of problems for the corporation. Every employee that uses these is effectively an unwitting spy. And if that is the case, then the meetings might as well be recorded and uploaded to YouTube (or whatever people watch these days)[1]. [1]: Maybe this is the future. Which I am okay with, but only if the entire planet has to do it, and the penalties for not doing it are irrecoverably severe.
- chvid 4mo agoShow me man the man and I will show you the crime. Modernized. Industrial AI scale.
- SecretDreams 4mo agoGoing to also be harder to hide completely legal, but not ideal stuff. Like randomly complaining about your boss to a colleague or casually discussing a feature you're stuck working on that you think is a bad idea.
- stego-tech 4mo agoThis. The fact LLMs can also amplify existing closed-set research means even smaller shops can now search through a flood of documents to find smoking guns or critical evidence, much faster. I’ve been saying it since the mid-10s, but it’s worth repeating: data isn’t gold, it’s more like oxygen in a room in that the higher the concentration, the more likely it is to poison the inhabitants or explode with an errant spark (lawsuit). Collect only what’s needed to perform the function, and store it only as long as necessary for compliance. Anything else is going to spool counsel.
- mock-possum 4mo agoWhat are you trying to get away with I wonder?
- stego-tech 4mo agoChillax Palantir, your pro-surveillance throwaway incidentally makes such large data harvesting companies a larger target. Limiting data retention doesn't mean hiding bad things, it means limiting exposure in general. The more of a thing - anything - that you have, the bigger a target you are to bad actors. By extension, companies holding vast sums of data beyond what's needed to process a given transaction or remain compliant with the law end up placing themselves at risk of being targeted and said data used as leverage against them. You don't limit data to hide bad shit you're doing, you limit it to avoid others using it to do bad shit against you or your customers. If someone or something is engaged in bad shit, there will always be evidence somewhere regardless of data retention policies.
- array_key_first 4mo agoProbably nothing, he's just not naive. You would have to have the intelligence of a small child to legitimately believe that authorities are only ever acting in benevolence, never with ulterior motives, and that they can never make mistakes. It's a matter of risk analysis here; we want to minimize the risk of shit going wrong.
- deleted 4mo ago[deleted]
- LanceH 4mo ago> But the real danger with these IMO is that they're turning casual conversations into a permanent record, and one that will be completely discoverable in court, should the company get into trouble later. I would add that their is no guarantee their are correct as well.
- mock-possum 4mo agoYou’d use a computer generated transcript as a guide, not as proof - the proof is the recording of the person actually saying the thing, not the LLMs best guess of what it imagined the person saying. “At timestamp X, person Y said Z” says the robot, and then you dutifully scrub the audio to timestamp X to verify.
- deleted 4mo ago[deleted]
- infecto 4mo agoThe nuance here too is that just because someone has concern about materials being discoverable does not mean the company is doing something illegal. Corporate law as it pertains to legislation (US in this perspective) is a dance between company and current administration. When it comes to antitrust and other related legislation the equilibrium is shades of gray that changes between both administration changes but sometimes from the same administration. Companies look to optimize their outcomes and the government is optimizing not so much for legality but what the current administration sets as the main concern.
- camdenreslink 4mo agoThe AI note summaries in meetings I'm in are frequently totally inaccurate. They are actually inaccurate in two ways: they fabricate things that were never said (but always kind of close to something that was said), and they emphasize the totally wrong thing (e.g. acting like the entire conversation was about one topic when that was just a very small part). I sincerely hope these aren't used in court.
- rayiner 4mo agoThey will be discovered and used in litigation, and the results will be hilarious. Think about how much lawyers pick apart language (like statutes or the constitution) that was written deliberately by humans and subject to review and revision. Now we're going to have lawyers, e.g., seizing on word choice in AI notes that might have a sinister connotation when the original wording was innocuous.
- nullc 4mo ago> word choice in AI notes that might have a sinister connotation Potentially sinister due to the biases of the model, as the model may have been trained using internet content that has a lot more fictional titillating evil overlord board meetings than the actual mind-numbing real thing. Training that included extremist anti-corporate dogma might even bias the language models towards hallucinating the worst possible misinterpretation. I've seen whisper hallucinate whole legal arguments whole cloth when the AGC was broken in it and the audio went quiet-- so I think the language models in it are more than powerful enough to politically load a transcript. Good practice should be to minimize any unnecessary stored records because ANY record just means more processing costs in discovery and god knows how much extra cost in litigation should it happen to have an unfavorable interpretation in light of some impossible to anticipate future litigation. But if AI transcription must be used it would be might be prudent to save a copy of the original audio along with it.
- Lammy 4mo ago> seizing on word choice in AI notes that might have a sinister connotation Ironic use of “sinister” when you probably mean “nefarious” and don't mean to perpetuate silly old superstitions about “left-handed” people being evil :p
- yagizdagabak 4mo agomy fear exactly. same with something like Meta glasses. and i feel like we have moved quickly from the regulatory problems to "'tis a fact of life"
- Bombthecat 4mo agoNot only there Also social settings will change, when everything you say stays on record forever in every meeting...
- papageek 4mo agoNever write if you can speak; never speak if you can nod; never nod if you can wink. -Lomasney (has aged well it seems)
- smrtinsert 4mo agoSeems like something that will add to their billable hours
- cosmotic 4mo agoHow can ignorance of the law not be a valid defense while intentionally not recording known illegal activity be a effective defense?
- pfisch 4mo agoThere is no duty to record everything everyone does. No one is legally compelled to record their actions except for a few rare situations...
- piker 4mo ago> But the real danger with these IMO is that they're turning casual conversations into a permanent record, and one that will be completely discoverable in court, should the company get into trouble later. The parts that aren’t privileged. On the other hand, perhaps the truth-seeking function of the justice system will be better equipped than before when we had to rely on (more) faulty human recollection.
- piker 4mo agoNo idea why this was downvoted? That’s what the justices system tries to do.. find the truth. Missing or obfuscated evidence works against that.
- jjk166 4mo agoIs it reasonable to expect any call going through a computer to be off the record, even without AI? Recordings were always discoverable, the only difference is that a paralegal doesn't need to manually go through every recording to determine if it's relevant.
- projectazorian 4mo agoExactly. “Don’t write anything down you wouldn’t want to see in the newspaper” just became “Don’t say anything in your meetings or 1:1s that you wouldn’t want to see in the newspaper.” I’m overall an AI optimist but this is going to blow up in people’s faces very quickly. (I would explain this to my manager but he has AI note taking turned on in all his meetings!) And that’s not even getting into the use of it for sensitive clinical notes in eg. mental health…
- PeterStuer 4mo agoIf you are on a call, it's already potentally recorded and transcribed. This just makes it so you also have that capability.