8 ms·

While there's no linter for writing contracts, my experience (as a commercial lawyer) is that frontier LLMs are far better and error checking and far quicker at

While there's no linter for writing contracts, my experience (as a commercial lawyer) is that frontier LLMs are far better and error checking and far quicker at writing than the average senior lawyer. The main thing holding back further deployment (in my jurisdiction) are concerns around data residency, privilege and how fundamentally it will break an industry that is so heavily reliant on time based billing.

nprateem 24d ago
Trouble is they're unreliable. I compared insurance quotes last week. Needed cover for 2 brands, 1 company. Opus jumps up and down saying both brands need listing on the policy schedule. Human broker said not. I told opus and it's the usual "thanks you're right" bollocks because it bothered to read in more detail and found that all business activities are covered.
Zizizizz 24d ago
I agree with you in respect to the greatest risks in that industry. Just thought it's worth mentioning that LLM's unsurprisingly are not immune to hallucinating cases. https://www.scientificamerican.com/article/why-lawyers-keep-citing-fake-cases-invented-by-ai/ https://www.scientificamerican.com/article/why-lawyers-keep-... Using them for coding makes it easy to self check its work (assuming those pieces of work are "verifiable"). As proofreading will still need to happen, what do you think the appetite for lawyers is to do this kind of work? Do you think it will drive fees down significantly? Empower younger lawyers at firms who probably are the ones doing this checking for the partners? (Or will that just create a further divide). I'm genuinely asking as I am not in law but all my family is and it's nice to see someone here that's thought about the impacts in that space.
- ray_kay777 24d ago
  With access to good databases and tool use, hallucinations are basically a thing of the past (and very easy to verify). The hallucination issue typically comes from people using free ChatGPT or similar - tech usage and literacy is often poor and there's a lack of understanding of the risks of using something without proper legal database tools. Most lawyers would not have the faintest idea of what tool use in the context of an LLM even means. It's still seen as a magic box rather than a tool for real work. Will it drive down fees significantly? Doubt it, there's not enough pressure on them, and the industry is resistant to change. Firms don't want to make a big deal about using it because clients will then ask why they're not getting a discount. Will it empower younger lawyers? Not as much as I'd like. I'm very fortunate to have an employer that lets me use Claude Code for my work (to a limited extent). I think for 99.9% of lawyers it's not an option available to them, through a mix of concerns around AI usage and concerned IT departments. There could be great benefits, but it would rely on having to break out of traditional private practice which would make it difficult to get enough work. I think in the near term LLMs will have much the same impact on the legal industry as it has on the software industry.
seer 24d ago
But isn’t “writing a linter for contracts” the thing we need and probably would solve? Coding agents are great because they have compilers, linters, test cases etc to ground themselves in. With tools like OKF I’m sure most knowledge work would be distilled to its core data - it’s AST if you will, and then allow models to guard against hallucinations. Checking if a case law exists is a tool call, you can demand provenance, it’s all _buildable_. Hallucinations are “solvable” this way, so the rest is just time and adoption…
bostik 24d ago
> concerns around data residency Funny you should say that. I read somewhere (not on HN, but I think it was a post linked from here) that a number of law firms who deal with extremely sensitive documents have started buying amped-up Macbooks with 512GB of memory to be able to run local models. These are businesses who literally - and for once this word fits - cannot afford to let some of those documents get anywhere outside their corporate walls.
- senordevnyc 24d ago
  Macbook Pros max out at 128GB
  - bostik 23d ago
    Turns out the books do, studio setups can go up to 512GB. There was a thread earlier this year about the option getting pulled from selection (https://news.ycombinator.com/item?id=47296302 https://news.ycombinator.com/item?id=47296302) but I'd guess you can still top the thing up yourself.
algo_trader 24d ago
>my experience (as a commercial lawyer) How do you think the industry will respond to contract clause slop ?? I can foresee each side inserting 100s of innocent clauses with minute dependencies that provide hidden advanatges
- ray_kay777 24d ago
  Good question - but the state of commercial contracts (pre-AI slop) is horrifically dysfunctional, inefficient, poorly thought through and inaccessible to the average person, so the bar is quite low. I think there will be some new malicious behaviours like that to keep an eye out for but I haven't seen them in practice (yet).

nprateem 24d ago

Trouble is they're unreliable. I compared insurance quotes last week. Needed cover for 2 brands, 1 company. Opus jumps up and down saying both brands need listing on the policy schedule. Human broker said not. I told opus and it's the usual "thanks you're right" bollocks because it bothered to read in more detail and found that all business activities are covered.

Zizizizz 24d ago

I agree with you in respect to the greatest risks in that industry. Just thought it's worth mentioning that LLM's unsurprisingly are not immune to hallucinating cases. https://www.scientificamerican.com/article/why-lawyers-keep-citing-fake-cases-invented-by-ai/ https://www.scientificamerican.com/article/why-lawyers-keep-... Using them for coding makes it easy to self check its work (assuming those pieces of work are "verifiable"). As proofreading will still need to happen, what do you think the appetite for lawyers is to do this kind of work? Do you think it will drive fees down significantly? Empower younger lawyers at firms who probably are the ones doing this checking for the partners? (Or will that just create a further divide). I'm genuinely asking as I am not in law but all my family is and it's nice to see someone here that's thought about the impacts in that space.

ray_kay777 24d ago
With access to good databases and tool use, hallucinations are basically a thing of the past (and very easy to verify). The hallucination issue typically comes from people using free ChatGPT or similar - tech usage and literacy is often poor and there's a lack of understanding of the risks of using something without proper legal database tools. Most lawyers would not have the faintest idea of what tool use in the context of an LLM even means. It's still seen as a magic box rather than a tool for real work. Will it drive down fees significantly? Doubt it, there's not enough pressure on them, and the industry is resistant to change. Firms don't want to make a big deal about using it because clients will then ask why they're not getting a discount. Will it empower younger lawyers? Not as much as I'd like. I'm very fortunate to have an employer that lets me use Claude Code for my work (to a limited extent). I think for 99.9% of lawyers it's not an option available to them, through a mix of concerns around AI usage and concerned IT departments. There could be great benefits, but it would rely on having to break out of traditional private practice which would make it difficult to get enough work. I think in the near term LLMs will have much the same impact on the legal industry as it has on the software industry.

seer 24d ago

But isn’t “writing a linter for contracts” the thing we need and probably would solve? Coding agents are great because they have compilers, linters, test cases etc to ground themselves in. With tools like OKF I’m sure most knowledge work would be distilled to its core data - it’s AST if you will, and then allow models to guard against hallucinations. Checking if a case law exists is a tool call, you can demand provenance, it’s all _buildable_. Hallucinations are “solvable” this way, so the rest is just time and adoption…

bostik 24d ago

> concerns around data residency Funny you should say that. I read somewhere (not on HN, but I think it was a post linked from here) that a number of law firms who deal with extremely sensitive documents have started buying amped-up Macbooks with 512GB of memory to be able to run local models. These are businesses who literally - and for once this word fits - cannot afford to let some of those documents get anywhere outside their corporate walls.

senordevnyc 24d ago
Macbook Pros max out at 128GB
- bostik 23d ago
  Turns out the books do, studio setups can go up to 512GB. There was a thread earlier this year about the option getting pulled from selection (https://news.ycombinator.com/item?id=47296302 https://news.ycombinator.com/item?id=47296302) but I'd guess you can still top the thing up yourself.

algo_trader 24d ago

>my experience (as a commercial lawyer) How do you think the industry will respond to contract clause slop ?? I can foresee each side inserting 100s of innocent clauses with minute dependencies that provide hidden advanatges

ray_kay777 24d ago
Good question - but the state of commercial contracts (pre-AI slop) is horrifically dysfunctional, inefficient, poorly thought through and inaccessible to the average person, so the bar is quite low. I think there will be some new malicious behaviours like that to keep an eye out for but I haven't seen them in practice (yet).