8 ms·

No complaint. It's more of a warning about how the main players (OpenAI, LangChain) share notebooks and cookbooks that illustrate how to make the LLMs "query" t

by diego 3y ago

No complaint. It's more of a warning about how the main players (OpenAI, LangChain) share notebooks and cookbooks that illustrate how to make the LLMs "query" the databases. At the very least one would expect some language telling people to not do that in production. And it's not unique to SQL, this is just an extreme example.

wpride 3y ago
Maybe "complaint" was the wrong word but I disagree with the conclusion that LLMs are "not for trustworthy production systems" for the reasons I stated. Full disclosure, I wrote a blog post called "Text to SQL in Production." Maybe I should add a follow-up covering our guardrails. I agree that they are necessary. https://canvasapp.com/blog/text-to-sql-in-production https://canvasapp.com/blog/text-to-sql-in-production
skissane 3y ago
> At the very least one would expect some language telling people to not do that in production. And it's not unique to SQL, this is just an extreme example. In professional communication, is it necessary to repeat the obvious all the time? Does an article in a medical journal or a law journal need to explicitly remind its readers of 101 level stuff? If an unqualified person reads the article, misinterprets it because they don’t understand the basics of the discipline, and causes some harm as a result-how is that the responsibility of the authors of the article? Why should software engineering be any different?
- bluepod4 3y ago
  > In professional communication, is it necessary to repeat the obvious all the time? Based on the “repeat” dev articles I’ve seen on HN over the many years and the “repeat” mistakes replicated in the actual workplace, I think it is necessary. > Why should software engineering be any different? I don’t think it is. But also see my point below. I understand the example you were trying to use but it wasn’t very effective. Dev blogs are not equivalent to medical or law journals in many ways that I don’t need to list. Academic computer science white papers are a bit closer. Thinking about this more, in my experience and across multiple fields, I always see a phenomenon where either colleagues/classmates/whoever reference a _popular_ but _problematic_ resource which leads to a shitshow.
  - skissane 3y ago
    > Dev blogs are not equivalent to medical or law journals in many ways that I don’t need to list. Academic computer science white papers are a bit closer. Okay, there are law blogs and medicine blogs too, which are directly comparable to dev blogs. And by that I mean blogs targeted at legal and medical professionals, not blogs on those topics targeted at consumers. For example, BMJ's Frontline Gastroenterology blog [0], whose target audience is practicing and trainee gastroenterologists, and its authors write for their target audience – it is public and anyone can read it, but I don't think the authors spend too much time worrying "what if an unqualified person reads this and misinterprets it due to a lack of basic medical knowledge?" Or similarly, consider Opinio Juris, the most popular international law blog on the Internet. When a blog post contains the sentence "As most readers will know, lex specialis was created by the International Court of Justice in the Nuclear Weapons Case, to try to explain the relationship between international humanitarian law (IHL) and international human rights law (IHRL)", [1] you know you are not reading something aimed at a general audience. [0] https://blogs.bmj.com/fg/ https://blogs.bmj.com/fg/ [1] http://opiniojuris.org/2020/01/13/the-soleimani-case-and-the-last-nail-in-the-lex-specialis-coffin/ http://opiniojuris.org/2020/01/13/the-soleimani-case-and-the...
    - bluepod4 3y ago
      > but I don't think the authors spend too much time worrying "what if an unqualified person reads this and misinterprets it due to a lack of basic medical knowledge?" 1) You don’t sound too sure about this. Your previous comment sounded like speculation also. Do you actually read these blogs and/or journals? 2) Again, you’re making comparisons that aren’t equivalent. Your argument fails when you replace “unqualified person” with “unqualified target person”. My pizza delivery driver is not reading dev blogs. The junior and senior engineers on my team over the years who passed 5 rounds of interviews yet still make simple but devastating mistakes are reading these blogs. > lex specialis 1) In your previous comment, you said that medical and law journals _don’t_ explain every basic little thing. And now you provided a quote where the law blog is explicitly explaining a very basic thing even to their _qualified target audience_. If “most readers” already know something, then what’s the point of re-explaining it? You’re proving my point instead. 2) Another comparison that isn’t equivalent. Even if an “unqualified” person were to read a _professional_ law or medical blog/journal, what’s the worst that could happen? Nothing. The answer to that question above will definitely change if we’re talking about _nonprofessional_ content (e.g. TikTok law and medical advice). Frankly, more dev blogs veer towards the “unprofessional” side than “professional”.

by diego 3y ago

wpride 3y ago

Maybe "complaint" was the wrong word but I disagree with the conclusion that LLMs are "not for trustworthy production systems" for the reasons I stated. Full disclosure, I wrote a blog post called "Text to SQL in Production." Maybe I should add a follow-up covering our guardrails. I agree that they are necessary. https://canvasapp.com/blog/text-to-sql-in-production https://canvasapp.com/blog/text-to-sql-in-production

skissane 3y ago

> At the very least one would expect some language telling people to not do that in production. And it's not unique to SQL, this is just an extreme example. In professional communication, is it necessary to repeat the obvious all the time? Does an article in a medical journal or a law journal need to explicitly remind its readers of 101 level stuff? If an unqualified person reads the article, misinterprets it because they don’t understand the basics of the discipline, and causes some harm as a result-how is that the responsibility of the authors of the article? Why should software engineering be any different?

bluepod4 3y ago
> In professional communication, is it necessary to repeat the obvious all the time? Based on the “repeat” dev articles I’ve seen on HN over the many years and the “repeat” mistakes replicated in the actual workplace, I think it is necessary. > Why should software engineering be any different? I don’t think it is. But also see my point below. I understand the example you were trying to use but it wasn’t very effective. Dev blogs are not equivalent to medical or law journals in many ways that I don’t need to list. Academic computer science white papers are a bit closer. Thinking about this more, in my experience and across multiple fields, I always see a phenomenon where either colleagues/classmates/whoever reference a _popular_ but _problematic_ resource which leads to a shitshow.
- skissane 3y ago
  > Dev blogs are not equivalent to medical or law journals in many ways that I don’t need to list. Academic computer science white papers are a bit closer. Okay, there are law blogs and medicine blogs too, which are directly comparable to dev blogs. And by that I mean blogs targeted at legal and medical professionals, not blogs on those topics targeted at consumers. For example, BMJ's Frontline Gastroenterology blog [0], whose target audience is practicing and trainee gastroenterologists, and its authors write for their target audience – it is public and anyone can read it, but I don't think the authors spend too much time worrying "what if an unqualified person reads this and misinterprets it due to a lack of basic medical knowledge?" Or similarly, consider Opinio Juris, the most popular international law blog on the Internet. When a blog post contains the sentence "As most readers will know, lex specialis was created by the International Court of Justice in the Nuclear Weapons Case, to try to explain the relationship between international humanitarian law (IHL) and international human rights law (IHRL)", [1] you know you are not reading something aimed at a general audience. [0] https://blogs.bmj.com/fg/ https://blogs.bmj.com/fg/ [1] http://opiniojuris.org/2020/01/13/the-soleimani-case-and-the-last-nail-in-the-lex-specialis-coffin/ http://opiniojuris.org/2020/01/13/the-soleimani-case-and-the...
  - bluepod4 3y ago
    > but I don't think the authors spend too much time worrying "what if an unqualified person reads this and misinterprets it due to a lack of basic medical knowledge?" 1) You don’t sound too sure about this. Your previous comment sounded like speculation also. Do you actually read these blogs and/or journals? 2) Again, you’re making comparisons that aren’t equivalent. Your argument fails when you replace “unqualified person” with “unqualified target person”. My pizza delivery driver is not reading dev blogs. The junior and senior engineers on my team over the years who passed 5 rounds of interviews yet still make simple but devastating mistakes are reading these blogs. > lex specialis 1) In your previous comment, you said that medical and law journals _don’t_ explain every basic little thing. And now you provided a quote where the law blog is explicitly explaining a very basic thing even to their _qualified target audience_. If “most readers” already know something, then what’s the point of re-explaining it? You’re proving my point instead. 2) Another comparison that isn’t equivalent. Even if an “unqualified” person were to read a _professional_ law or medical blog/journal, what’s the worst that could happen? Nothing. The answer to that question above will definitely change if we’re talking about _nonprofessional_ content (e.g. TikTok law and medical advice). Frankly, more dev blogs veer towards the “unprofessional” side than “professional”.