6 ms·

Show me the use cases you have supported in production. Then I might read all the 30 pages praising the dozens (soon to be hundreds?) of “best practices” to bui

by dbs 2y ago

Show me the use cases you have supported in production. Then I might read all the 30 pages praising the dozens (soon to be hundreds?) of “best practices” to build LLMs.

joe_the_user 2y ago
I have a friend who uses ChatGPT for writing quick policy statement for her clients (mostly schools). I have a friend who uses it to create images and descriptions for DnD adventures. LLMs have uses. The problem I see is, who can an "application" be anything but a little window onto the base abilities of ChatGPT and so effectively offers nothing more to an end-user. The final result still have to be checked and regular end-users have to do their own prompt. Edit: Also, I should also say that anyone who's designing LLM apps that, rather than being end-user tools, are effectively gate keepers to getting action or "a human" from a company deserves a big "f* you" 'cause that approach is evil.
harrisoned 2y ago
It certainly has use cases, just not as many as the hype lead people to believe. For me: -Regex expressions: ChatGPT is the best multi-million regex parser to date. -Grammar and semantic check: It's a very good revision tool, helped me a lot of times, specially when writing in non-native languages. -Artwork inspiration: Not only for visual inspiration, in the case of image generators, but descriptive as well. The verbosity of some LLMs can help describe things in more detail than a person would. -General coding: While your mileage may vary on that one, it has helped me a lot at work building stuff on languages i'm not very familiar with. Just snippets, nothing big.
- int_19h 2y ago
  GPT-4 has amazing translation capabilities, too. Actually usable for long conversations.
thallium205 2y ago
We have a company mail, fax, and phone room that receives thousands of pages a day that now sorts, categorizes, and extracts useful information from them all in a completely automated way by LLMs. Several FTEs have been reassigned elsewhere as a result.
robbiemitchell 2y ago
Processing high volumes of unstructured data (text)… we’re using a STAG architecture. - Generate targeted LLM micro summaries of every record (ticket, call, etc.) continually - Use layers of regex, semantic embeddings, and scoring enrichments to identify report rows (pivots on aggregates) worth attention, running on a schedule - Proactively explain each report row by identifying what’s unusual about it and LLM summarizing a subset of the microsummaries. - Push the result to webhook Lack of JSON schema restriction is a significant barrier to entry on hooking LLMs up to a multi step process. Another is preventing LLMs from adding intro or conclusion text.
- BoorishBears 2y ago
  > Lack of JSON schema restriction is a significant barrier to entry on hooking LLMs up to a multi step process. How are you struggling with this, let alone as a significant barrier? JSON adherence with a well thought out schema hasn't been a worry between improved model performance and various grammar based constraint systems in a while. > Another is preventing LLMs from adding intro or conclusion text. Also trivial to work around by pre-filling and stop tokens, or just extremely basic text parsing. Also would recommend writing out Stream-Triggered Augmented Generation since the term is so barely used it might as well be made up from the POV of someone trying to understand the comment
  - robbiemitchell 2y ago
    Asking even a top-notch LLM to output well formed JSON simply fails sometimes. And when you’re running LLMs at high volume in the background, you can’t use the best available until the last mile. You work around it with post-processing and retries. But it’s still a bit brittle given how much stuff happens downstream without supervision.
    - fancy_pantser 2y ago
      Constrained output with GBNF or JSON is much more efficient and less error-prone. I hope nobody outside of hobby projects is still using error/retry loops.
fnordpiglet 2y ago
We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to direct quantitative reasoning or structured workflows. Double digit percents of our growth in the last year are entirely due to them. The biggest challenge is tool chain, limits on inference capacity, and developer understanding of the abilities, limits, and techniques for using LLMs effectively. I often see these messages from the community doubting the reality, but LLMs are a powerful tool in the tool chest. But I think most companies are not staffed with skilled enough engineers with a creative enough bent to really take advantage of them yet or be willing to fund basic research and from first principles toolchain creation. That’s ok. But it’s foolish to assume this is all hype like crypto was. The parallels are obvious but the foundations are different.
- mvdtnz 2y ago
  Yet another post claiming "dozens" of production use cases without listing a single one.
  - fnordpiglet 2y ago
    I’ve listed plenty in my comment history. I don’t generally feel compelled to trot them all out all the time - I don’t need to “prove” anything and if you think I’m lying that’s your choice. Finally, many of our uses are trade secrets and a significant competitive advantage so I don’t feel the need to disclose them to the world if our competitors don’t believe in the tech. We can keep eating their lunch.
    - deleted 2y ago
      [deleted]
- threeseed 2y ago
  No one is saying that all of AI is hype. It clearly isn't. But the facts are that today LLMs are not suitable for use cases that need accurate results. And there is no evidence or research that suggests this is changing anytime soon. Maybe for ever. There are very strong parallels to crypto in that (a) people are starting with the technology and trying to find problems and (b) there is a cult like atmosphere where non-believers are seen as being anti-progress and anti-technology.
obiefernandez 2y ago
The book I'm writing is almost finished and is based almost entirely on production use cases: https://leanpub.com/patterns-of-application-development-using-ai https://leanpub.com/patterns-of-application-development-usin...
cqqxo4zV46cp 2y ago
Or maybe they could choose to focus their attention on people that aren’t needlessly aggressive and adversarial.
mloncode 2y ago
Hi, Hamel here. I'm one of the co-authors. I'm an independent consultant and not all clients allow me to talk about their work. However, I have two that do, which I've discussed in the article. These are two production use cases that I have supported (which again, are explicitly mentioned in the article): 1. https://www.honeycomb.io/blog/introducing-query-assistant https://www.honeycomb.io/blog/introducing-query-assistant 2. https://www.youtube.com/watch?v=B_DMMlDuJB0 https://www.youtube.com/watch?v=B_DMMlDuJB0 Other co-authors have worked on significant bodies of work: Bryan Bischoff lead the creation of Magic in Hex: https://www.latent.space/p/bryan-bischof https://www.latent.space/p/bryan-bischof Jason Liu created the most popular OSS libraries for structured data called instructor https://github.com/jxnl/instructor https://github.com/jxnl/instructor, and works with some of the leading companies in the space like Limitless and Raycast (https://jxnl.co/services/#current-and-past-clients https://jxnl.co/services/#current-and-past-clients) Eugene Yan works with LLMs extensively at Amazon and uses that to inform his writing: https://eugeneyan.com/writing/ https://eugeneyan.com/writing/ (However he isn't allowed to share specifics about Amazon) I believe you might find these worth looking at.
- anon373839 2y ago
  I know it’s a snarky comment you responded to, but I’m glad you did. Those are great resources, as is your excellent article. Thanks for posting!
hubraumhugo 2y ago
I think it comes down to relatively unexciting use cases that have a high business impact (process automation, RPA, data analysis), not fancy chatbots or generative art. For example, we focused on the boring and hard task of web data extraction. Traditional web scraping is labor-intensive, error-prone, and requires constant updates to handle website changes. It's repetitive and tedious, but couldn't be automated due to the high data diversity and many edge cases. This required a combination of rule-based tools, developers, and constant maintenance. We're now using LLMs to generate web scrapers and data transformation steps on the fly that adapt to website changes, automating the full process end-to-end.
bbischof 2y ago
Hello, it’s Bryan, an author on this piece. I’d you’re interested in using one of the LLM-applications I have in prod, check out https://hex.tech/product/magic-ai/ https://hex.tech/product/magic-ai/ It has a free limit every month to give it a try and see how you like it. If you have feedback after using it, we’re always very interested to hear from users.
deleted 2y ago
[deleted]

by dbs 2y ago

Show me the use cases you have supported in production. Then I might read all the 30 pages praising the dozens (soon to be hundreds?) of “best practices” to build LLMs.

joe_the_user 2y ago

I have a friend who uses ChatGPT for writing quick policy statement for her clients (mostly schools). I have a friend who uses it to create images and descriptions for DnD adventures. LLMs have uses. The problem I see is, who can an "application" be anything but a little window onto the base abilities of ChatGPT and so effectively offers nothing more to an end-user. The final result still have to be checked and regular end-users have to do their own prompt. Edit: Also, I should also say that anyone who's designing LLM apps that, rather than being end-user tools, are effectively gate keepers to getting action or "a human" from a company deserves a big "f* you" 'cause that approach is evil.

harrisoned 2y ago

It certainly has use cases, just not as many as the hype lead people to believe. For me: -Regex expressions: ChatGPT is the best multi-million regex parser to date. -Grammar and semantic check: It's a very good revision tool, helped me a lot of times, specially when writing in non-native languages. -Artwork inspiration: Not only for visual inspiration, in the case of image generators, but descriptive as well. The verbosity of some LLMs can help describe things in more detail than a person would. -General coding: While your mileage may vary on that one, it has helped me a lot at work building stuff on languages i'm not very familiar with. Just snippets, nothing big.

int_19h 2y ago
GPT-4 has amazing translation capabilities, too. Actually usable for long conversations.

thallium205 2y ago

We have a company mail, fax, and phone room that receives thousands of pages a day that now sorts, categorizes, and extracts useful information from them all in a completely automated way by LLMs. Several FTEs have been reassigned elsewhere as a result.

robbiemitchell 2y ago

Processing high volumes of unstructured data (text)… we’re using a STAG architecture. - Generate targeted LLM micro summaries of every record (ticket, call, etc.) continually - Use layers of regex, semantic embeddings, and scoring enrichments to identify report rows (pivots on aggregates) worth attention, running on a schedule - Proactively explain each report row by identifying what’s unusual about it and LLM summarizing a subset of the microsummaries. - Push the result to webhook Lack of JSON schema restriction is a significant barrier to entry on hooking LLMs up to a multi step process. Another is preventing LLMs from adding intro or conclusion text.

BoorishBears 2y ago
> Lack of JSON schema restriction is a significant barrier to entry on hooking LLMs up to a multi step process. How are you struggling with this, let alone as a significant barrier? JSON adherence with a well thought out schema hasn't been a worry between improved model performance and various grammar based constraint systems in a while. > Another is preventing LLMs from adding intro or conclusion text. Also trivial to work around by pre-filling and stop tokens, or just extremely basic text parsing. Also would recommend writing out Stream-Triggered Augmented Generation since the term is so barely used it might as well be made up from the POV of someone trying to understand the comment
- robbiemitchell 2y ago
  Asking even a top-notch LLM to output well formed JSON simply fails sometimes. And when you’re running LLMs at high volume in the background, you can’t use the best available until the last mile. You work around it with post-processing and retries. But it’s still a bit brittle given how much stuff happens downstream without supervision.
  - fancy_pantser 2y ago
    Constrained output with GBNF or JSON is much more efficient and less error-prone. I hope nobody outside of hobby projects is still using error/retry loops.

fnordpiglet 2y ago

We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to direct quantitative reasoning or structured workflows. Double digit percents of our growth in the last year are entirely due to them. The biggest challenge is tool chain, limits on inference capacity, and developer understanding of the abilities, limits, and techniques for using LLMs effectively. I often see these messages from the community doubting the reality, but LLMs are a powerful tool in the tool chest. But I think most companies are not staffed with skilled enough engineers with a creative enough bent to really take advantage of them yet or be willing to fund basic research and from first principles toolchain creation. That’s ok. But it’s foolish to assume this is all hype like crypto was. The parallels are obvious but the foundations are different.

mvdtnz 2y ago
Yet another post claiming "dozens" of production use cases without listing a single one.
- fnordpiglet 2y ago
  I’ve listed plenty in my comment history. I don’t generally feel compelled to trot them all out all the time - I don’t need to “prove” anything and if you think I’m lying that’s your choice. Finally, many of our uses are trade secrets and a significant competitive advantage so I don’t feel the need to disclose them to the world if our competitors don’t believe in the tech. We can keep eating their lunch.
  - deleted 2y ago
    [deleted]
threeseed 2y ago
No one is saying that all of AI is hype. It clearly isn't. But the facts are that today LLMs are not suitable for use cases that need accurate results. And there is no evidence or research that suggests this is changing anytime soon. Maybe for ever. There are very strong parallels to crypto in that (a) people are starting with the technology and trying to find problems and (b) there is a cult like atmosphere where non-believers are seen as being anti-progress and anti-technology.

obiefernandez 2y ago

The book I'm writing is almost finished and is based almost entirely on production use cases: https://leanpub.com/patterns-of-application-development-using-ai https://leanpub.com/patterns-of-application-development-usin...

cqqxo4zV46cp 2y ago

Or maybe they could choose to focus their attention on people that aren’t needlessly aggressive and adversarial.

mloncode 2y ago

Hi, Hamel here. I'm one of the co-authors. I'm an independent consultant and not all clients allow me to talk about their work. However, I have two that do, which I've discussed in the article. These are two production use cases that I have supported (which again, are explicitly mentioned in the article): 1. https://www.honeycomb.io/blog/introducing-query-assistant https://www.honeycomb.io/blog/introducing-query-assistant 2. https://www.youtube.com/watch?v=B_DMMlDuJB0 https://www.youtube.com/watch?v=B_DMMlDuJB0 Other co-authors have worked on significant bodies of work: Bryan Bischoff lead the creation of Magic in Hex: https://www.latent.space/p/bryan-bischof https://www.latent.space/p/bryan-bischof Jason Liu created the most popular OSS libraries for structured data called instructor https://github.com/jxnl/instructor https://github.com/jxnl/instructor, and works with some of the leading companies in the space like Limitless and Raycast (https://jxnl.co/services/#current-and-past-clients https://jxnl.co/services/#current-and-past-clients) Eugene Yan works with LLMs extensively at Amazon and uses that to inform his writing: https://eugeneyan.com/writing/ https://eugeneyan.com/writing/ (However he isn't allowed to share specifics about Amazon) I believe you might find these worth looking at.

anon373839 2y ago
I know it’s a snarky comment you responded to, but I’m glad you did. Those are great resources, as is your excellent article. Thanks for posting!

hubraumhugo 2y ago

I think it comes down to relatively unexciting use cases that have a high business impact (process automation, RPA, data analysis), not fancy chatbots or generative art. For example, we focused on the boring and hard task of web data extraction. Traditional web scraping is labor-intensive, error-prone, and requires constant updates to handle website changes. It's repetitive and tedious, but couldn't be automated due to the high data diversity and many edge cases. This required a combination of rule-based tools, developers, and constant maintenance. We're now using LLMs to generate web scrapers and data transformation steps on the fly that adapt to website changes, automating the full process end-to-end.

bbischof 2y ago

Hello, it’s Bryan, an author on this piece. I’d you’re interested in using one of the LLM-applications I have in prod, check out https://hex.tech/product/magic-ai/ https://hex.tech/product/magic-ai/ It has a free limit every month to give it a try and see how you like it. If you have feedback after using it, we’re always very interested to hear from users.

deleted 2y ago

[deleted]