12 ms·

How does what you've described solve the coffee/espresso problem? You can't query SQL such that records like 'espresso' return coffee?

brudgers 1y ago
Wouldn’t a beverage LLM would already “know” espresso is coffee?
- muzani 1y ago
  Yup, that's exactly what parent comment is saying. Let's say your beverage LLM is there to recommend drinks. You once said "I hate espresso" or even something like "I don't take caffeine" at one point to the LLM. Before recommending coffee, Beverage LLM might do a vector search for "coffee" and it would match up to these phrases. Then the LLM processes the message history to figure out whether this person likes or dislikes coffee. But searching SQL for `LIKE '%coffee%'` won't match with any of these.
  - brudgers 1y ago
    I think the problem being addressed is A. Last month user fd8120113 said “I don’t like coffee” B. Today they are back for another beverage recommendation SQL is the place to store the relevant fact about user fd8120113 so that you can retrieve it into the LLM prompt to make a new beverage recommendation, today. It’s addressing the “how many fucking times do I fucking need to tell you I don’t like fucking coffee” problem, not the word salad problem. The ggp comment is strawmanning.
    - shepardrtc 1y ago
      Right but if the user hates espresso but loves black coffee, how do you properly store that in SQL? "I hate espresso" "I love coffee" What if the SQL query only retrieves the first one?
      - brudgers 1y ago
        Good queries are hard. Database design is hard. System architecture is hard. My comment described the problem. The solution is left as an exercise for the reader. Keep in mind that people change their minds, misspeak, and use words in peculiar ways.
  - 9rx 1y ago
    If an LLM understands that coffee and expresso are both relevant, like the earlier comment suggests, why wouldn't it understand that it should search for something like `foo LIKE '%coffee%' OR foo LIKE '%expresso%'`? In fact, this is what ChatGPT came up with: SELECT * FROM documents WHERE text ILIKE '%coffee%' OR text ILIKE '%espresso%' OR text ILIKE '%latte%' OR text ILIKE '%cappuccino%' OR text ILIKE '%americano%' OR text ILIKE '%mocha%' OR text ILIKE '%macchiato%'; (I gave it no direction as to the structure of the DB, but it shouldn't be terribly difficult to adapt to your exact schema)
    - jimbokun 1y ago
      You are slowly approaching the vector solution. There are an unlimited number of items to add to your “like” clauses. Vector search allows you to efficiently query for all of them at once.
      - 9rx 1y ago
        The handwavvy assertion was that relational database solutions[1] work better in practice. [1] Despite also somehow supporting MongoDB...
    - mr_toad 1y ago
      Implementations that use vector database do not use LLMs to generate queries against those databases. That would be incredibly expensive and slow (and yes there is a certain irony there). Main advantages of a vector lookup are built-in fuzzy matching and the potential to keep a large amount of documentation in memory for low latency. I can’t see an RDMS being ideal for either. LLMs are slow enough already, adding a slow document lookup isn’t going to help.
      - 9rx 1y ago
        The main disadvantage of vector lookup, allegedly, is that it doesn't work as well in practice. Did you, uh, forget to read the thread?
  - esafak 1y ago
    The negation part is a query understanding problem. https://en.wikipedia.org/wiki/Query_understanding https://en.wikipedia.org/wiki/Query_understanding
  - sdesol 1y ago
    I haven't looked at the code, but it might do what I do with my chat app which is talked about at https://github.com/gitsense/chat/blob/main/packages/chat/widgets/app/components/search/docs/gitsense/search-strategy.md https://github.com/gitsense/chat/blob/main/packages/chat/wid... The basic idea is, you don't search for a single term but rather you search for many. Depending on the instructions provided in the "Query Construction" stage, you may end up with a very high level search term like beverage or you may end up with terms like 'hot-drinks', 'code-drinks', etc. Once you have the query, you can do a "Broad Search" which returns an overview of the message and from there the LLM can determine which messages it should analyze further if required. Edit. I should add, this search strategy will only work well if you have a post message process. For example, after every message save/upddate, you have the LLM generate an overview. These are my instructions for my tiny overview https://github.com/gitsense/chat/blob/main/data/analyze/tiny-overview/file-content/default/1.md https://github.com/gitsense/chat/blob/main/data/analyze/tiny... that is focused on generating the purpose and keywords that can be used to help the LLM define search terms.
    - adastra22 1y ago
      That’s going to be incredibly fragile. You could fix it by giving the query term a bunch of different scores, e.g. its caffeine-ness, bitterness, etc. and then doing a likeness search across these many dimensions. That would be much less fragile. And now you’ve reinvented vector embeddings.
      - sdesol 1y ago
        You could instruct the LLM to classify messages with high level tags like for coffee, drinks, etc. always include beverage. Given how fast interference has become and given current supported context window sizes for most SOTA models, I think summarizing and having the LLM decide what is relevant is not that fragile at all for most use cases. This is what I do with my analyzers which I talk about at https://github.com/gitsense/chat/blob/main/packages/chat/widgets/app/components/chat-builder/trees/help/documentation/understanding-your-personalized-ai-search-assistant/1.md https://github.com/gitsense/chat/blob/main/packages/chat/wid...

brudgers 1y ago

Wouldn’t a beverage LLM would already “know” espresso is coffee?

muzani 1y ago
Yup, that's exactly what parent comment is saying. Let's say your beverage LLM is there to recommend drinks. You once said "I hate espresso" or even something like "I don't take caffeine" at one point to the LLM. Before recommending coffee, Beverage LLM might do a vector search for "coffee" and it would match up to these phrases. Then the LLM processes the message history to figure out whether this person likes or dislikes coffee. But searching SQL for `LIKE '%coffee%'` won't match with any of these.
- brudgers 1y ago
  I think the problem being addressed is A. Last month user fd8120113 said “I don’t like coffee” B. Today they are back for another beverage recommendation SQL is the place to store the relevant fact about user fd8120113 so that you can retrieve it into the LLM prompt to make a new beverage recommendation, today. It’s addressing the “how many fucking times do I fucking need to tell you I don’t like fucking coffee” problem, not the word salad problem. The ggp comment is strawmanning.
  - shepardrtc 1y ago
    Right but if the user hates espresso but loves black coffee, how do you properly store that in SQL? "I hate espresso" "I love coffee" What if the SQL query only retrieves the first one?
    - brudgers 1y ago
      Good queries are hard. Database design is hard. System architecture is hard. My comment described the problem. The solution is left as an exercise for the reader. Keep in mind that people change their minds, misspeak, and use words in peculiar ways.
- 9rx 1y ago
  If an LLM understands that coffee and expresso are both relevant, like the earlier comment suggests, why wouldn't it understand that it should search for something like `foo LIKE '%coffee%' OR foo LIKE '%expresso%'`? In fact, this is what ChatGPT came up with: SELECT * FROM documents WHERE text ILIKE '%coffee%' OR text ILIKE '%espresso%' OR text ILIKE '%latte%' OR text ILIKE '%cappuccino%' OR text ILIKE '%americano%' OR text ILIKE '%mocha%' OR text ILIKE '%macchiato%'; (I gave it no direction as to the structure of the DB, but it shouldn't be terribly difficult to adapt to your exact schema)
  - jimbokun 1y ago
    You are slowly approaching the vector solution. There are an unlimited number of items to add to your “like” clauses. Vector search allows you to efficiently query for all of them at once.
    - 9rx 1y ago
      The handwavvy assertion was that relational database solutions[1] work better in practice. [1] Despite also somehow supporting MongoDB...
  - mr_toad 1y ago
    Implementations that use vector database do not use LLMs to generate queries against those databases. That would be incredibly expensive and slow (and yes there is a certain irony there). Main advantages of a vector lookup are built-in fuzzy matching and the potential to keep a large amount of documentation in memory for low latency. I can’t see an RDMS being ideal for either. LLMs are slow enough already, adding a slow document lookup isn’t going to help.
    - 9rx 1y ago
      The main disadvantage of vector lookup, allegedly, is that it doesn't work as well in practice. Did you, uh, forget to read the thread?
- esafak 1y ago
  The negation part is a query understanding problem. https://en.wikipedia.org/wiki/Query_understanding https://en.wikipedia.org/wiki/Query_understanding
- sdesol 1y ago
  I haven't looked at the code, but it might do what I do with my chat app which is talked about at https://github.com/gitsense/chat/blob/main/packages/chat/widgets/app/components/search/docs/gitsense/search-strategy.md https://github.com/gitsense/chat/blob/main/packages/chat/wid... The basic idea is, you don't search for a single term but rather you search for many. Depending on the instructions provided in the "Query Construction" stage, you may end up with a very high level search term like beverage or you may end up with terms like 'hot-drinks', 'code-drinks', etc. Once you have the query, you can do a "Broad Search" which returns an overview of the message and from there the LLM can determine which messages it should analyze further if required. Edit. I should add, this search strategy will only work well if you have a post message process. For example, after every message save/upddate, you have the LLM generate an overview. These are my instructions for my tiny overview https://github.com/gitsense/chat/blob/main/data/analyze/tiny-overview/file-content/default/1.md https://github.com/gitsense/chat/blob/main/data/analyze/tiny... that is focused on generating the purpose and keywords that can be used to help the LLM define search terms.
  - adastra22 1y ago
    That’s going to be incredibly fragile. You could fix it by giving the query term a bunch of different scores, e.g. its caffeine-ness, bitterness, etc. and then doing a likeness search across these many dimensions. That would be much less fragile. And now you’ve reinvented vector embeddings.
    - sdesol 1y ago
      You could instruct the LLM to classify messages with high level tags like for coffee, drinks, etc. always include beverage. Given how fast interference has become and given current supported context window sizes for most SOTA models, I think summarizing and having the LLM decide what is relevant is not that fragile at all for most use cases. This is what I do with my analyzers which I talk about at https://github.com/gitsense/chat/blob/main/packages/chat/widgets/app/components/chat-builder/trees/help/documentation/understanding-your-personalized-ai-search-assistant/1.md https://github.com/gitsense/chat/blob/main/packages/chat/wid...