ddematheu

4 ms·

1.
by ddematheu 2y ago
Interesting performance with GPT 3.5, what does performance on Llama look like? What about smaller models like Llama 3.1?
2.
by ddematheu 3y ago
I don't disagree with all your points. That said, what we have built has proven useful for us as we have built pipelines for customers and think it might be useful for others. Probably the main point I disagree with you is that RAG is
3.
by ddematheu 3y ago
LlamaIndex is pretty awesome. There are a couple areas where we think we are driving some differentiation. 1. The management of metadata as a first class citizen. This includes capturing metadata at every stage of the pipeline. 2. Be infra
4.
by ddematheu 3y ago
It is dangerous, part of the reason that we haven't productized that further. One of the ideas we had to productize the capabilities further was to leverage edge / lambda functions to compartmentalize the code generated. (Plus it
5.
by ddematheu 3y ago
Yeah, we were playing around with doing some semantic chunking. Works okay for some use cases. We have some ideas to go further on that. Generally we have found that recursive chunking and character chunking tend to be short sighted.
6.
by ddematheu 3y ago
Haven't connected.
7.
by ddematheu 3y ago
Co-founder here :) Today, it is mostly about convenience. We provide abstractions in the form of a pipeline that encompasses a data source, embed and sink definition. This means that you don't have to think about embedding your query o
8.
by ddematheu 3y ago
Lies or not lies, the point was the train on the authentic message that the candidate wanted to provide. Try to be as unbiased as possible.
9.
by ddematheu 3y ago
How real-time is it? Just app or API?
10.
Show HN: Building ElectionGPT, a RAG powered chatbot grounded on candidate data(medium.com)
1 points by ddematheu 3y ago|3 comments
11.
by ddematheu 3y ago
Some engineers find it fun, other might not. Same as everything. IMO the fun parts are actually prototyping and figuring out the right pattern I want to use for my solution. Once you have done that, scaling and dealing with robustness tends
12.
by ddematheu 3y ago
What about then sucked?
13.
by ddematheu 3y ago
Co-author of the article here. We do support updates for some sources. Deletes not yet. For some sources we do polling which is then dumped on the queues. For other we have listeners that subscribe to changes. What are the challenges you ar
14.
by ddematheu 3y ago
Through the platform (Neum AI) we support the ability to do this with Postgres, it is just a cloud platform so not a python library. Curious on what type of customization are you looking to add that you would want something like a library?
15.
by ddematheu 3y ago
To some degree. The amount of data that will be brought into search solutions will be enormous, seems like a good time to try to reimagine what that process might look like
16.
by ddematheu 3y ago
Makes sense. Interesting on the fact that summaries affect quality sometimes. For synthetic data scenarios are you also thinking about synthetic queries over the data? (Try to predict which chunks might be more used than others)
17.
by ddematheu 3y ago
The queues and storage are the foundation on which some of these other integrations can be built on top. Agree fully on the need for LLMs within the pipelines to help with data analysis. Our current perspective has been on leveraging LLMs a
18.
by ddematheu 3y ago
What type of latency requirements are you dealing with? (i.e. look up time, ingestion time) Were you using postgres already or migrated data into it?
19.
by ddematheu 3y ago
Not at scale. Currently we do some extraction for metadata, but pretty simple. Doing LLM based pre-processing of each chunk like this can be quite expensive especially with billions of them. Summarizing each document before ingestion could
20.
by ddematheu 3y ago
Co-author of article here. Yeah a ton of the time and effort has gone into building robustness and observability into the process. When dealing with millions of files, a failure half way through it is imperative to be able to recover. RE: W
21.
by ddematheu 3y ago
Co-author of the article here. You are right. Retrieval accuracy is important as well. From an accuracy perspective, any tools you have found useful in helping validate retrieval accuracy? In our current architecture, all the different piec
22.
Getting started with LLMs and structured data(neum.ai)
1 points by ddematheu 3y ago|1 comments
23.
by ddematheu 3y ago
TL;DR for blog: If you are using RAG across structured data, make sure you consider the role of metadata as it is likely that not all the fields within your data carry semantic meaning and might lead to bad embeddings. Leverage hybrid searc
24.
Doing Q&A with Thousands of Documents(neum.ai)
2 points by ddematheu 3y ago|1 comments
25.
by ddematheu 3y ago
Q&A with documents is the most common use cases for LLMs today. But scaling from one or two documents to thousands can be challenging. This blog explores the challenges in scaling such solutions and what Neum AI is going to help.