5 ms·
Mistral 7B has a context window of 32k. I use it in production for medical summarization tasks supporting appeals and physician advisory services.
by pilotneko 2y ago
Mistral 7B has a context window of 32k. I use it in production for medical summarization tasks supporting appeals and physician advisory services.
- jiggawatts 2y agoWhy are you confident to use such a tiny model on something so critical? I would never use anything smaller than a 70B model for anything even vaguely medical related!
- N2yhWNXQN3k9 2y agoIts summarisation, who cares if its right as long as you feel confident after reading it? /s. In my experience, even GPT-4o is terrible at revealing information from things longer than a few pages. It might be an issue with dimensionality reduction in general though. If you think about it, you can't really take away much of what is contained within any given amount of text with text, unless the source was produced extremely inefficiently. Producing outlines or maybe a form of abstract, it seems to be okay at, but you would never really know where it fails unless you read the entirety of the source text first to begin with. IMO, its not worth risking unless you plan to read the source anyway or its not really important.
- someothherguyy 2y agoTry to walk through a Wikipedia article having an LLM summarize every few paragraphs, its often wildly inaccurate.
- add-sub-mul-div 2y agoA key thing to remember in this specific moment in history is that the vast majority of people will be as lazy as they can get away with being. People want to leave work and go home and if an LLM lets them do that faster, who cares if it's accurate? It can absorb the blame.
- bigstrat2003 2y ago> Why are you confident to use such a tiny model on something so critical? Don't use any LLM for something critical. They can't be trusted innately due to their design, why on earth would you use them for something where you need reliability?
- mdaniel 2y ago> why on earth would you use them for something where you need reliability? As very best I can tell the answer is "because bags of cash"
- pilotneko 2y agoI don't use Mistral 7B alone, this is just a component in a RAG-based system. A system that is 1) not clinical facing, 2) not used in clinical decision making, 3) provides in-line references sources for end users to validate information, and 4) is inherently human-in-the-loop.
- HaZeust 2y agoHope your prod is being done currently in this political... Climate.