7 ms·
Jina AI launches open-source 8k text embedding
- 3cats-in-a-coat 3y agoGreat company name.
- smcl 3y agoI'm gonna try to explain this because I thought the same thing, though you may enjoy it for another reason. Among Czech or other slavic software people - "jiná AI" could be like "another AI" and, to me at least, brings to mind the "yet another {thing}" naming convention (yacc = "yet another compiler compiler" for example).
- deleted 3y ago[deleted]
- burcs 3y agoThis is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?
- udev4096 3y agoIs it tho? It's not really open source if they don't give us the information regarding training datasets
- jerpint 3y agoIt definitely is open source even if they don’t disclose all details behind the training
- selcuka 3y agoHow do you define "source", then? By this logic any freely downloadable executable software (a.k.a. freeware) is also open source, even though they don't disclose all details on how to build it.
- mogwire 3y agoSource would be the way the data is produced so that you can replicate it yourself and make changes. If I hand you a beer for free that’s freeware. If I hand you the recipe and instructions to brew the beer that is open source. We muddy the waters too much lately and call “free” to use things “open source”.
- TeMPOraL 3y ago> If I hand you a beer for free that’s freeware. If I hand you the recipe and instructions to brew the beer that is open source. Yeah, but what those "open source" models are is like you handing me a bottle of beer, plus the instructions to make the glass bottle. You're open-sourcing something, just not the part that matters. It's not "open source beer", it's "beer in an open-source bottle". In the same fashion, those models aren't open source - they're closed models inside a tiny open-source inference script.
- imranhou 3y agoPerhaps one more thing that is missing in context is that I'm also getting the right to alter that beer by adding anything I like to it and redistributing it, without knowing its true recipe.
- SOLAR_FIELDS 3y agoThe very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra information from only what has been made available. If that is not possible, because the reproduction methodology is closed (a common reason, like in this case) then what has been made available is not truly open. We can sit here and technically argue whether or not the subject matter violated some arbitrary “open source” definition but it still doesn’t change the fact that it’s not truly open in spirit
- Gasp0de 3y agoThey compare it to OpenAI's ada model though, which is light-years away from ChatGPT.
- infecto 3y agoDoes that not conflate two different things though? Embedding model != LLM Model ?
- simonw 3y agoDon't confuse the current Ada embedding model the old Ada GPT3 model. It turns out OpenAI have used the name "Ada" for several very different things, purely because they went through a phase of giving everything Ada/Babbage/Curie/DaVinci names because they liked the A/B/C/D thing to indicate which of their models were largest.
- infecto 3y agoWishful thinking? Embeddings to me were never the interesting or bleeding edge thing at OpenAI. Maybe the various ada models at one point reigned supreme but there have been open-source models at the top of the leaderboard for a while and from a cost/performance perspective, often even the Bert models did a really fine job.
- omneity 3y agoImpressive work. I wonder what would be the best way to use 8k embeddings. It’s a lot of information to keep in a vector, so things like “precision” of the embedding space and its ability to distinguish very similar large documents will be key. Maybe it can be useful for coarse similarity matching, for example to detect plagiarism?
- sroussey 3y ago8K is the context length. Their vector dimension size is actual much smaller, which is great for a number of use cases, though maybe not the ones you are thinking about.
- deleted 3y ago[deleted]
- omneity 3y agoYes that’s also how I understood it. Maybe it was ambiguously expressed, but I mean “8k tokens as input is a lot of information to encode”
- andrewstuart 3y agoAnyone got links to examples of text embedding?
- BoorishBears 3y agoEasiest example is taking three words: Universe, University, College. - University and Universe are similar alphabetically. - University and College are similar in meaning. Take embeddings for those three words and `University` will be near `College`, while `Universe` will be further away, because embeddings capture meaning: University<-->College<-------------->Universe _ With old school search you'd need to handle the special case of treating University and College as similar, but embeddings already handle it. With embeddings you can do math to find how similar two results are, based on how close their vectors are. The closer the embeddings, the closer the meaning.
- osigurdson 3y agoAnother interesting point is that math can be performed on embedding vectors: emb("king") - emb("man") + emb("woman") = emb("queen").
- minimaxir 3y agoThat's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Using it for embeddings generated from LLMs or Embedding layers will not give as fun results; in practice the only thing you can do is average or cluster them.
- TeMPOraL 3y ago> That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Is it though? I thought the LLM-based embeddings are even more fun for this, as you have many more interesting directions to move in. I.e. not just: emb("king") - emb("man") + emb("woman") = emb("queen") But also e.g.: emb(<insert a couple paragraph long positive book review>) + av(sad) + bv(short) - c*v(positive) = emb(<a single paragraph, negative and depressing review>) Where a, b, c are some constants to tweak, and v(X) is a vector for quality X, which you can get by embedding a bunch of texts expressing the quality X and averaging them out (or doing some other dimensional reduction trickery). I've suggested this on HN some time ago, but only been told that I'm confused and the idea is not even wrong. But then, there was this talk on some AI conference recently[0], where the speaker demonstrated exactly this kind of latent space translations of text in a language model. -- [0] - https://www.youtube.com/watch?v=veShHxQYPzo&t=13980s https://www.youtube.com/watch?v=veShHxQYPzo&t=13980s - "The Hidden Life of Embeddings", by Linus Lee from Notion.
- Nitrolo 3y agoIs there something like oobabooga to easily run this in a click-and-run way? Where I can load up a model, a text, and ask it questions?
- brucethemoose2 3y agoiirc ooba has its own integrated vectordb called superbooga. I bet you could hack this in.
- simonw 3y agoSee my comment here: https://news.ycombinator.com/item?id=38020655 https://news.ycombinator.com/item?id=38020655 for a CLI tool that lets you do this. Note that embedding models are a different kind of thing from a Large Language Model, so it's not the kind of model you can ask questions. It's a model which can take text and turn it into an array of floating point numbers, which you can then use to implement things like semantic search and related documents. More on that here: https://simonwillison.net/2023/Oct/23/embeddings/ https://simonwillison.net/2023/Oct/23/embeddings/
- minimaxir 3y agoThe Hugging Face page for the model has a two line load-and-encode Python code demo: https://huggingface.co/jinaai/jina-embeddings-v2-base-en https://huggingface.co/jinaai/jina-embeddings-v2-base-en
- sroussey 3y agoDoes anyone know what they are using for this comparison and ranking? And where does instruct-xl stand in the mix?
- sroussey 3y agoOh duh, it’s right in the post and instructor-xl is number 9. And so many new participants now!
- sroussey 3y agoThe ranking are here: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard It’s amazing how many new and better ones there are since I last looked a few months ago. Instructor-xl was number 1, now it is number 9, and its size is more than 10x the number 2 ranked! Things move fast!
- RossBencina 3y agoSome relevant stats from the link: 8192 token input sequence length 768 embedding dimensions 0.27GB model (with 0.07GB model also available) Tokeniser: BertTokenizer [1], 30528 token vocab [2] Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. [1] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blob/main/tokenizer_config.json https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blo... [2] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blob/main/vocab.txt https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blo...
- LoganDark 3y ago> Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. Words that aren't in the vocabulary can still be represented by multiple tokens. Some models can input and output valid UTF-8 at the byte level (rather than needing a unique token for each codepoint). For example RWKV-World.
- space_fountain 3y agoA large vocabulary means less tokens are needed to represent the same information
- krasin 3y ago[dead]
- jncraton 3y agoThis is great to see. It looks like the size of the embedding vector is half the size of text-embedding-ada-002 (768 vs 1536) while providing competitive performance. This will save space in databases and make lookups somewhat faster. For those unaware, if 512 tokens of context is sufficient for your use case, there are already many options that outperform text-embedding-ada-002 on common benchmarks: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard
- minimaxir 3y agoThe 768D-sized embeddings compared to OpenAI's 1536D embeddings are actually a feature outside of index size. In my experience, OpenAI's embeddings are overspecified and do very poorly with cosine similarity out of the box as they match syntax more than semantic meaning (which is important as that's the metric for RAG). Ideally you'd want cosine similarity in the range of [-1, 1] on a variety of data but in my experience the results are [0.6, 0.8].
- karxxm 3y agoYou wrote „out of the box“, did you find a way to improve this?
- teaearlgraycold 3y agoYou can do PCA or some other dimensionality reduction technique. That’ll reduce computation and improve signal/noise ratio when comparing vectors.
- e1g 3y agoTheir OpenAI benchmark is GPT3 (text-embedding-ada-002), not GPT4.
- simonw 3y ago"text-embedding-ada-002" isn't GPT3, it's a different kind of model. Embedding models and Large Language Models aren't the same thing.
- e1g 3y agoLLMs and embedding models are certainly different, but it's a useful benchmark to calibrate expectations. OpenAI released text-embedding-ada-002 a year ago, and they describe the ada model as[1] "the original GPT-3 base model [...] capable of very simple tasks, usually the fastest model in the GPT-3 series". It's fair to expect GPT3-level results - not GPT 3.5 and certainly not open-source tiny GPT4 as some might think when they read "rivaling OpenAI". [1] https://platform.openai.com/docs/models/whisper https://platform.openai.com/docs/models/whisper
- minimaxir 3y agoWhen people talked about GPT-3 they always referred to davinci which is the largest model, not ada.
- simonw 3y agoNo, you're confusing two things here. "text-ada-001" is LLM in the GPT3 family, described as "Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost" "text-embedding-ada-002" is entirely different - that page describes it as "Our second generation embedding model, text-embedding-ada-002 is a designed to replace the previous 16 first-generation embedding models at a fraction of the cost."
- minimaxir 3y agotl;dr OpenAI is bad at product naming.
- tayo42 3y agoYou can't fine tune without using their library tied to their cloud? Did I misunderstand? Do you need fine tune?
- minimaxir 3y ago[dead]
- simonw 3y agoI just shipped a new llm-embed-jina plugin for my LLM tool which provides access to these new Jina models: https://github.com/simonw/llm-embed-jina https://github.com/simonw/llm-embed-jina Here's how to try it out. First, install LLM. Use pip or pipx or brew: brew install llm Next install the new plugin: llm install llm-embed-jina You can confirm the new models are now available to LLM by running: llm embed-models You should see a list that includes "jina-embeddings-v2-small-en" and "jina-embeddings-v2-base-en" To embed a string using the small model, run this: llm embed -m jina-embeddings-v2-small-en -c 'Hello world' That will output a JSON array of 512 floating point numbers (see my explainer here for what those are: https://simonwillison.net/2023/Oct/23/embeddings/#what-are-embeddings https://simonwillison.net/2023/Oct/23/embeddings/#what-are-e...) Embeddings are only really interesting if you store them and use them for comparisons. Here's how to use the "llm embed-multi" command to create embeddings for the 30 most recent issues in my LLM GitHub repository: curl 'https://api.github.com/repos/simonw/llm/issues?state=all&filter=all' \ | jq '[.[] | {id: .id, title: .title}]' \ | llm embed-multi -m jina-embeddings-v2-small-en jina-llm-issues - \ --store This creates a collection called "jina-llm-issues" in a default SQLite database on your machine (the path to that can be found using "llm collections path"). To search for issues in that collection with titles most similar to the term "bug": llm similar jina-llm-issues -c 'bug' Or for issues most similar to another existing issue by ID: llm similar jina-llm-issues 1922688957 Full documentation on what you can do with LLM and embeddings here: https://llm.datasette.io/en/stable/embeddings/index.html https://llm.datasette.io/en/stable/embeddings/index.html Alternative recipe - this creates embeddings for every single README.md in the current directory and its subdirectories. Run this somewhere with a node_modules folder and you should get a whole lot of interesting stuff: llm embed-multi jina-readmes \ -m jina-embeddings-v2-small-en \ --files . '**/README.md' --store Then search them like this: llm similar jina-readmes -c 'backup tools'
- X6S1x6Okd1st 3y agoThank you so much for all the work you've put into llm!
- dazzaji 3y agoExcellent! And you were just saying how risky it is to rely long-term on OpenAI text embeddings in your post on the topic. The timing for this open source option worked out nicely.
- neximo64 3y agoDoes it match OpenAI on number of params?
- minimaxir 3y agoNo one knows since OpenAI has not disclosed the number of paramerers their embeddings model uses.
- andy99 3y agoWhat is the use case for an 8k token embedding? My (somewhat limited) experience with long context models is they aren't great for RAG. I get the impression they are optimized for something else, like writing 8k+ tokens rather than synthesizing responses. Isn't the normal way of using embedding to find relevant text snippets for a RAG prompt? Where is it better to have coarser retrieval?
- kristopolous 3y agoIs this what you mean by RAG? https://www.promptingguide.ai/techniques/rag https://www.promptingguide.ai/techniques/rag?
- teaearlgraycold 3y agoYes
- simonw 3y agoI have an explanation of RAG in the context of embeddings here: https://simonwillison.net/2023/Oct/23/embeddings/#answering-questions-with-retrieval-augmented-generation https://simonwillison.net/2023/Oct/23/embeddings/#answering-...
- Grimburger 3y agoYou could just sum it up for us all rather than do a divert to your blog? It's Retrieval Augmented Generation btw. To quote: > The key idea is this: a user asks a question. You search your private documents for content that appears relevant to the question, then paste excerpts of that content into the LLM (respecting its size limit, usually between 3,000 and 6,000 words) along with the original question. > The LLM can then answer the question based on the additional content you provided.
- simonw 3y ago> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401 https://arxiv.org/abs/2005.11401
- moralestapia 3y agoAda is one of the (if not the) worst model offered by OpenAI, though ...
- simonw 3y agoYou're thinking of the old "ada" GPT-3 model - the one that was a companion to "davinci" and "babbage". I believe "text-embedding-ada-002" is entirely unrelated to those old GPT-3 models. It's a recent embedding model (released in December 2022 - https://openai.com/blog/new-and-improved-embedding-model https://openai.com/blog/new-and-improved-embedding-model ) which OpenAI claim is their best current best available embedding model. I understand your confusion: OpenAI are notoriously bad at naming things!
- moralestapia 3y agoOh, thanks for clarifying! Edit: looking at the press release, the improvement over old Ada is ... marginal? And Ada-01 is/was a poor performing model, tbh. I guess I'll have to run some tests, but at first sight it doesn't seem that wow-ey.
- LASR 3y agoSo just to be super clear, this is an embedding model. It generates no text. It’s not outputting words. Maybe I am assuming incorrectly, but I think the poor performance you are referring to is the old Ada completion model, where the output is text. That was poor indeed.
- itake 3y agoThis article is not kind to the old ada embeddings model: https://medium.com/@nils_reimers/openai-gpt-3-text-embeddings-really-a-new-state-of-the-art-in-dense-text-embeddings-6571fe3ec9d9 https://medium.com/@nils_reimers/openai-gpt-3-text-embedding... If the new ada model only has marginal improvements, it seems open source is way to go.
- Zuiii 3y agoColor me surprised! it looks like its actually open source (Apache 2.0) and not the usual false advertising by some two-faced company or institution. Links here: * https://huggingface.co/jinaai/jina-embeddings-v2-base-en https://huggingface.co/jinaai/jina-embeddings-v2-base-en * https://huggingface.co/jinaai/jina-embeddings-v2-small-en https://huggingface.co/jinaai/jina-embeddings-v2-small-en
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- nicognaw 3y agoJina AI itself is also a great framework to expose APIs from deep neural net models and deploy them to Kubernetes clusters, which I think is very promising, but they didn't get as much hype as I predicted that they deserved.
- pknerd 3y agoPardon my ignorance in advance but could it be used to "chat" with PDFs and websites? I am looking for OpenAI alternatives as I am in learning phase
- clarkmcc 3y agoCheck out my little side project for chatting with PDFs. You should be able to load most models including this one. https://github.com/clarkmcc/chitchat https://github.com/clarkmcc/chitchat
- pknerd 3y agoThis looks cool so can it be used to feed Website/Products data in CSV/JSON format and "chat" with it?
- clarkmcc 3y agoPretty much! Right now it only supports md, pdf, txt, and html, but supporting additional formats is trivial: https://github.com/clarkmcc/chitchat/blob/main/src-tauri/src/context_file.rs#L18-L25 https://github.com/clarkmcc/chitchat/blob/main/src-tauri/src....
- canadaduane 3y agoNo, this is an embedding model, not a text completion model.
- lofties 3y agoNo. “Chatting with PDFs” is (mostly) taking a users chat message, retrieve relevant content via e.g embedding search, then feed that into an LLM with a prompt that’s something along the lines of “given this information, can you answer this question”. This tool helps with embedding part. I’ve built a bunch of ”chat with your PDFs” bots, do reach out if you have any questions me at brian.jp.
- pknerd 3y ago
- marinhero 3y agoHow well do LLMS like this work with a non-English language? Or are these open source models limited to English?
- simonw 3y agoQuite a few of the top ranked models on this leaderboard are multilingual: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/BAAI/bge-large-en-v1.5 https://huggingface.co/BAAI/bge-large-en-v1.5 FlagEmbedding for example describes itself as covering Chinese and English.
- ttul 3y agoThat depends on whether the training data contained languages other than English.
- anigbrowl 3y agoStability has a Japanese port which is getting lots of work https://twitter.com/StabilityAI_JP/status/1716998578244407597 https://twitter.com/StabilityAI_JP/status/171699857824440759...
- m3at 3y agoThis is not an embedding model though. Yes you can always extract some embeddings from somewhere, but for most LLMs those won't perform well for retrieval (which makes sense as it's not what the models are optimizing for)
- anigbrowl 3y agoThis isn't an embedding model, but it is a group of people working in this general area in a language other than English. Maybe they'll get to an embedding model next?
- backendEngineer 3y agooh thank god I first read Jira...
- eshack94 3y agoYou're not the only one... glad I misread that.
- deleted 3y ago[deleted]
- nomagick 3y ago[dead]
- dcastm 3y agoI wonder how much better is this, compared to taking the average ( or some other aggregation) of embeddings with a smaller context length. Has anyone done a similar comparison?
- pietro72ohboy 3y agoThe issue with averaging is that over large inputs, it drowns out small signal. For example, there is a chance that it completely loses a reference to something made only in a single sentence somewhere in a large document.
- extasia 3y agoIs this a text encoder model, BERT style?
- Kutsuya 3y agothis is super cool! I wish there was an easy to understand and follow guide on how to make your own embedding, for llama2 for example. All I can find are various guides that already assume you know everything there is to training an embedding. I just want to make an embedding between a conversation of me and my friend and simulate talking to them. Is this a hard thing to train to begin with? If anyone knows or could help me with this, I would be very grateful!
- infecto 3y agoI will butcher this so if any experts see this please don't flame me. I think you might be conflating ideas? You could definitely fine-tune existing embedding models or train your own from scratch but the goals of embeddings models are different than a LLM conversation. Embedding models are used for things like, classifying, search, image captioning...maybe at a high level anything where you have high dimensionality that you need to condense? What you are asking for sounds like fine tuning an existing LLM...where the data will be tokenized but the outcomes are different? There is a lot of writeups on how people have done it. You should especially follow some of the work on Huggingface. To replicate talking to your friend though, you will need a very large dataset to train off of I would think and its unclear to me if you can just fine-tune it or you would need to train a model from scratch. So a dataset with 10s of thousands of examples and then you need to train it on a GPU. https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...
- Kutsuya 3y agoThank you for sending this. It's still quite puzzling to me if it's actually possible or not. Maybe what I want to train is a style? But then again, it should also remember other important things related to the friend..
- sainez 3y agoParent comment is on the right track. It sounds like you want to fine tune an llm to mimic the conversation style between you and your friend. Then you can use a general embedding model to implement RAG so that the application can "recall" pieces of your conversation.
- deleted 3y ago[deleted]
- itronitron 3y agoIt's weird to think there are entire companies built around providing access to a pre-computed vector space model.
- pietz 3y agoI'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger documents into BGE for example. Furthermore, I think that most (all?) benchmarks in the MTEB leaderboard deal with very small documents. So there is nothing here that validates how well this model does on larger documents. If anything, I'd pick a higher ranking model because I put little trust in one that only ranks 17th on small documents. Should I expect it to magically get better when the documents get larger? Plus, you can expect that this model was designed to perform well on the datasets in MTEB while the OpenAI model probably wasn't. Many also stated that a 8k context embeddings will not be very useful in list situations. When would anyone use this model?
- infecto 3y agoI have been trying to understand the hype as well. Happy to see all the work happening in this space still. I was pretty curious about the context limit. I am not an expert in this area but I always thought the biggest problem was the length of your original text. So typically you might only encode a sentence or a selection of sentences. You could always stuff more in but they you are potentially losing the specificity, I would think that is a function of the dimensionality. This model is 768, are they saying I can stuff 8k tokens worth of text and can utilize it just as well as I have with other models on a per 1-3 sentence level?
- infecto 3y agoThinking about it some more as I read through more comments. I guess in the stated case of research papers it can make sense if your task is looking for the common themes and not specific details. If you are embedding a sentence or a paragraph you miss out on the connection between those sentences across the whole paper...or at least its harder to manage that. By encoding a large number of pages from the paper (or the entire paper) you can hopefully do a better job of capturing the theme of that paper. This also opens up another question though, how would that compare to using a LLM to summarize that paper and then just embed on top of that summary.
- nwhnwh 3y agoWhat does this even do?
- tyingq 3y agoSee this story from yesterday: https://news.ycombinator.com/item?id=37985489 https://news.ycombinator.com/item?id=37985489
- woofwoofwoof 3y agoJust noticed that they (jina.ai) have offices both in Berlin and China. I am wondering how they will they operate with the presence of chip export restrictions and other side effects of USA / China tensions.
- urduntupu 3y ago[dead]
- do-me 3y agoJust quantized the models for onnx usage in e.g. transformers.js and got 4x reduced file size: - 𝟐𝟖.𝟓 𝐌𝐁 jina-embeddings-v2-small-en (https://huggingface.co/do-me/jina-embeddings-v2-small-en https://huggingface.co/do-me/jina-embeddings-v2-small-en) - 𝟏𝟎𝟗 𝐌𝐁 jina-embeddings-v2-base-en (https://huggingface.co/do-me/jina-embeddings-v2-base-en https://huggingface.co/do-me/jina-embeddings-v2-base-en) However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small version seems to be unaffected. Might this be some kind of side effect due to the way they deal with large contexts? If you want to test, you can head over to SemanticFinder (https://do-me.github.io/SemanticFinder/ https://do-me.github.io/SemanticFinder/), go to advanced settings, choose the Jina AI base model (at the very bottom) and run with "Find". You'll see that all other models perform just fine and find "food"-related chunks but the base version doesn't.
- Havoc 3y agoWhy quantize something that is already very small (270mb)?
- pietz 3y agoJust making up stuff here, but smaller models are great for serverless compute like functions, which would also benefit from lighter computation. Don't forget, some people are dealing with hundreds of millions of documents. Accelerating this by 4x may be worth a small performance hit.
- luke-stanley 3y agoWhen I go to this leaderboard: https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard I click on the "Classification" tab, then I see "jina-embeddings-v2-base-en" at number 12, with an average score of 73.45. But the highest scoring model there is llmrails/ember-v1 with 75.99 average score but it only supports 512 tokens, so if you need 8K tokens to be embedded, I guess they are the best. Do people need 8K of tokens for embedding? Maybe not but they might need more than 512 often enough. It could save a summary extraction step.
- cztomsik 3y agoSmall context window means you cannot embed the whole document, you are embedding just a part. So, if there is some information at the bottom which is dependent on something which is at the top, your embedding could be entirely wrong.
- egorfine 3y agoOne thing that is missing in comparison: OpenAI's model is multilingual. And not only it supports and embeds a variety of languages, it also computes the same coordinates for the same semantics in different languages. I.e. if you embed "russia is a terrorist state" and "россия - страна-террорист", both of these embeddings will have almost the same coordinates.
- m3kw9 3y agoI don’t really know what that means but it seems useful
- jimmySixDOF 3y agoI heard one of the developers on a regular Open Source DIY AI X/twitter space [1] & they are targeting two new models German/English and French/English for the next release https://x.com/thursdai_pod https://x.com/thursdai_pod