juliensalinas

6 ms·

1.
by juliensalinas 1y ago
Relying on GenAI for support like that without a human in the loop is a huge mistake...
2.
by juliensalinas 1y ago
You can read the full comparison here: https://nlpcloud.com/genai-inference-engines-tensorrt-llm-vs...
3.
Comparing GenAI Inference Engines: TensorRT-LLM, VLLM, HF TGI, and LMDeploy
1 points by juliensalinas 1y ago|1 comments
4.
by juliensalinas 1y ago
Sounds very cool. I'm curious how you manage to monitor Linkedin though. The only tool that seems capable of monitoring Linkedin is https://kwatch.io , so if you manage to achieve that too it's impressive.
5.
by juliensalinas 2y ago
Social listening on HN, Reddit, X... I used https://kwatch.io and jumped into the relevant conversations to mention my product.
6.
by juliensalinas 2y ago
I use KWatch.io ( https://kwatch.io ) for social listening and it works very well for HN monitoring in my case. They also support other platforms (Reddit, Linkedin, Twitter..). But they don't propose advanced features like da
7.
by juliensalinas 2y ago
Many are trying to install and deploy their own LLaMA 3 model, so here is a tutorial I just made showing how to deploy LLaMA 3 on an AWS EC2 instance: https://nlpcloud.com/how-to-install-and-deploy-llama-3-into-... Deployin
8.
by juliensalinas 3y ago
How does this solution compare to platforms like https://kwatch.io or brand24 for hacker news monitoring? Does it monitor hacker news in real time?
9.
by juliensalinas 3y ago
We tried hard to move some of our inference workloads to TPUs at NLP Cloud, but finally gave up (at least for the moment) basically for the reasons you mention. We now only perform our fine-tunings on TPUs using JAX (see https://
10.
by juliensalinas 3y ago
Claude (Anthropic) might be the closest direct alternative to ChatGPT (but it's not available in alls countries). You might also want to try ChatDolphin by NLP Cloud (a company I created 3 years ago as an OpenAI alternative): https:&#
11.
by juliensalinas 3y ago
For those who want to try Mistral 7b, here is a video that shows how to do it on an A10 GPU on AWS: https://www.youtube.com/watch?v=88ByWjM-KGM
12.
by juliensalinas 3y ago
Thank you.
13.
by juliensalinas 3y ago
Thank you for the update! Do you happen to know if there are quality comparisons somewhere, between llama.cpp and exllama? Also, in terms of VRAM consumption, are they equivalent?
14.
by juliensalinas 3y ago
Oh it seems you're right, I had missed that. As far as I can see llama.cpp with CUDA is still a bit slower than ExLLaMA but I never had the chance to do the comparison by myself, and maybe it will change soon as these projects are evol
15.
by juliensalinas 3y ago
LLaMA 30B or 60B can be very impressive when correctly prompted. Deploying the 60B version is a challenge though and you might need to apply 4-bit quantization with something like https://github.com/PanQiWei/AutoGPTQ o
16.
by juliensalinas 3y ago
llama.cpp focuses on optimizing inference on a CPU, while exllama is for inference on a GPU.
17.
by juliensalinas 3y ago
Totally agree. Actually a couple of months ago Sam Altman even admitted that they had a very hard time doing proper "engineering" (meaning that they had the right team to create a very good LLM but not the right team to production
18.
by juliensalinas 3y ago
NLP Cloud (especially the Dolphin and Fine-tuned GPT-NeoX 20B models)
19.
Correctly using foundational AI models and instruct AI models
4 points by juliensalinas 3y ago|0 comments
20.
by juliensalinas 3y ago
You might want to try our ChatDolphin model on NLP Cloud that is very similar to Vicuna and uncensored: https://nlpcloud.com/home/playground/text-generation (select the ChatDolphin model at the top right). I hope
21.
by juliensalinas 3y ago
As the founder of NLP Cloud ( https://nlpcloud.com ) I can only guess how costly it must be for OpenAI to maintain several versions of GPT-4 in parallel. I think that the main reason why they don't provide you with a way to p
22.
by juliensalinas 3y ago
You might want to try NLP Cloud: https://nlpcloud.com
23.
by juliensalinas 4y ago
Some alternatives are explained in this article: https://nlpcloud.com/chatgpt-open-source-alternatives.html But it will require some prompt engineering in order to get the same level of instruction as ChatGPT.
24.
by juliensalinas 4y ago
Smaller models like Curie can work quite well too. But they are less "instruct-like" models so you will need to properly use few-shot learning (aka "prompt engineering") in order to get good results: https://n
25.
by juliensalinas 4y ago
You might want to have a look at this article that mentions a couple of open-source alternatives: https://nlpcloud.com/chatgpt-open-source-alternatives.html None of them are easy to self-host though...
26.
by juliensalinas 4y ago
The best open-source alternatives you can find today are GPT-NeoX 20B, GPT-J, Bloom, and OPT. But these are all generative models à la GPT-3. In order to turn them into a chatbot you will need to use few-shot learning: https://nl
27.
by juliensalinas 4y ago
You could follow EleutherAI's official guide: https://github.com/EleutherAI/gpt-neox You could also use a hosted service that proposes GPT-NeoX like https://nlpcloud.com or https://goose.ai
28.
by juliensalinas 4y ago
I worked on the Stable Diffusion and GPT-J integrations on NLP Cloud ( https://nlpcloud.com/ ). Both can be used in FP16 without any noticeable quality drop (in my opinion). Stable diffusion requires 7GB of VRAM on a Tesla T4
29.
by juliensalinas 4y ago
I've been testing BLOOM for a while and it seems it is working very well with good few-shot learning. See this post about prompt examples: https://nlpcloud.com/effectively-using-gpt-j-gpt-neo-gpt-3-a... All of these ex
30.
by juliensalinas 4y ago
For the moment I can't use it. I'm getting the following error: <html> <head><title>403 Forbidden</title></head> <body> <center><h1>403 Forbidden</h1>&l

More ›