rnosov

5 ms·

1.
by rnosov 3y ago
The report cites both GPT-3.5 and GPT-4 scores on page 7 [1]. I've checked the numbers and they compare FreeWilly2 to GPT-3.5. For example, HellaSwag score of 85.5% corresponds to GPT-3.5. [1] https://arxiv.org/pdf&#x2F
2.
by rnosov 3y ago
> we are training a full suite of models, with the first becoming available in the coming weeks. Sounds like they already have the compute and began training.
3.
by rnosov 3y ago
For example, you can look up semantically relevant ( to a user query ) paragraphs from some internal document. Then, include them in the LLM context so it would know how to answer the query. Basically, that's the idea behind many ChatG
4.
by rnosov 3y ago
row 7 is the name of the dataset, you might need to load it yourself
5.
by rnosov 3y ago
I'm going through the dataset with your datasette tool and it looks like it might be a good idea to clean things up a bit. There are many duplicates[1], creepypastas[2] and other strange things in there. [1] https://lite.dat
6.
by rnosov 3y ago
They are leveraging Apple’s Metal Performance Shaders[1] not the neural engine. From the chart, it looks like you might get ~20x max boost on inference over plain CPU. Obviously, it's not like having RTX 4090 but better than nothing. [
7.
by rnosov 3y ago
It looks like you're on python 3.11 which has some issues with Pytorch. Downgrade to python 3.10 and try running it again.
8.
by rnosov 3y ago
A question to the author. Can you perform an ablation study with respect to the chunks? In other words, if you put in the context irrelevant/random chunks from the document would the quality of answers decrease/stay similar? Poten
9.
by rnosov 3y ago
1) I've looked at both codebases and this one is definitely a derivative of the nanoGPT. You can compare all three implementations yourself as they are actually surprisingly compact and readable. 2) The issue whether weights are copyri
10.
by rnosov 3y ago
I might be missing something but it looks to me that actually running this "open" model requires special hardware only accessible with a cloud subscription with 60 000 USD / week minimum spend[1]. Can anyone confirm if you ca
11.
by rnosov 3y ago
The FTC link has an example of the only newspaper in town refusing to deal with customers who are also running ads on a radio station. Do you think if the newspaper dressed such refusal as a ToS violation it would fly with FTC? Google might
12.
by rnosov 3y ago
That would be anticompetitive practice that is actually against the law in many countries[1]. In the unlikely event of OpenAI ever engaging in such things they will be sued into oblivion. [1] https://en.wikipedia.org/wiki&#x
13.
by rnosov 3y ago
ToS are not the law. It would be similar to your power company claiming copyright over the code written using "their" electricity. Not going to happen. I wouldn't be too concerned.
14.
by rnosov 3y ago
They are instruction tuning it using the dataset released by stanford-alpaca team. The dataset itself is synthetic (created using GPT-3) and somewhat noisy and in my view can be easily recreated if OpenAI ever tries to go after it (which is
15.
by rnosov 3y ago
This has nothing to do with facebook. The foundational model here is GPT-J which is opensource and safe to use. Sadly, it is inferior to state-of-the-art models such as LLaMA.
16.
by rnosov 3y ago
This is a really troubling development. Anyway, kudos to you for standing up to the man.
17.
by rnosov 3y ago
There are benchmarks in the original LLaMA paper[1]. Specifically, on page 4 LLaMA 13B seems to beat GPT-3 in BoolQ, HellaSwag, WinoGrande, ARC-e and ARC-c benchmarks (not by much though). Examples that you've seen are likely to be bas
18.
by rnosov 3y ago
You describe supervisor approach as: > One common suggestion is to have another LLM look at the input intently with the instruction to determine whether it is malicious. Preflight prompt check is actually opposite of that in a sense that
19.
by rnosov 3y ago
Quite an interesting article. The Vice example is hilarious. But for all doom and gloom you haven't addressed the most obvious mitigation - Preflight Prompt Check [1]. It would be trivial to detect toxic prompts and halt further inject
20.
by rnosov 3y ago
There is a big AI division of Google located in the UK (DeepMind). Could be related to that.
21.
by rnosov 3y ago
> https://arxiv.org/abs/2210.17323 I've read the paper and to be honest I'm not sure what to make of it. Their headline benchmark is perplexity on WikiText2 which would not be particularly relevant to most
22.
by rnosov 4y ago
I don't think that the model they are using is private. You can download it[1] and run yourself. Looks like it is GPT-NeoX which was fine tuned using the open source LAION OIG dataset[2]. [1] https://huggingface.co/toge
23.
by rnosov 4y ago
You might want to tune the sampler. For example, set it to a lower temperature. Also, the 4bit RTN quantisation seems to be messing up the model. Perhaps, the GPTQ quantisation will be much better.
24.
by rnosov 4y ago
The post has a workaround for the PyTorch issue with Python 3.11. If you follow the repo instructions it will give you some rather strange looking errors.
25.
by rnosov 4y ago
Seconded. Do write it up. I see vast.ai listing interruptible instance with a single A100 80GB at $1/hour which is pretty reasonable. ChatGPT plus is $20/month which would be roughly 20 hours of use and I wont't be lectured l
26.
by rnosov 4y ago
Hmmm, the Github repo suggests that you might be able to run the 65B model on a single A100 80gb card. At the moment, the spot price on Google cloud for this card is $1.25/hour which makes it not so crazy expensive...
27.
by rnosov 4y ago
You can read the original LLaMA paper which is pretty accessible[1]. For example, they claim to outperform GPT-3 on HellaSwag benchmark ( finishing sentences ). You can find examples of unfinished sentences in the HellaSwag paper [2] on pag
28.
by rnosov 4y ago
They forgot trailing zero. It should be 130 sq ft[1]. Looks like no window too. [1] https://www.lifeatseacruises.com/pricing-and-cabins
29.
by rnosov 4y ago
There are plenty of places on dry land that are a LOT cheaper than 30k/year. Probably would be more comfortable too.
30.
by rnosov 4y ago
I'm not entirely sure what sort of courses you are planning to offer, but what I can tell you is that course material quality at traditional higher education providers (bottom tier) is appalling (and they know it). If you can get it ri

More ›